RFC: Dependency Manager Install Location

[mfts]

Motivation

One of the core benefits of tea: it's relocatable.
tea does not pollute your system environment, everything gets "installed" in a relocatable folder ~/.tea.

tea can also install languages like node and ruby and it's dependency managers, npm and gem, respectively.
However, these language dependency managers install packages to global directories like ~/.npm and ~/.gem.
Tools from these ecosystems expect to find packages in these locations.

In this RFC, we would love to hear feedback from the community.

Proposal

There are a couple of options that we are weighing:

1. dependency managers' default location, i.e. ~/.gem
2. general location, i.e. ~/.local/bin
3. inside tea's directory, i.e. ~/.tea/foo

Option №1 Dependency manager's default location

Languages and their dependency managers have set default locations for their package installations. Tea should not change how or where gem installs Ruby gems. Tea packages ruby and gem and leaves the rest to them.

• For instance, if we install Ruby with tea, it will be installed here: ~/.tea/ruby-lang.org/v3.2.1/bin
• Ruby will install gems to ~/.gem

Option №2 General location

Tea is a package manager that cares about developer experience and by following that mantra, a general location, which is already in the user's $PATH should be picked as a logical install location.

• For instance, if we install Ruby with tea, it will be installed here: ~/.tea/ruby-lang.org/v3.2.1/bin.
• Ruby will install gems to ~/.local/bin or /usr/local/bin or ~/.tea/bin.

Option №3 A tea location

Tea is relocatable and stows packages in a "sandbox". If you want to remove a particular package from your tea installation, you simply remove the package folder: rm -r ~/.tea/foo.org

• For instance, if we install Ruby with tea, it will be installed here: ~/.tea/ruby-lang.org/v3.2.1/bin
• Ruby will install gems to ~/.tea/ruby-lang.org/v3.2.1/bin

The benefit: if I want to remove a particular version of Ruby, along with all installed gems, I simply remove ~/.tea/ruby-lang.org/v3.2.1.

[davdroman]

Great question! It's been in the back of my mind as of late.

My take is that 3 is the most logical one and aligns with tea's goal of full portability without muddying the developer environment with randomly versioned gem binaries.

However I'd only prefer it to option 2 if it allows the user to install and execute gems by Ruby version:

$ tea +ruby-lang.org^3 gem install cocoapods
$ tea +ruby-lang.org^3 pod install

Without support for that, needing to add specific gem bins to PATH defeats said portability goal IMO.

As an aside, +ruby-lang.org^3 feels quite long. I wish tea had common aliases like +ruby^3. Maybe this is already planned? 🤞

[jhheider]

If you have tea magic installed, then tea +ruby-lang.org^3 ruby ... is equivalent to ruby^3 .... For the above scenario of running gem with a specific ruby version, hopefully this is in a project that can define its specific tool version requirements in one of the detectible configuration files (README.md, package.json, etc.) to spare your typing it each time.

But if none of the above applies, I agree the command is verbose. ;)

[magnusviri]

Here are my 2 cents.

There are 3 use cases.

Developer experience. I've used Python and a little Node. What I've noticed with Python is that Python libraries and projects are so dependent on the version of python, that it all is bundled together, ala Option 3. Node installs the libraries in each project directory but installs Node so that it's in the path. I prefer the python way because if I'm working on 2 different Node projects requiring different versions of Node, I have to use nvm to switch, which is a pain.

My experience distributing a Python project is not good because it really needs to be in a virtual environment so that users can have different versions of Python. I think this is really a problem with Python because Python changes fast. I think tea can really solve this problem.

I have a lot of software deployment experience and it's always been intermixed with the OS (like yum/apt) or Brew's approach. This is a horrible developer experience but a great user experience. I'm hoping that tea will do option 3 but with the ability to add symlinks to any location in the PATH.

Other languages may work better with different options. So I can only speak about what I know.

[DitchComfort]

Yeah not an easy question. I fully understand the benefits of option 3. Especially for developers etc. However, if you’re not an actual developer but just someone who likes to play around and/or improve your machine, and perhaps fiddling around with some tools… I would absolutely like to have a more brew-like experience when installing tools (or at least an option/flag). Just one example, I usually want to install coreutils and other GNU tools on my Mac, Homebrew have the option to add to (man)path or prefix with a letter g. With tea I need to manually symlink every tool. And I don’t get the right manpath. Or create a bunch of unnecessary aliases. The same goes for several other tools as well. So I always end up using brew, just one simple command and everything just works.
I think if more casual people are going to use tea, there has to be an easier solution or a middle ground. Because I don’t really care about the dev side of things, I just want everything to work when I install something (yes magic is cool…)

[mxcl]

This is about how tools like gem install their packages.

You won’t have to manually symlink things forever, we promise. We’re exploring how we can avoid “installations” as part of what makes tea unique and powerful. But eventually once we think we've done all we can we'll add an install command.

[mxcl]

Thanks for the feedback everyone. I think the RFC is overly complicated, it is in fact a decision between two different approaches:

1. Tools like gem install into ~/.tea
2. They install outside ~/.tea to their default locations

With a secondary consideration for both of where should the binfiles go? eg. if I gem install cowsay does cowsay end up as ~/.tea/ruby-lang.org/gems/bin/cowsay, /usr/local/bin/cowsay or ~/.local/bin/cowsay.

[mxcl]

Really I think it's even simpler than this. Should tea be the only tool to touch ~/.tea and I think the answer is quite simple: yes.

[magnusviri]

I had another idea that relates to this. I was thinking of how to install the stable diffusion apps and they download the same model files that are over 4 gigs. The best way to manage the models is to put them in a global location.

Second, they also save the output images inside of the folder that contains the code, which to me is... an anti-pattern (probably not the right word, but you get the idea). I am pretty sure InvokeAI and Automatic1111 can be configured, so they'd have to be configured. But then what do we configure the model and output directories to be?

[jhheider]

The contents of ~/.tea should be stable IMO. Other tools should not mess with it.

This is a compelling argument against re-sandboxing other package managers.

[magnusviri]

Yeah I think it makes the most sense that only tea modifies ~/.tea. I think I said previously that I wanted option 3 but I don't. I think it's option 1.

also, pip installs to where ever Python is by default. I don't know enough about Python to know if that can be changed. Except pipenv seems to change it.

[mxcl]

I think we're all confused here. pip only installs to one location (python prefix I believe). But if it's in a virtual env then that's the location.

Like I may be wrong, I'm still deep in pip/python shit.

I'm not sure how pipenv works.

[magnusviri]

pipenv just automates the creation and activation of python virtual environments. It also creates a Pipfile, which is similar to Node.js package.json files. It looks like it does a ton of stuff and that everything is configurable. In my experience, by default it installs to ~/.local/share/virtualenvs/. I'm considering using PIPENV_VENV_IN_PROJECT to put it in the installed stuff in the same directory as the Pipfile, making it even more like npm.

[magnusviri]

I think we're all confused here. pip only installs to one location (python prefix I believe). But if it's in a virtual env then that's the location.

And yes, you're correct about the above.

[mxcl]

I’m formulating the following idea:

• Default to ~/.tea/local when running as a standard user-id
  • so pip install foo will go to ~/.tea/local/lib/python/etc
• Default to /usr/local when running as superuser-id
• Patch all tools to adhere to https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
  • eg $XDG_DATA_HOME
• Have magic set the XDG_ variables
  • plan to have a ~/.tea/config.toml at some point that allows overriding how magic is set up

[abitrolly]

There should an option to have a custom location for dependency cache, and project local path for specific dependency graph needed for that project.

For example, the global cache for Gitpod needs to be in /workspace/.tea to survive restarts, and for project specific dependencies tea should be able to somehow understand what the project needs from files like .tool-versions.

[mxcl]

In v1 TEA_DIR is the cache location (following deno’s lead with DENO_DIR and presumably they followed someone else’s lead)

.tool-versions

We should probs support that. I wonder if we should do it by installing asdf though. Probs. It's what people would expect.

[abitrolly]

No asdf, plz. The reason I am using tea is because asdf plugins install PostgreSQL only from source which takes ages.

[mkhoeini]

I love tea. Thanks for all your efforts. Just to share my 2 cents on this:

I used to have asdf and now I'm trying to get rid of it using tea. There are a couple more dependencies that are missing in tea that If I find time to make a contribution to have them in tea I will succeed. :)

By this point, I think there is a thing called package manager fatigue. asdf actually helped with it to some degree by having a lot of dependencies. My hope is that tea at some point will be as versatile as homebrew and as useful for development as asdf. I don't wish tea to manage asdf for me. And having homebrew as the recommended way to install tea is also odd to me. My wish is to see tea replace all of them :D

Nix package manager goes a step further and tries to subsume even npm and pip. The sentiment is clear to me. We want one package manager that works correctly for everything. But tea's promise is even greater: we will solve the problem of software distribution for you and through it, we will be able to keep track of dependency graphs in the open-source community and compensate real maintainers of important packages. How would you be able to do it if you delegate work to asdf or npm or anything else?

I understand that replacing everything is probably unrealistic. But my intuition is that tea should at least replace asdf and homebrew. I might be wrong though :)