Using AI to create package.yml files

[magnusviri]

I've been studying AI and I wondered if it would be possible to fine tune a large language model with the differences between the existing tea package.yml files with the equivalent Homebrew, yum, apt, and Chocolately formula. Then feed the model all of the formula that it doesn't already have to produce package.yml files. My AI knowledge is superficial, so I have no idea how to do this, but I'm going to start learning, just in case someone else way smarter than me doesn't just decide to do it over the weekend or something like that.

[mxcl]

I’m sure it’s possible.

However I think we could get better results from just implementing a ruby script to translate Homebrew pkgs. Though the AI approach is potentially WAY LESS work.

Longer term we want our package.yml format to be something pkg authors include in their repos so this effort stops getting duplicated.

[davdroman]

Longer term we want our package.yml format to be something pkg authors include in their repos so this effort stops getting duplicated.

I was going to make a joke suggesting a random standard name "OpenPKG" à la OpenAPI, but then I looked it up and it's already a thing! http://www.openpkg.org/

[adriens]

I think I'll makea feature request in that sens to jreleaser for the java ecosystem.

Also, being able to link to sdkman or jbang script could be very useful to add a very larje jvm ecosystem, see

• ➕ JBANG! pantry#810
• ➕ sdkman pantry#809
• [packagers] Support tea.xyz as package manager jreleaser/jreleaser#1280

[magnusviri]

I've thought of writing a formula parser also. In fact, I kind of started one but Ruby is towards the bottom of my experience list (I learned it but then never used it...). So I wrote a parser in Perl but didn't get very far.

I sent an email to surgehq.ai and told them what I wanted to do and asked for a quote. I had Marc's email so I CC'ed him. I'm curious what they'll quote me. I know they have free tiers, so yeah, it will be interesting to see what they say. I almost want to do this more because I'm dying to know if it will work.

[magnusviri]

SurgeHQ got back to me and said they are looking for annual customers. So basically "no".

I just applied to the GPT-4 waitlist. With 32,000 context tokens, I bet it can do this. I'm going to start cleaning up the brew files and removing stuff that doesn't really convert over (like bottle hashes).

Also, FYI, I'm planning on taking some extended "vacation" from my day job in April to work on this. I'm hoping to also package up stable-diffusion and llama on that same vacation.

[mfts]

Did you see the stable diffusion webui script that Max put together in the readme

$ git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui
$ tea \
  --cd stable-diffusion-webui \
  +python.org~3.10 +pip.pypa.io +gnu.org/wget +protobuf.dev +rust-lang.org \
  ./webui.sh

[magnusviri]

No! I'll have to check it out!

[mfts]

I just fed chat.openai.com with GPT-4 the example package.yml from brewkit and asked it to give me the instructions for libx11

It's of course not perfect yet, but with clearer instructions, it might get there:

distributable:
  url: https://www.x.org/releases/individual/lib/libX11-{{version}}.tar.gz
  strip-components: 1

versions:
  github: freedesktop/xorg-libX11/tags

strip: /^v/

dependencies:
  tea.xyz/xorg/proto/xproto: ^7.0.31
  tea.xyz/xorg/proto/xextproto: ^7.3.0
  tea.xyz/xorg/proto/kbproto: ^1.0.7

build:
  dependencies:
    tea.xyz/gx/cc: c99
    tea.xyz/gx/make: '*'
    tea.xyz/xorg/util/macros: ^1.19.3
  script: |
    ./configure $ARGS
    make --jobs {{ hw.concurrency }} install
  env:
    ARGS:
      - --prefix="{{prefix}}"

provides:
  - lib/libX11.so
  - lib/libX11.a
  - include/X11

test:
  dependencies:
    tea.xyz/xorg/lib/libx11: '*'
  script:
    test "$(pkg-config --modversion x11)" = {{version}}

[magnusviri]

Are you kidding me? I am trying really hard to get my brain out of crisis mode and you're not helping.

Are you saying in a matter of minutes you were able to show that it will work? How do you have GPT-4 access? Are you a plus member?

[magnusviri]

And what exactly was your prompt?

[magnusviri]

@mfts I figured out how to get gpt-4 access (by paying/using my free trial). I wanted to try this out. I am thinking of trying this tomorrow. I was hoping you could send me the prompt you used.

[mfts]

Sorry I didn't see your replies earlier.

I have GPT-4 access via the chat plus subscription.

The prompt I used is fairly simple:

"""
You are PackageGPT, a large language model that understands software packages and how to build them from source.

Follow instructions.

---

Here's an example package.yaml file for the tea package manager:

---

Please create a package.yml file for the "package name" software package. The build script key should contain the instructions to build the package from source.
"""

I used something like this

[magnusviri]

Behold: https://github.com/magnusviri/convert2tea

I used covert2tea to create this: pkgxdev/pantry#890. I had to edit it by hand a little bit. Besides the fact that I spent almost all day writing convert2tea, it saved me a lot of mental thought. And it's AI coolness!

https://xkcd.com/1319/

Ugh, I did that without any breaks. I need a long break to process this new turn of events.

[jhheider]

Well, you know what must be done:

You must use convert2tea to package convert2tea for tea, then integrate it into teaxyz/brewkit:bin/pkg-init. (Don't forget to use the GitHub release function with a semver tag name to make your life easy.)

;)

Really cool work, @magnusviri. Love when a rabbit-hole comes together.

[mxcl]

wow. so in theory we could just fire this up and get it to iterate until the package.yml works?

[jhheider]

I do so like telling ChatGPT: "that didn't work, I still get ${x} error," and having it try again.

[mfts]

That’s super cool!

[magnusviri]

wow. so in theory we could just fire this up and get it to iterate until the package.yml works?

Hm. Ok, I'll work on that. I have ideas on how to get it to get more accurate if it doesn't work either, like using more example conversions, and even scraping the project build instructions or yum or apt packages as well. I could try different prompt templates.

I was also going to try running this locally against alpaca.

[adriens]

Does anybody has a GPT-4 access ?

[magnusviri]

@mfts has access. He tried it above.

[jhheider]

Paid ChatGPT access (20USD/mo) gets you up to 25 queries per day currently, subject to load.

[mxcl]

We have a company OpenAI API key

[adriens]

I'll give a try to https://github.com/magnusviri/convert2tea on :

• ➕ gomplate (go.mod) pantry#817

[magnusviri]

Well, I've populated a script with about 200 formula from the top 300 list and it's creating multiple versions of each package and saving them. They're all over the place. Some are ok, some are garbage, and some are "I'm sorry but as an AI language model..." There's no way I can actually run pkg build on any of these without cleaning them up.

I'm going to run it all night and see if I run out of trial money. I might just create a repo of the package files if people want to look at them. I don't have gpt-4 access, only gpt-3.5. I'm not sure if gpt-4 would do a better job. One thing I quickly noticed is that this thing can't go find github urls if the original recipe lists something else. So I think cleanup is unavoidable. At least a lot of the grunt work is done though. IDK, it seems to me that clean up is easier than staring at a template package.yml file.

On the bright side, I can rest assured that AI isn't going to replace me any time soon! LOL.

[adriens]

AMAZIN 💪

[magnusviri]

I've updated my repo with all the changes I've done the last day, and I've added the output, which includes 105 projects. It's kind of messy in those project folders. I've also been tweaking stuff as I go along, so later projects might look slightly differently than the first projects. I've submitted a pull request for the first one on that list, abseil.io, so I've removed that from the output (before I did the push).

[jhheider]

Can it package itself? :)

[magnusviri]

I'm actually asking gpt to convert from Homebrew to yml. I think I need a new template if I'm going to package from scratch. That is a different slant on what I'm making and it might even be more useful.

[magnusviri]

By "new template" I mean new prompt.

[magnusviri]

I've gone through the first round of converting about 240 of the top 300. I've only looked at the first one and it needed a lot of work. Anyway, if anyone wants to look, the output is here and there's 1255 files (yml files and I saved the prompts too).

[magnusviri]

Oh, and it only cost me $4 (I've still got $14 left from my trial lol).

[mxcl]

Maybe it's time to set AutoGPT on it?

https://twitter.com/rowancheung/status/1646389808401448960

[rhizanthemum]

Ha, I scrolled down to type "AutoGPT"

[rhizanthemum]

is there a specification for the yml or you're trying to get AI to figure out the spec?

[rhizanthemum]

nm, found the pantry. I must say, having to look up the domain name for the package seems like anti-magic

[mxcl]

@ghostboi-nft I agree to the extent that on my todo is “accept primary tool name instead of domain for +pkg.com” †

However the domain is a unique name and thus we are going to continue to insist on it in package.yml. There's millions of open source packages and there are thousands of naming conflicts. We need something unique.

It would be nice if there was a vscode extension to assist with this. In the mean time there is pkg search or you can search the pantry at https://tea.xyz

IMO the magic is strictly for the end-user. Packaging is frustrating and while I want it to be as nice as possible to get more contribution, in reality the most frustrating part is by far build-woe and not having to google a package to figure out its domain.

[rhizanthemum]

I'm not here to hate or argue or anything, but given the choice of brew install chruby and tea +?????????.???, I feel I'd prefer the former in most cases.

I get it's early in the project, but just my experience so far. I also understand and respect the fact that your knowledge is this area is vastly superior to mine, and brew is one of the most important and most influential/ubiquitous projects I've used throughout my career, so I'm just going to bow out and trust that you'll figure something out :)

Anywho! I was going to see if I could help with the conversion prompt or fine-tune for this. I have a feeling GPT-4 would be able to do this in one or two shots, and would probably even give us some code to use that we could run as a cronjob to keep things up to date.

...but it seems like you could just iterate through the formulae rb files in a ruby environment and you'd have access to all the information to dump the data to yaml and that would probably be easier than trying to fine-tune GPT-4. What am I missing?

[magnusviri]

I'm thinking. As I process the AI generated yml files, I've noticed something. The AI seems to hallucinate some ideas that might actually be a good idea. For example, here's one.

patches:
  - url: https://raw.githubusercontent.com/Homebrew/formula-patches/8059b3027a4aa68d8