Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add hugo vendor and resources.Vendor #13309

Closed
bep opened this issue Jan 25, 2025 · 4 comments · May be fixed by #13310
Closed

Add hugo vendor and resources.Vendor #13309

bep opened this issue Jan 25, 2025 · 4 comments · May be fixed by #13310
Labels
Milestone

Comments

@bep
Copy link
Member

bep commented Jan 25, 2025

Note

This proposal is a bit half-baked. I've certainly thought about it a lot over the years, and a concrete implementation popped into my head while sleeping last night, so I thought I'd get it down on paper while it's fresh.

I have spent an excessive amount of time trying to avoid npm as much as possible in Hugo's assets building. However, in some cases, it's hard to avoid. TailwindCSS just released their v4 version, and it's a great piece of software that works well with Hugo.

Unfortunately, this requires npm to be practical.1

We have a new theme for gohugo.io built with TailwindCSS v4 on its way. Unfortunately, currently, you need to do this the first time you're running the site:

npm i
hugo server

This is not ideal if you just want to fix some typos, and it's not great when we include the Hugo documentation in the source distribution (inside /docs in the main repo). I'm betting that people can build the v0.141.0 version of the documentation fairly easily in 20 years. With an npm build step in the mix, all bets are off.

I have thought about ways to fix this, including testing out my own npm Go Module proxy. Below is my latest take on how to solve this in a general way, which should also work for e.g., Dart Sass.

Using TailwindCSS as an example, this is how I imagine it would look in the templates:

{{ with resources.Get "css/styles.css" }}
{{ $opts := dict
   "inlineImports" true
   "optimize" (not hugo.IsDevelopment)
}}
{{ with . | css.TailwindCSS $opts | resources.Vendor }}
 // minify/fingerprint if not dev, insert link
{{ end }}
{{ end }}

resources.Vendor may take options in the future, but for now, I see it as a simple marker that tells Hugo to vendor/use the vendored version of the previous transformation using a shallow key of that transformation's:

  • Transformation key/name
  • Input Resource's Path and Content
  • Options

This is more or less how we do it today for the file cache inside /resources.

We already have a hugo mod vendor command that writes to _vendor. I suggest that we move those down to _vendor/mod so we can add another _vendor/resources root folder for these.

If we then consolidate hugo mod vendor into a new top-level command, we could make it look like:

hugo vendor // all
hugo vendor -mod
hugo vendor -resources

Running hugo vendor -resources will:

  1. Remove _vendor/resources.
  2. Rebuild all vendored assets (transformations tagged with resources.Vendor).

When running hugo or hugo server, we will by default use vendored assets/modules. To enable development, we need to rethink the current --ignoreVendorPaths flag and module config a little.

The above should obviously also work for themes/Hugo Modules. In fact, given how our current vendoring works, it will be great to finally be able to use themes/modules with complex build setups without having anything but hugo installed.

Footnotes

  1. It is possible to download the TailwindCSS CLI binary and put it in the PATH for a npm-less setup (and it may also be worth looking into WebAssembly when we go further down that route), but you quickly find the need for npm once you want to include the great Typography plugin. I guess it's also possible to use the build.useResourceCacheWhen=always, but that's only practical in very simple setups.

@bep bep added the Proposal label Jan 25, 2025
@bep bep added this to the v0.143.0 milestone Jan 25, 2025
@bep bep pinned this issue Jan 25, 2025
@jmooring
Copy link
Member

As far as I can tell from experimentation, the existing useResourceCacheWhen only helps with Sass transformations:

  • We don't cache Babel transformations
  • We don't cache Tailwind transformations
  • We throw a "You need to install PostCSS" error even though we cache PostCSS transformations

It sounds like the resources.Vendor approach would handle all of the above, storing the cached transformations in the _vendor/resources directory.

Questions:

  1. Would this mean that the current assets cache ([caches.assets]) is no longer required or used?
  2. Would we still use the useResourceCacheWhen setting to control when we use _vendor/resources?
  3. How will module and theme authors include transformed resources in source control? Today they can include the resources directory. Would they somehow include _vendor/resources instead?

@bep
Copy link
Member Author

bep commented Jan 26, 2025

@jmooring OK, so now I have slept a little on this. I haven't edited my text above, but I think it holds mostly water, but I think we'll:

  • Drop the resource.Vendor construct. It mostly adds confusion/complexity, I think. I think ...
  • We formalise the list of "vendorable resources" (Babel, Tailwind, PostCSS, js.Build, js.Batch and possibly Sass).
  • We create a better ignoreVendorPaths config setup/default where you can say e.g. ignore vendoring for resources/tailwindcss/**.. or something; and I'm guessing we can have it default off for, say, Sass.
  • [caches.assets] will still have its (important use) as a file cache (but maybe some of it will be adjusted, not used anymore). I'm probably abusing the term vendoring above, but it technically fits so nicely into what we have, so I think I'm going to ... live with it. A cache is something shorter lived and less formalised, and can be deleted at any time.
  • useResourceCacheWhen would (eventually) go away.

A note to self: I think we need to be a little lenient with the key used to vendor these resources, e.g. from the example above:

{{ $opts := dict
   "inlineImports" true
   "optimize" (not hugo.IsDevelopment)
}}

If I run

hugo vendor -e production
hugo server

I want the vendored resource when running the server.

EDIT IN: OK, looking at the assets cache definition now, I would say that that cache would go away after this.

bep added a commit to bep/hugo that referenced this issue Jan 26, 2025
@bep
Copy link
Member Author

bep commented Jan 26, 2025

There's a very rough implementation of this in #13310

  • Has a hugo vendor command.
  • Only works for TailwindCSS and js.Build, but I have managed to build a site with TailwindCSS app with typography plugin + a fair amount of JS libraries via npm ... without any node_modules folder, so I will call this promising.

bep added a commit to bep/hugo that referenced this issue Jan 27, 2025
bep added a commit to bep/hugo that referenced this issue Jan 28, 2025
@bep
Copy link
Member Author

bep commented Jan 29, 2025

I'm closing this for now. I will come up with something very similar, but I will outline that more precisely in a different issue.

@bep bep closed this as completed Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants