FEATURE: Support static web hosting

[smil2k]

When the maven repository format was introduced, there was a design goal, that maven repository could be hosted on a static hosting server.
I was wondering whether it would make sense to lower the hosting requirements of a OCI images by using the same principle.

This would allow skopeo users (mainly podman and oci-o) to lower the cost of hosting of OCI containers directly utilizing blobs like s3 or azure blob to store the images. The layout of the files could be similar to what local directories has, the only change would be to add transport mechanisms (wagons in maven lingo) to get them. Eg. s3 wagon, https or azure blob etc..

Supporting only download and upload is fairly trivial to implement.

Search is a better question. Typically blobs have directory listing features, but I would not mind not having it at all. All my use cases use absolute names.

What is your thoughts about this feature?

[mtrmac]

Thanks for reaching out.

Tools to build such web servers already exist, e.g. https://github.com/NicolasT/static-container-registry uses Skopeo underneath. So far that didn’t need specialized Skopeo support.

WRT search and the like, once contemplating adding server-side request handlers at all, I don’t know why one would stop at any particular point, and not build a full-featured registry server. It just seems simpler to me to deploy a dedicated full-featured registry server than try to … deploy a dedicated not-full-featured registry server; I can see no benefit to trying to be in the middle of the spectrum, it has the disadvantages of both extremes.

[smil2k]

Thank you for the quick answer.

The docker authorization and repo is a bit overcompkicated for my taste. Also it does not lend itself for hosting it as a blob. My aim is to have serverless repos, so I only have to pay for network and storage. No VMs.

This is why I was thinking in adding other protocols to read these "local files" besides fopen.

Regarding search: s3 and other object stores have search options, so it is possible to implement them in the client if necessary. This could allow supporting podman search eventually.

I would be happy to contribute as a pull request if it is welcome.

[mtrmac]

The docker authorization and repo is a bit overcompkicated for my taste.

That seems simply incompatible with pure static hosting: The code needs to return a properly-formatted 401 without credentials, and a 200 with content otherwise. That would strongly rely on the capabilities of the static web server.

The linked static server project, similarly, spends bulk of the code on specifically interfacing with Nginx — more than doing anything that specialized about container images.

It’s just that, correspondingly, bulk of the CI and infrastructure would need to be focused on specifically interfacing with that web server, and testing that interoperability.

Clearly that’s software that can be written, but there’s ~0 overlap with the existing Skopeo CI environment and requirements, and little overlap with the feature set, so I don’t see much benefit for embedding any specific web server knowledge into Skopeo itself, or into the Skopeo repo.

[smil2k]

I think I was not clear enough about my intentions. Maybe this stems from my lack of knowledge on skopeo.

My understanding is:

• Skopeo is the library which is used by podman and co to download images. (copy images from remote repository to local repo).
• Skopeo is a command line tool to use the library as a cli.

Currently the library supports remote repository format Docker v2 (OCI remote), and local image stores container/storage, docker, oci and "non-standard" dir.

My suggestion would be to enable a new remote repository format option: remote "OCI layout" repository. The OCI directory format has an index.json which contains the tags and images available (it is possible to use for search) and it defines a file structure which can be used as is in a static web hosting environment.

So the only addition would be adding a transport medium for OCI local repository layouts and omit all the "401" per directory complexities docker registries do. This is the thing I referred as "wagons" in maven world. Transport mechanisms to download and upload to remote oci layouts.

Making a generic read only repository is easy as using only GET requests and search using the index.json. (+basic auth)

Adding push support is a bit more challenging because of the index.json describes everything: it needs to be downloaded and reuploaded. S3 and google offer ETag based optimistic locking, azure blobs provides file locking to solve concurrency.

This type of remote can be placed under any directory, so it is possible to reduce size of index.json and the number of write collisions to the minimum by placing each image under a separated directory.

[mtrmac]

It isn’t that simple, because the registry is not that trivial a static host; the protocol requires it to set various HTTTP headers in data responses. Content-Type at least (more specific than just “JSON”), and perhaps Docker-Content-Digest.

So, the writer would need to be HTTP-server-specific.