Vote: Archive the artifacts repo

[jdolitsky]

The upcoming spec changes address the question of artifacts. The following repo served us well at the time, but now can only cause confusion for people new to OCI: https://github.com/opencontainers/artifacts

for more info see opencontainers/artifacts#63

I'll call the vote for @opencontainers/tob. Please approve, request changes, reply with LGTM, or not (and hopefully say why!).

• @sudo-bmitch
• @vbatts
• @jdolitsky
• @estesp
• @jonjohnsonjr
• @nishakm
• @samuelkarp
• @dmcgowan
• @cyphar

A 2/3 approval is required here, so 6/9 of the TOB members must approve.

[jdolitsky]

@opencontainers/tob PTAL

[sudo-bmitch]

LGTM with the assumption that we update the readme with a pointer to the image-spec guidance.

[samuelkarp]

Can we update the readme first? It'd be nice for the vote here to cover one action (archival), not multiple (archival + not-fully-specified readme updates).

[jdolitsky]

Just opened opencontainers/artifacts#67 to address the README

[SteveLasker]

I'm supportive of unifying the content across the specs, as this was the goal of image-spec/pull #770: Artifact spec additions and distribution-spec/pull #65, starting way back in March of 2019.

As a result, there was an OCI TOB Vote to create the artifacts repo, which included:

• Create a new artifacts repository, named github.com/opencontainers/artifacts
• Update the OCI distribution-spec to generically reference OCI manifest and OCI index, with OCI Image as one of many artifact types it can store.
• Update the OCI image-spec to reference images as an implementation of artifacts, using OCI manifest and OCI index

With broad adoption of OCI Artifacts and the evolution of artifacts/pull: OCI artifact manifest, Phase 1-Reference Types #29, we are getting closer to achieving the goal for leveraging the investments users, cloud providers and ISVs have in their registries. Enabling artifact authors to distribute new types, without having to create yet more package managers helps the community quickly innovate new ideas.

To complete the effort, and archive the artifact repo, the TOB may want to consider addressing the charter of OCI Artifacts:

The artifacts repository will serve 3 primary goals:

1. artifact authors - guidance for authoring new artifact types. Including a clearing house for well known artifact types.
2. registry operators and vendors - guidance for how operators and vendors can support new artifact types, including how they can opt-in or out of well known artifact types. Registry operators that already implement media-type filtering will not have to change. The artifact repo will provide context on how new media-types can be used, and how media-types can be associated with a type of artifact.
3. clearing house for well known artifacts - artifact authors can submit their artifact definitions, providing registry operators a list by which they can easily support.

The "clearing house for well-known types" evolved to Registering Unique Types with IANA, which alleviated OCI artifact maintainers from becoming the bottleneck.

The main value for OCI Artifacts became: Artifact Authors Guidance which helps authors and registry operators understand how to implement and support new artifact types. With the addition of the artifactType, subject property and /referrers api used across the specs, the info will need to be updated to reflect the new functionality.

Regarding the /artifacts repo being out of date, this was intentional as it was agreed to pause (artifacts PR #29) while the reference types made the circle from being implemented under the oras/artifact-spec, back to the OCI Working Group: Reference Types and now back to the Image and Distribution specs. The OCI Artifacts repo was a stability point while the formalization is being completed.

The guidance in OCI Artifacts is still accurate, as the guidance is both backwards and forward compatible. What's being added is formalization of the config.mediaType to artifactType and the additional reference type capabilities. I'm looking forward to the 1.1 release so the community and start to adopt the final version.

As incremental steps, before archiving the artifacts repo, I'd suggest:

1. Update the Image and Distribution specs charter to include guidance for authors and registry operators.
2. Once the image and artifacts repo finalize a 1.1 release, the artifacts repo can make updates to direct users to the new content.(see Artifacts pr #68) This would resolve the many links to OCI Artifact and its guidance spread across blogs and documentation.
3. Upon completion of the above steps, archive the repo.
   🎉

[jdolitsky]

@SteveLasker This sounds mainly good:

As incremental steps, before archiving the artifacts repo, I'd suggest:

1. Update the Image and Distribution specs charter to include guidance for authors and registry operators.
2. Once the image and artifacts repo finalize a 1.1 release, the artifacts repo can make updates to direct users to the new content.Update README.md artifacts#68 This would resolve the many links to OCI Artifact and its guidance spread across blogs and documentation.
3. Upon completion of the above steps, archive the repo.

Can we put together PRs to add an ARTIFACTS.md file in image-spec/distirbution-spec that captures all of this for number 1 on your list? We can keep the files in this repo to link back to, then just make the README similar to:

---

OCI Artifacts (archived)

This repo has been archived since July 2023.

The creation of this repository and subsequent efforts within OCI (i.e. Working Group: Reference Types (archived)) have resulted in the following pages which provide guidance for OCI Artifact Authors relating to implementing a single OCI specification:

• Supporting OCI Artifacts in image-spec (v1.1+)
• Supporting OCI Artifacts in distribution-spec (v1.1+)

---

??

[SteveLasker]

Once the 1.1 spec is released, there’s lots of good conversations.

[jdolitsky]

there's a good conversation right in front of you!

[SteveLasker]

Yup, and I'm looking forward to this making its completion. However, the churn that’s happened along the way has caused confusion, frustration, and randomization across cloud providers and implementers.
The artifacts repo has been a point of stability for how authors and registry operators can enable capabilities today, across all registries. And, a call for a Releases Please #23 demonstrated the need for a versioned, stable release.
I was more hopeful with the first RCs, but the late-breaking changes of Remove artifact manifest #999, after several cloud providers and projects implemented the RC, is the validated concern.
If the image and distribution spec had documentation that covered the charter of OCI Artifacts, specifically updated content of the Artifact Authors Guidance, that would be a great start to point people to.

[SteveLasker]

I'd also suggest the emojis don't exactly build trust around inclusive, positive collaboration.

[dlorenc]

This is basically the exact opposite of my interpretation of how the last few years have gone in the OCI.

Based on past experiences discussing things with you, I don't think we're going to have a productive discussion so I'm not going to continue but I felt it worthwhile to at least provide a counter interpretation on the history of OCI artifacts for future readers.

[sudo-bmitch]

The main value for OCI Artifacts became: Artifact Authors Guidance which helps authors and registry operators understand how to implement and support new artifact types. With the addition of the artifactType, subject property and /referrers api used across the specs, the info will need to be updated to reflect the new functionality.

Regarding the /artifacts repo being out of date, this was intentional as it was agreed to pause (artifacts PR #29) while the reference types made the circle from being implemented under the oras/artifact-spec, back to the OCI Working Group: Reference Types and now back to the Image and Distribution specs. The OCI Artifacts repo was a stability point while the formalization is being completed.

The guidance in OCI Artifacts is still accurate, as the guidance is both backwards and forward compatible. What's being added is formalization of the config.mediaType to artifactType and the additional reference type capabilities. I'm looking forward to the 1.1 release so the community and start to adopt the final version.

My concerns with the artifact repo are that it is out-of-date, which result in it not being forward compatible (tooling written to that repo's guidance will incorrectly process artifacts made according to the current image-spec guidance), and in at least one place I believe the advice conflicts with descriptor requirement in the 1.0 image-spec.

The "out of date" comes from the advice to use the manifest.config.mediaType as the artifactType. We have moved that to the secondary way to determine the artifactType, with the new artifactType field taking precedence when defined. This means that an artifact pushed with both a config.mediaType and an artifactType field defined to different values will be interpreted differently by tooling that follows the artifact repo vs the image-spec repo.

There is then a suggestion that a null config reference can be used, which I'm not entirely sure what that means. The config descriptor is required in the spec. And the config blob content must match the config descriptor media type. We have also found that some registries refuse a blob of 0 length.

As incremental steps, before archiving the artifacts repo, I'd suggest:

1. Update the Image and Distribution specs charter to include guidance for authors and registry operators.

I believe the specs themselves now includes guidance for authors and registry operators. Updating the charter for these specs can be brought up as a separate issue/PR to the TOB. I don't believe we need to wait for that change before deciding on this issue, since it is recommending we do something which we are already doing.

2. Once the image and artifacts repo finalize a 1.1 release, the artifacts repo can make updates to direct users to the new content.(see Artifacts pr #68) This would resolve the many links to OCI Artifact and its guidance spread across blogs and documentation.

Given that the artifacts repo conflicts with guidance in the image-spec, and the guidance in image-spec can be followed with existing 1.0 conformant registries, I feel the lesser of the bad options is to archive the artifact repo now rather than leaving up the conflicting guidance. In the scenarios where a dedicated config blob can be used, the guidance in the image-spec RC aligns with the artifact repo. And in the scenario where there is no config blob content, the image-spec RC advice conforms to the descriptor spec requirements where the artifact repo guidance does not.

[jdolitsky]

I feel the lesser of the bad options is to archive the artifact repo now rather than leaving up the conflicting guidance

+1, hence this vote

[mikebrow]

See proposed archive readme on the main branch ... https://github.com/opencontainers/artifacts/tree/main

[samuelkarp]

LGTM, thanks @mikebrow!

[SteveLasker]

As noted above, we're looking forward to Image and Distribution specs releasing, and adding content for Artifacts Authors and registry operators. We've even queued up content from the Artifacts repo to redirect folks. Although the link is fairly empty in comparison to the: Artifact Authors Guidance

My concerns with the artifact repo are that it is out-of-date, which result in it not being forward compatible
The "out of date" comes from the advice to use the manifest.config.mediaType as the artifactType. We have moved that to the secondary way to determine the artifactType, with the new artifactType field taking precedence when defined. This means that an artifact pushed with both a config.mediaType and an artifactType field defined to different values will be interpreted differently by tooling that follows the artifact repo vs the image-spec repo.

I'm 100% behind the artifactType type property, as the config.mediaType was always a workaround. However, it is the workaround that got the community to where it is today. To say that guidance wouldn't be supported would suggest we're not being backward compatible.

Are we really suggesting all the artifacts deployed today would break if they set the config.mediaType? Once the image and distribution specs are released, I would hope new tooling would set the artifactType, but until that release is done, I don't think we can really expect tooling to adopt those changes. And, to the point continually made, it will take time for those changes to be made, and deployed. This leaves the Artifacts repo, config.mediaType as the only released solution.

Which leaves the question, why is the TOB seemingly rushing to archive a repo that got the community where it is today? Seems we could focus on completing the docs and releasing the specs, so there's a natural and obvious transition point.

[jdolitsky]

Support for artifacts via config.mediaType is covered by the updated specs.

From distribution-spec:

If the artifactType is empty or missing in the image manifest, the value of artifactType MUST be set to the config descriptor mediaType value.

From image-spec

the config.mediaType value MUST be set to a value specific to the artifact type

Why exactly does it matter what got the community where it is today? These are important, widely-used specifications. The emphasis should be placed on functionality and clarity vs. the history and persons involved.

Compare this repo to the wg-reference-types repo which was created on Dec 15, 2021 and archived less than one year later on Sep 23, 2022. Nobody objected to archiving that when the time came... why should this repo be treated any different?

[jonjohnsonjr]

LGTM

[estesp]

LGTM, thanks

[SteveLasker]

@jdolitsky yes, the artifactType behavior is described in the, not yet released specs, And, several cloud providers and projects implemented the artifact manifest, released in rc2 which was thought to be the final RC. Then, was removed (PR #999) in rc3. Which makes it hard to simply suggest implementors should not differentiate between a project that's been stable and available for years, and new, developing guidance. Even though I really, really want to see it released.

As for treating working groups and projects differently, they are, as defined by the TOB working groups,

I'm really not understanding why there's such a rush to archive a stable project, while image and distribution complete their process, letting the natural flow occur.

[jonjohnsonjr]

There is no rush. We have been discussing it for a year.

[SteveLasker]

Who's "we". When this TOB vote was initiated, in what I assume was a TOB meeting, there was no invite to the artifact maintainers to have said discussion.
If there's no rush, the content in the specs isn't yet released, and the content in the artifacts repo is accurate for the currently released specs, then I'm not sure why we're having a forcing vote. Particularly, when the artifact maintainers have already agreed, and prepped the content. We just need to release the distribution and image specs. Then, we're done.

[jonjohnsonjr]

Who's "we".

opencontainers/artifacts#61

and the content in the artifacts repo is accurate for the currently released specs, then I'm not sure why we're having a forcing vote.

It's not, though. Its existence is misleading and harmful to the broader ecosystem.

Particularly, when the artifact maintainers have already agreed, and prepped the content. We just need to release the distribution and image specs. Then, we're done.

I am having a really hard time parsing this. Do you mean that we should wait to archive the artifacts repo until after the releases?

[cyphar]

LGTM (and if I'm counting correctly, that means this motion passes).

FWIW, I'm not sure whether the precise discussion on whether we want to completely delete the text of the repo or just add a header pointing to the new guidance is necessarily a TOB issue -- but a motion to archive it presumably means whatever statement is put into the artefacts README needs to unreservedly state the archived status of the project (as is the case with the wg-reference-types repo, for instance).

[vbatts]

LGTM

[sabre1041]

@cyphar @jonjohnsonjr @jdolitsky While not specifically mentioning you, wanted to share some insights/thoughts from someone who is semi new into the OCI space (compared to the TOB).

The artifacts repo and the WG blazed the trail for OCI artifacts and their incorporation into the image and distribution specs that are associated with the 1.1 release. It would be good to see this repository stand for historical purposes in an archived state, but making it explicitly clear the current state of the repository and providing references to the official documentation, guidance and status of Artifacts in OCI. Given that the release of 1.1 is when the changes of Artifacts becomes official, the archival of the repository should occur immediately following the release. Associated documentation on the current and future direction has already started to be incorporated into #70.

[jdolitsky]

The vote has passed 7/9. Thanks everyone.