Add devcontainers ecosystem

[joshspicer]

This change provides initial support for a devcontainers ecosystem, leaning heavily on the devcontainers/cli for ecosystem-specific implementation.

This updater will look in the provided --directory for a devcontainer.json (or .devcontainer.json if directory === /) and update all dev container Features that are pinned to an out-of-date major version. The lockfile will also be updated if one is already present in the target directory.

I aim to add future updater granularity/modes in the future.

Requires at least dev container CLI v0.54.0:

Demos (with the dependabot/cli)

LOCAL_GITHUB_ACCESS_TOKEN="$LOCAL_GITHUB_ACCESS_TOKEN"  dependabot  \
     --updater-image ghcr.io/dependabot/dependabot-updater-devcontainers  \
      update \
      devcontainers \
      joshspicer/versioning-project-two-configs \
      -o /tmp/updates.yaml \
      --directory /.devcontainer

updater | +-----------------------------------------------------------------------+
updater | |                  Changes to Dependabot Pull Requests                  |
updater | +---------+-------------------------------------------------------------+
updater | | created | ghcr.io/codspace/versioning/foo:1 ( from 1.1.0 to 2.11.1 )  |
updater | | created | ghcr.io/codspace/versioning/baz:1.0 ( from 1.0.0 to 2.0.0 ) |
updater | +---------+-------------------------------------------------------------+

LOCAL_GITHUB_ACCESS_TOKEN="$LOCAL_GITHUB_ACCESS_TOKEN"  dependabot  \
     --updater-image ghcr.io/dependabot/dependabot-updater-devcontainers  \
      update \
      devcontainers \
      joshspicer/versioning-project-two-configs \
      -o /tmp/updates.yaml \
      --directory /

updater | +----------------------------------------------------------------------+
updater | |                 Changes to Dependabot Pull Requests                  |
updater | +---------+------------------------------------------------------------+
updater | | created | ghcr.io/codspace/versioning/foo:1 ( from 1.1.0 to 2.11.1 ) |
updater | +---------+------------------------------------------------------------+

Closes #7000.

[deivid-rodriguez]

Made a first review pass, seems pretty neat!

[deivid-rodriguez]

I rebased the PR, addressed several issues, and implemented the new way of handling updates. Now we provide updates to all fetched files, relative to the configured directory (by default "/").

For example:

$ bin/dry-run.rb devcontainers joshspicer/versioning-project-two-configs 
=> cloning into /home/dependabot/tmp/joshspicer/versioning-project-two-configs
=> parsing dependency files
Running command: devcontainer outdated --workspace-folder . --config .devcontainer.json --output-format json
Running command: devcontainer outdated --workspace-folder . --config devcontainer.json --output-format json
=> updating 3 dependencies: ghcr.io/codspace/versioning/foo:1, ghcr.io/codspace/versioning/bar:1, ghcr.io/codspace/versioning/baz:1.0

=== ghcr.io/codspace/versioning/foo:1 (1.1.0)
 => checking for updates 1/3
 => latest available version is 2.11.1
 => latest allowed version is 2.11.1
 => requirements to unlock: own
 => requirements update strategy: 
Running command: devcontainer upgrade
Running command: devcontainer upgrade
 => bump ghcr.io/codspace/versioning/foo:1 from 1.1.0 to 2.11.1

    ± .devcontainer.json
    ~~~
    --- /tmp/original20231221-316-tawni8	2023-12-21 17:35:09.209309011 +0000
    +++ /tmp/updated20231221-316-8gbvq6	2023-12-21 17:35:09.209309011 +0000
    @@ -1,7 +1,7 @@
     {
         "image": "mcr.microsoft.com/devcontainers/typescript-node:18",
         "features": {
    -        "ghcr.io/codspace/versioning/foo:1": {},
    +        "ghcr.io/codspace/versioning/foo:2": {},
             "ghcr.io/codspace/versioning/bar:1": {}
         }
     }
    ~~~
    2 insertions (+), 2 deletions (-)

    ± .devcontainer/devcontainer.json
    ~~~
    --- /tmp/original20231221-316-6o3e3w	2023-12-21 17:35:09.212309011 +0000
    +++ /tmp/updated20231221-316-j9vt98	2023-12-21 17:35:09.212309011 +0000
    @@ -1,7 +1,7 @@
     {
         "image": "mcr.microsoft.com/devcontainers/typescript-node:18",
         "features": {
    -        "ghcr.io/codspace/versioning/foo:1": {},
    +        "ghcr.io/codspace/versioning/foo:2": {},
             "ghcr.io/codspace/versioning/bar:1": {},
             "ghcr.io/codspace/versioning/baz:1.0": {}
         }
    ~~~
    2 insertions (+), 2 deletions (-)

    ± .devcontainer/devcontainer-lock.json
    ~~~
    --- /tmp/original20231221-316-xio06s	2023-12-21 17:35:09.214309011 +0000
    +++ /tmp/updated20231221-316-ugjs8o	2023-12-21 17:35:09.214309011 +0000
    @@ -10,10 +10,10 @@
           "resolved": "ghcr.io/codspace/versioning/baz@sha256:37f36051adf6da0a43764b9669b945e0f06e11973e02c57ad261b03bb1057cb7",
           "integrity": "sha256:37f36051adf6da0a43764b9669b945e0f06e11973e02c57ad261b03bb1057cb7"
         },
    -    "ghcr.io/codspace/versioning/foo:1": {
    -      "version": "1.1.0",
    -      "resolved": "ghcr.io/codspace/versioning/foo@sha256:80d2d7b58afeaf907451c6f4e24de47b09a327a24a21a2d3323b7abf76d14be5",
    -      "integrity": "sha256:80d2d7b58afeaf907451c6f4e24de47b09a327a24a21a2d3323b7abf76d14be5"
    +    "ghcr.io/codspace/versioning/foo:2": {
    +      "version": "2.11.1",
    +      "resolved": "ghcr.io/codspace/versioning/foo@sha256:e98cdc5066cff85c5076dfec32058d53e5b9bbc75b125d84adcdf295674c14ee",
    +      "integrity": "sha256:e98cdc5066cff85c5076dfec32058d53e5b9bbc75b125d84adcdf295674c14ee"
         }
       }
     }
    \ No newline at end of file
    ~~~
    5 insertions (+), 5 deletions (-)

=== ghcr.io/codspace/versioning/bar:1 (1.0.0)
 => checking for updates 2/3
 => latest available version is 1.0.0
    (no update needed as it's already up-to-date)

=== ghcr.io/codspace/versioning/baz:1.0 (1.0.0)
 => checking for updates 3/3
 => latest available version is 2.0.0
 => latest allowed version is 2.0.0
 => requirements to unlock: own
 => requirements update strategy: 
Running command: devcontainer upgrade
 => bump ghcr.io/codspace/versioning/baz:1.0 from 1.0.0 to 2.0.0

    ± .devcontainer/devcontainer.json
    ~~~
    --- /tmp/original20231221-316-44zv9n	2023-12-21 17:35:14.339309013 +0000
    +++ /tmp/updated20231221-316-e1khaj	2023-12-21 17:35:14.340309013 +0000
    @@ -3,6 +3,6 @@
         "features": {
             "ghcr.io/codspace/versioning/foo:1": {},
             "ghcr.io/codspace/versioning/bar:1": {},
    -        "ghcr.io/codspace/versioning/baz:1.0": {}
    +        "ghcr.io/codspace/versioning/baz:2": {}
         }
     }
    ~~~
    2 insertions (+), 2 deletions (-)

    ± .devcontainer/devcontainer-lock.json
    ~~~
    --- /tmp/original20231221-316-fk8v4b	2023-12-21 17:35:14.343309013 +0000
    +++ /tmp/updated20231221-316-f439qx	2023-12-21 17:35:14.343309013 +0000
    @@ -5,10 +5,10 @@
           "resolved": "ghcr.io/codspace/versioning/bar@sha256:0eb80a7a45ea6ac6d2057798608be4cacb3d3667d4818118e17acc5037d687d4",
           "integrity": "sha256:0eb80a7a45ea6ac6d2057798608be4cacb3d3667d4818118e17acc5037d687d4"
         },
    -    "ghcr.io/codspace/versioning/baz:1.0": {
    -      "version": "1.0.0",
    -      "resolved": "ghcr.io/codspace/versioning/baz@sha256:37f36051adf6da0a43764b9669b945e0f06e11973e02c57ad261b03bb1057cb7",
    -      "integrity": "sha256:37f36051adf6da0a43764b9669b945e0f06e11973e02c57ad261b03bb1057cb7"
    +    "ghcr.io/codspace/versioning/baz:2": {
    +      "version": "2.0.0",
    +      "resolved": "ghcr.io/codspace/versioning/baz@sha256:3420e9d222352d5bee4d7f2c99dd7492295f5518041650ade9e8ecd0d6ce49d8",
    +      "integrity": "sha256:3420e9d222352d5bee4d7f2c99dd7492295f5518041650ade9e8ecd0d6ce49d8"
         },
         "ghcr.io/codspace/versioning/foo:1": {
           "version": "1.1.0",
    ~~~
    5 insertions (+), 5 deletions (-)
🌍 Total requests made: '0'

I saw a few more issues that I'll be addressing next week:

• Don't parse the requirement as part of the dependency name: it should be ghcr.io/codspace/versioning/foo not ghcr.io/codspace/versioning/foo:1.
• Respect style of existing requirements for now: ghcr.io/codspace/versioning/baz:1.0 should be updated to ghcr.io/codspace/versioning/baz:2.0, not to ghcr.io/codspace/versioning/baz:2.

Also found an issue in the CLI that I reported at devcontainers/cli#712.

[deivid-rodriguez]

I fixed all the issues mentioned previously and got CI to pass.

However, I was thinking about the smart approach taken here of asking devcontainers outdated directly in the FileParser step about the latest version and "bypassing" the UpdateChecker step and I believe it's going to come short for some common Dependabot features. For example:

• Users need to be able to skip certain versions from configuration. See https://docs.github.com/en/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file#ignore.
• Similarly (same doc as above), users may choose to ignore certain update types for a dependency, for example, patch and minor versions.

So I think we still need to implement a basic UpdateChecker that can access all available versions of the feature and do some filtering over them.

We'll need to work on this after holidays!

[deivid-rodriguez]

Alternatively, and probably better since it would only require changes to the devcontainers CLI, we could add an --ignore-range flag to the devcontainers outdated CLI command to support the first use case, and --ignore-major, --ignore-minor, and --ignore-patch flags to support the latter.

[deivid-rodriguez]

Found and fixed a few more issues, including one that I reported upstream (and added workaround for it here).

Also cleaned up git history to keep things a bit more tidy.

This is ready for a review!

[joshspicer]

✅ Did some bug bashing on the changes and I see no release blockers!

Some follow up fixes that I can explore moving forward, but I think given the new timeline this is good to go!

• Features that are pinned to an outdated SHA will get updated to the major version tag, and there's a bit of a strange message generated. Pinning to a SHA is not widely advertised and i'd argue it's still helpful to be aware of new Feature releases, even if pinned.
  • This could be rolled into adding some switches (like David's already suggested 🎉 ) to provide more granular control over when the updater is run
  • 
• Features that are inaccessible or are pinned with invalid tags cause a crash.
  • This implies an invalid devcontainer.json, or a devcontainer.json that requires some auth to query private container registries. Following up to support private CRs is known so that's OK
  • Repro with ghcr.io/devcontainers/features/docker-in-docker:1.1 (ie: a version that has never been tagged!)

Error: ERR: Feature 'ghcr.io/devcontainers/features/docker-in-docker:1.1' could not be processed.  You may not have permission to access this Feature, or may not be logged in.  If the issue persists, report this to the Feature author.
    at q7 (/usr/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:285:24143)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async AC (/usr/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:285:26711)
    at async Su (/usr/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:285:26937)
    at async vu (/usr/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:390:1818)
    at async meA (/usr/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:613:4322)
    at async /usr/lib/node_modules/@devcontainers/cli/dist/spec-node/devContainersSpecCLI.js:613:3357
        from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:169:in `bind_call'
        from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:169:in `validate_call_skip_block_type'
        from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:111:in `block in create_validator_slow_skip_block_type'
        from /home/dependabot/devcontainers/lib/dependabot/devcontainers/file_updater/config_updater.rb:68:in `run_devcontainer_upgrade'
        from /home/dependabot/devcontainers/lib/dependabot/devcontainers/file_updater/config_updater.rb:54:in `update_manifest'
        from /home/dependabot/devcontainers/lib/dependabot/devcontainers/file_updater/config_updater.rb:24:in `block (2 levels) in update'
        from /home/dependabot/common/lib/dependabot/shared_helpers.rb:264:in `with_git_configured'
        from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:272:in `bind_call'
        from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:272:in `validate_call'
        from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:193:in `block in create_validator_slow'
        from /home/dependabot/devcontainers/lib/dependabot/devcontainers/file_updater/config_updater.rb:23:in `block in update'
        from /home/dependabot/common/lib/dependabot/shared_helpers.rb:57:in `block in in_a_temporary_repo_directory'
        from /home/dependabot/common/lib/dependabot/shared_helpers.rb:57:in `chdir'
        from /home/dependabot/common/lib/dependabot/shared_helpers.rb:57:in `in_a_temporary_repo_directory'
        from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:272:in `bind_call'
        from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:272:in `validate_call'
        from /home/dependabot/dependabot-updater/vendor/ruby/3.1.0/gems/sorbet-runtime-0.5.11193/lib/types/private/methods/call_validation.rb:193:in `block in create_validator_slow'
        from /home/dependabot/devcontainers/lib/dependabot/devcontainers/file_updater/config_updater.rb:22:in `update'
        from /home/dependabot/devcontainers/lib/dependabot/devcontainers/file_updater.rb:78:in `update'
        from /home/dependabot/devcontainers/lib/dependabot/devcontainers/file_updater.rb:25:in `block in updated_dependency_files'
        from /home/dependabot/devcontainers/lib/dependabot/devcontainers/file_updater.rb:21:in `each'
        from /home/dependabot/devcontainers/lib/dependabot/devcontainers/file_updater.rb:21:in `updated_dependency_files'
        from bin/dry-run.rb:706:in `block in <main>'
        from bin/dry-run.rb:614:in `each'
        from bin/dry-run.rb:614:in `<main>'

[deivid-rodriguez]

@joshspicer I looked at the issues you found:

• For SHA256 versions, I'm ignoring them for now, so we don't even try to update them. If someone is using those, I guess it's their preference and they have some motivation, so better for Dependabot to not be annoying to those people. By the way, I'm doing the same for deprecated features because the CLI doesn't handle them very well at the moment. I looked for a "/" in the feature name to detect non deprecated features, is that correct?
• As per the second issue, after I added support for ignores, it's now working "fine": no error, and it's updating the invalid tag to the latest version. I think that's acceptable.

I also fixed some other issues I found and added support for ignores!

[deivid-rodriguez]

I merged the first commit in this PR as #8858, so that I can build updater images for this PR (since workflow files are picked up from the main branch).

[deivid-rodriguez]

Let's do this!