Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Prow test dumper script to crosscheck settings with GitHub branchprotections #915

Merged
merged 1 commit into from
Nov 27, 2024

Conversation

tuminoid
Copy link
Member

@tuminoid tuminoid commented Nov 26, 2024

Improve prow_test_dumper.py not only to dump test settings from Prow's config.yaml, to actually crosscheck versus GitHub branch protections. This highlights any mismatches we might have, accounting for both missing but also extra checks.

Also, set defaults to metal3-io file locations so --job-dir and --config do not need to be input in our use case. Make --config also flag and not positional arg.

GITHUB_TOKEN must be exported or supplied via --token, and it must have maintainer rights to read branchprotection settings. Using token without sufficient rights lead to erroring out.

This reveals we have one misconfigured branch in CAPM3/release-1.6 branch (irrelevant as it is EOS, not fixing it until this is merged, so it can work as POC this works).

Output in case of misconfigs:

$ GITHUB_TOKEN=<token> ./hack/prow_test_dumper.py 
metal3-io/cluster-api-provider-metal3/main:
- metal3-centos-e2e-integration-test-main
- metal3-ubuntu-e2e-integration-test-main

metal3-io/cluster-api-provider-metal3/release-1.6:
- metal3-centos-e2e-integration-test-release-1-6 (MISSING FROM GITHUB!)
- metal3-ubuntu-e2e-integration-test-release-1-6 (EXTRA IN GH, OR MISSING FROM PROW CONFIG!)

metal3-io/cluster-api-provider-metal3/release-1.7:
- metal3-centos-e2e-integration-test-release-1-7

...

@metal3-io-bot metal3-io-bot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Nov 26, 2024
@tuminoid
Copy link
Member Author

/hold
I'm going to make it work without token as well, defaulting to old behavior.

@metal3-io-bot metal3-io-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 26, 2024
Improve prow-test-dumper.py not only to dump test settings from prow
config.yaml, to actually crosscheck versus GitHub branch protections.
This highlights any mismatches we might have, accounting for both
missing but also extra checks.

Also, set defaults to metal3-io file locations so --job-dir and
--config do not need to be input in our use case.

GITHUB_TOKEN with maintainer read rights must be exported in env, or
supplied with --token to do this. If token is not found, old behavior
of just dumping the config is done, but no crosschecking. Using token
with invalid/missing rights error out.

Signed-off-by: Tuomo Tanskanen <[email protected]>
@tuminoid tuminoid force-pushed the tuomo/improve-test-dumper branch from 35bfbdd to 2a8c4ad Compare November 26, 2024 10:25
@tuminoid
Copy link
Member Author

/unhold
Old behavior without GITHUB_TOKEN restored.

/cc @kashifest @lentzi90

@metal3-io-bot metal3-io-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 26, 2024
@tuminoid
Copy link
Member Author

/override metal3-ubuntu-e2e-integration-test-main

@metal3-io-bot
Copy link
Collaborator

@tuminoid: Overrode contexts on behalf of tuminoid: metal3-ubuntu-e2e-integration-test-main

In response to this:

/override metal3-ubuntu-e2e-integration-test-main

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copy link
Member

@lentzi90 lentzi90 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve
Works well without github token at least :)

@metal3-io-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lentzi90

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@metal3-io-bot metal3-io-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 26, 2024
@tuminoid
Copy link
Member Author

/approve Works well without github token at least :)

Output with token and a failing config:

GITHUB_TOKEN=<token> ./hack/prow_test_dumper.py 
metal3-io/cluster-api-provider-metal3/main:
- metal3-centos-e2e-integration-test-main
- metal3-ubuntu-e2e-integration-test-main

metal3-io/cluster-api-provider-metal3/release-1.6:
- metal3-centos-e2e-integration-test-release-1-6 (MISSING FROM GITHUB!)
- metal3-ubuntu-e2e-integration-test-release-1-6 (EXTRA IN GH, OR MISSING FROM PROW CONFIG!)

metal3-io/cluster-api-provider-metal3/release-1.7:
- metal3-centos-e2e-integration-test-release-1-7

...

@tuminoid
Copy link
Member Author

/cc @smoshiur1237

Copy link
Member

@smoshiur1237 smoshiur1237 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@metal3-io-bot metal3-io-bot added the lgtm Indicates that a PR is ready to be merged. label Nov 27, 2024
@metal3-io-bot metal3-io-bot merged commit da8adfa into metal3-io:main Nov 27, 2024
7 checks passed
@metal3-io-bot metal3-io-bot deleted the tuomo/improve-test-dumper branch November 27, 2024 10:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants