-
Notifications
You must be signed in to change notification settings - Fork 70
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade packit.yaml config to have integration tests #804
Conversation
Thank you for contributing to the Leapp project!Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergable.
To launch regression testing public members of oamg organization can leave the following comment:
Please open ticket in case you experience technical problem with the CI. (RH internal only) Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please consider rerunning the CI by commenting leapp-ci build (might require several comments). If the problem persists, contact leapp-infra. |
a77b3d5
to
f603155
Compare
Maybe some targets will fail right now because of:
|
/packit test |
.packit.yaml
Outdated
@@ -54,7 +54,7 @@ jobs: | |||
distros: [RHEL-7.9-ZStream] | |||
identifier: tests-79to84 | |||
tmt_plan: "^(?!.*upgrade_plugin)(?!.*c2r)(?!.*sap)(?!.*8to9)(?!.*morf)" | |||
tf_post_install_script: "#!/bin/sh\nsudo sed -i s/.*ssh-rsa/ssh-rsa/ /root/.ssh/authorized_keys;curl https://copr.fedorainfracloud.org/coprs/g/oamg/leapp/repo/epel-7/group_oamg-leapp-epel-7.repo --output /etc/yum.repos.d/copr_build.repo; yum install -y leapp*master*" | |||
tf_post_install_script: "#!/bin/sh\nsudo sed -i s/.*ssh-rsa/ssh-rsa/ /root/.ssh/authorized_keys; yum install -y leapp-repository*master*" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I remember our agreement about enabling tests with leapp-repositorymaster only now, but to have a plan for enabling testing with custom build from leapp-repository PR later.
I am just curious, how will this be done? Are we waiting for some feature to land in packit that will enable this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I think they implemented that already: packit/packit-service#1544
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And here's the link for the docs: https://packit.dev/docs/testing-farm/#running-tests-with-builds-from-another-pull-request
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am wondering if executing /packit test <namespace>/<repo>#<pr_id>
as recommended by packit will work fine if we have leapp-repositorymaster installation hardcoded as a tf_post_install_script..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Well, to be honest, that was a super recent change they made that may not be 100% working right now. In any way, I'm hoping to test that feature here instead of my demo repository, I figured that it would be nicer to have actual (real) results instead of the made up ones I have in my repo.
Since they included that command especially because we asked, I don't mind go to them and ask for an update on that if we see that the behavior is incorrect.
Failed to load packit config file:
For more info, please check out the documentation or contact the Packit team. |
1410943
to
15c6b6f
Compare
/packit test oamg/leapp-repository#master |
/packit test oamg/leapp-repository#989 |
/packit test oamg/leapp-repository#989 |
/packit test oamg/leapp-repository#988 |
/packit test oamg/leapp-repository#990 |
/packit test oamg/leapp-repository#990 |
/packit test |
/packit test oamg/leapp-repository#990 |
1 similar comment
/packit test oamg/leapp-repository#990 |
@fernflower the discovery of the rhel-7.9-rhui tests are failing right now because of this comparison packit is passing the distro name in a slightly differently way I'm not sure if packit offers a way to pass a different context/distro name in the request. @TomasTomecek could you help us here? If that's not possible, should we change the condition in the test then, @fernflower? |
Hello @r0x0d , I first thought that this is a problem with the execute:
how: tmt
script: /bin/true
enabled: false
adjust:
enabled: true
when: distro == rhel-7.9 With I've filed an issue for fmf: teemtee/fmf#186 and we are going to dicuss this tomorrow at tmt hacking session with other tmt contributors. We may add some form of case-insensitivity to context matching. For packit, we also have packit/packit-service#1797 to provide custom context, however I do not think that passing a custom You could try using |
Hi, @FrNecas! Thank you for your reply and for all that explanation. I will make sure that I follow both of those issues to keep updated with the current state of them. Also, for the workaround part, I will leave that decision to the @oamg/qe to see if it really fits in their use case or if they would like to go with some other route too. In any case, thanks again, I will wait a bit more for teemtee/fmf#186. |
Hi @r0x0d , if we change the condition in the test to 'distro == RHEL-7.9-rhui' then this test won't be run in case of a typical rhel-7.9 upgrade and we will need to duplicate the test, which is not really a favorable outcome because it becomes harder to maintain. It would be great to solve that on packit side imho, but of course if it doesn't work we can go back to discussing changes in the test. |
Seems reasonable to me. Let's wait a few more to see the outcome in the linked issues, maybe the solution on their side will be quick. |
@fernflower just a follow-up on that, me and @FrNecas were talking about this issue today, and we thought that leapp would benefit more from packit/packit-service#1797 as it will let us define custom contexts are you guys have currently in your GitHub actions testing farm workflow. The main breakdown on all of this is that tmt will stick with being case-sensitive and will document that for future uses. So, looking for alternatives, we realized that waiting and going for packit/packit-service#1797 would be more clean and suitable for leapp right now. |
Waiting for packit/packit-service#1797 |
This pull request introduces the execution of leapp integration tests as a packit job instead of the current behavior of using a GitHub Actions to trigger the tests by a comment. Signed-off-by: Rodolfo Olivieri <[email protected]>
Signed-off-by: Rodolfo Olivieri <[email protected]>
/packit test |
Okay, looks like installation of leapp-repository via postinstall script doesn't work - the package gets removed during guest setup artifacts' installation :( |
/packit test oamg/leapp-repository#master |
/packit build oamg/leapp-repository#master |
/packit test oamg/leapp-repository#1052 |
Signed-off-by: Rodolfo Olivieri <[email protected]>
/packit test oamg/leapp-repository#1052 |
Short summary: |
630ac68
to
6ff55fe
Compare
Signed-off-by: Rodolfo Olivieri <[email protected]>
6ff55fe
to
e1ef75b
Compare
/packit test oamg/leapp-repository#1052 |
/packit test oamg/leapp-repository#1052 |
/packit test oamg/leapp-repository#1052 |
Signed-off-by: Rodolfo Olivieri <[email protected]>
/packit test oamg/leapp-repository#1052 |
/packit test oamg/leapp-repository#1052 |
Moving this to draft not to be accidentally merged until packit/packit-service#1972 is delivered. |
We have finally got all the necessary features from tft to be able to install leapp in test env for packit-triggered tests! So #827 has this PR's commits with some updates on top and should be merged soon. |
This pull request introduces the execution of leapp integration tests as a packit job instead of the current behavior of using a GitHub Actions to trigger the tests by a comment.
Signed-off-by: Rodolfo Olivieri [email protected]