Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix IPU being blocked by resource limitations #1256

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

tomasfratrik
Copy link
Member

First resource limit is maximum number of open file descriptors limit, second one being limit for maximum writable file size.

With resource lib we set the limit for the current process (tested with my testing scripts) and its children processes. By testing pre-upgrade and upgrade I found a threshold of max. open file descriptors to be around 512, (the default value for ulimit -n is 1024) so a small multiplier of the default value shouldn't be a problem to set. With writing size there shouldn't be a problem to set it to "infinity".

Both problems from the tickets were solved, but there could be other side effects.

Resolves: RHEL-26459 and RHEL-16881

Copy link

Thank you for contributing to the Leapp project!

Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
If you want to request a review or rebuild a package in copr, you can use following commands as a comment:

  • review please @oamg/developers to notify leapp developers of the review request
  • /packit copr-build to submit a public copr build using packit

Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build.
However, here are additional useful commands for packit:

  • /packit test to re-run manually the default tests
  • /packit retest-failed to re-run failed tests manually
  • /packit test oamg/leapp#42 to run tests with leapp builds for the leapp PR#42 (default is latest upstream - master - build)

Note that first time contributors cannot run tests automatically - they need to be started by a reviewer.

It is possible to schedule specific on-demand tests as well. Currently 2 test sets are supported, beaker-minimal and kernel-rt, both can be used to be run on all upgrade paths or just a couple of specific ones.
To launch on-demand tests with packit:

  • /packit test --labels kernel-rt to schedule kernel-rt tests set for all upgrade paths
  • /packit test --labels beaker-minimal-8.10to9.4,kernel-rt-8.10to9.4 to schedule kernel-rt and beaker-minimal test sets for 8.10->9.4 upgrade path

See other labels for particular jobs defined in the .packit.yaml file.

Please open ticket in case you experience technical problem with the CI. (RH internal only)

Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra.

@tomasfratrik tomasfratrik marked this pull request as draft June 20, 2024 08:22
@tomasfratrik tomasfratrik changed the title Fix IPU being blocked by resource limitations [DRAFT] Fix IPU being blocked by resource limitations Jun 20, 2024
@tomasfratrik tomasfratrik force-pushed the resource-fix branch 2 times, most recently from ae14f2e to 1bfaa9b Compare June 24, 2024 08:50
@tomasfratrik tomasfratrik changed the title [DRAFT] Fix IPU being blocked by resource limitations Fix IPU being blocked by resource limitations Jun 24, 2024
@tomasfratrik tomasfratrik marked this pull request as ready for review June 24, 2024 10:55
@matejmatuska
Copy link
Member

/packit copr-build

@fernflower
Copy link
Member

/packit test

@tomasfratrik
Copy link
Member Author

I don't really like the redundancy of code here, but I'm not sure where should the common logic be placed.

@matejmatuska
Copy link
Member

I don't really like the redundancy of code here, but I'm not sure where should the common logic be placed.

I think commands/command_utils.py is an option. There is already some code the commands share.

@matejmatuska
Copy link
Member

/packit retest-failed

@tomasfratrik tomasfratrik force-pushed the resource-fix branch 2 times, most recently from 87f52e4 to e4afdea Compare July 2, 2024 09:16
@tomasfratrik
Copy link
Member Author

I don't really like the redundancy of code here, but I'm not sure where should the common logic be placed.

I think commands/command_utils.py is an option. There is already some code the commands share.

Thanks, I put it there

@pirat89
Copy link
Member

pirat89 commented Jul 2, 2024

/packit copr-build

@pirat89 pirat89 added this to the 8.10/9.6 milestone Jul 2, 2024
@pirat89
Copy link
Member

pirat89 commented Jul 4, 2024

all standard tests passed. executing extended tests

@pirat89
Copy link
Member

pirat89 commented Jul 4, 2024

/packit test

@pirat89
Copy link
Member

pirat89 commented Jul 4, 2024

ah.. I thought it's different PR :D mama mia!

@pirat89 pirat89 removed this from the 8.10/9.6 milestone Sep 21, 2024
@pirat89 pirat89 added the bug Something isn't working label Oct 25, 2024
@pirat89 pirat89 added this to the 8.10/9.6 milestone Oct 25, 2024
First resource limit is maximum number of open file descriptors limit,
second one being limit for maximum writable file size.

Resolves: RHEL-26459 and RHEL-16881
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants