Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split microservices into separate packages #11324

Merged
merged 1 commit into from
Oct 7, 2022

Conversation

goughes
Copy link
Contributor

@goughes goughes commented Oct 6, 2022

Fixes #10159

Status

Not tested

Description

This PR updates build_dependencies.py, creates a new reqmgr2ms-core system and adds separate packages for all the current microservices.

Is it backward compatible (if not, which system it affects?)

Yes, we leave the current reqmgr2ms package which is still used for RPM based builds.

Related PRs

Follow up to #10357 as I have created a "MSCore" directory and moved MSCore, MSManager and TaskManager modules to it.

External dependencies / deployment changes

We will need to update each microservice's config.py since the path for MSManager has changed.
https://gitlab.cern.ch/cmsweb-k8s/services_config/-/merge_requests/168
https://gitlab.cern.ch/cmsweb-k8s/services_config/-/merge_requests/167
https://gitlab.cern.ch/cmsweb-k8s/services_config/-/merge_requests/166
dmwm/deployment#1192

@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: failed
    • 30 new failures
    • 2 changes in unstable tests
  • Python3 Pylint check: failed
    • 18 warnings and errors that must be fixed
    • 39 warnings
    • 276 comments to review
  • Pylint py3k check: failed
    • 1 warnings
  • Pycodestyle check: succeeded
    • 83 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13615/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: failed
    • 26 new failures
    • 2 changes in unstable tests
  • Python3 Pylint check: failed
    • 18 warnings and errors that must be fixed
    • 39 warnings
    • 276 comments to review
  • Pylint py3k check: failed
    • 1 warnings
  • Pycodestyle check: succeeded
    • 83 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13617/artifact/artifacts/PullRequestReport.html

@goughes goughes force-pushed the split-ms-pypi branch 3 times, most recently from 2b7a3f4 to 01bf2a3 Compare October 6, 2022 19:06
@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: failed
    • 4 new failures
    • 3 changes in unstable tests
  • Python3 Pylint check: failed
    • 18 warnings and errors that must be fixed
    • 39 warnings
    • 276 comments to review
  • Pylint py3k check: failed
    • 1 warnings
  • Pycodestyle check: succeeded
    • 83 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13618/artifact/artifacts/PullRequestReport.html

Copy link
Contributor

@amaltaro amaltaro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Erik, changes look good to me. I left one comment along the code though.
I think we should stop building reqmgr2ms for PyPi, even though we still need it for RPMs.

@@ -14,7 +14,9 @@ set -x
# package passed as parameter, can be one of PACKAGES or "all"
TOBUILD=$1
# list of packages that can be built and uploaded to pypi
PACKAGES="wmagent wmagent-devtools wmcore reqmon reqmgr2 reqmgr2ms global-workqueue acdcserver"
PACKAGES="wmagent wmagent-devtools wmcore reqmon reqmgr2 reqmgr2ms global-workqueue acdcserver
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we should stop building/uploading reqmgr2ms to PyPi?

@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: failed
    • 4 new failures
    • 3 changes in unstable tests
  • Python3 Pylint check: failed
    • 15 warnings and errors that must be fixed
    • 39 warnings
    • 276 comments to review
  • Pylint py3k check: failed
    • 1 warnings
  • Pycodestyle check: succeeded
    • 84 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13619/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: failed
    • 1 new failures
    • 2 changes in unstable tests
  • Python3 Pylint check: failed
    • 14 warnings and errors that must be fixed
    • 39 warnings
    • 276 comments to review
  • Pylint py3k check: failed
    • 1 warnings
  • Pycodestyle check: succeeded
    • 84 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13620/artifact/artifacts/PullRequestReport.html

@goughes
Copy link
Contributor Author

goughes commented Oct 6, 2022

Thanks @amaltaro, I have removed reqmgr2ms from requirements.txt and the PyPI build script.

@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: succeeded
    • 4 changes in unstable tests
  • Python3 Pylint check: failed
    • 14 warnings and errors that must be fixed
    • 39 warnings
    • 276 comments to review
  • Pylint py3k check: failed
    • 1 warnings
  • Pycodestyle check: succeeded
    • 84 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13621/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: succeeded
    • 3 changes in unstable tests
  • Python3 Pylint check: failed
    • 14 warnings and errors that must be fixed
    • 39 warnings
    • 276 comments to review
  • Pylint py3k check: failed
    • 1 warnings
  • Pycodestyle check: succeeded
    • 84 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/13622/artifact/artifacts/PullRequestReport.html

@amaltaro amaltaro self-requested a review October 6, 2022 20:44
Copy link
Contributor

@amaltaro amaltaro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Erik, this looks good to go!

@amaltaro
Copy link
Contributor

amaltaro commented Oct 7, 2022

Thanks Erik!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Split reqmgr2ms into separate PyPI packages
3 participants