Releases: glideinWMS/glideinwms
GlideinWMS 3.10.6
Released on 1/25/2024
GlideinWMS v3_10_6 has been released.
It is available in the osg-development repository for RHEL 7, RHEL 8, and RHEL 9 architectures and is undergoing testing.
Platforms tested and officially supported for production are EL7 with Python 3.6 and EL9 with Python 3.9.
This release adds a few features and bug fixes. It is a recommended upgrade from 3.10.5.
The full release notes are available here and a CHANGELOG is in the repository.
GIT: git checkout v3_10_6
RPMS: Available in the osg-development repo and Koji:
DOCUMENTATION: http://glideinwms.fnal.gov/doc/
IMPORTANT
- The rebuilding of cvmfsexec distributions is disabled by default since 3.10.5, it must be enabled via the new factory knob. If you are upgrading from v3.9.5 or later, check that the cvmfsexec script is no more in the pre-reconfigure hooks directory.
- When using on-demand CVMFS, all Glideins after the first one on a node are failing. To avoid this we recommend using whole-node Glideins when using on-demand CVMFS.
- cvmfsexec builds for EL9 are incompatible with EL7. If you have an EL7 factory and use cvmfsexec, make sure the EL9 is disabled
What's Changed
- ready for release v3_10_5 by @mambelli in #361
- Improvements and fixes in glidein startup script by @mambelli in #373
- Add knobs to allow overloading of memory and CPU. fixes #370 by @mmascher in #374
- Update M2Crypto imports by @BrunoCoimbra in #377
- V310 tarball downloader by @mmascher in #366
- Fixed add_to_path() to avoid partial path matching by @mambelli in #379
- Add import_module function and fix MatchPolicy imports by @BrunoCoimbra in #380
- Advertising the submit attributes to the glidefactory ClassAd by @mambelli in #382
- Release v3 10 6 rc1 and spec file fix by @mambelli in #384
- Fixed syntax in gangliad ClassAd configuration by @mambelli in #385
- Fixed writing of missing dict file during upgrade by @mambelli in #391
- Workaround and extra logging to investigate bug in log rotation by @mambelli in #389
- ready for release v3_10_6_rc2 by @mambelli in #392
- Encoding + as PLUS in ClassAd attributes names by @mambelli in #393
Full Changelog: v3_10_5...v3_10_6
Contributors
Assets 2
GlideinWMS 3.10.5
Released on 9/11/2023
GlideinWMS v3_10_5 has been released.
It is available in the osg-development repository for RHEL 7, RHEL 8, and RHEL 9 architectures and is undergoing testing.
This release has been extensively tested on EL9 and Python 3.9. It is the first one we recommend with this platform for production.
It fixes a few bugs in 3.10.3. No new features added. Some bugs were fixed in the interim 3.10.4, but that release should be ignored, just consider the changes in both as 3.10.5.
The full release notes are available here and a CHANGELOG is in the repository.
GIT: git checkout v3_10_3
RPMS: Available in the osg-development repo and Koji:
DOCUMENTATION: http://glideinwms.fnal.gov/doc/
IMPORTANT
- The rebuilding of cvmfsexec distributions is now disabled by default, it must be enabled via the new factory knob. If you are upgrading from v3.9.5 or later, check that the cvmfsexec script is no more in the pre-reconfigure hooks directory.
- When using on-demand CVMFS, all Glideins after the first one on a node are failing. To avoid this we recommend using whole-node Glideins when using on-demand CVMFS.
- cvmfsexec builds for EL9 are incompatible with EL7. If you have an EL7 factory and use cvmfsexec, make sure the EL9 is disabled
What's Changed
- Release v3 10 3 by @mambelli in #347
- gconfig.py should not use python3 but gwms-python. Fixes #349 by @mmascher in #350
- Fix rrdtool fetch call by @mambelli in #352
- Fixed alt code still using python (ie python2) interpreter by @mambelli in #353
- ready for release v3_10_4 by @mambelli in #354
- fix some syntax errors in analyze_queues by @jdost321 in #357
- Added bash requirement on scripts using bashisms by @mambelli in #358
- Workaround for EL7 PyJWT generating bytes instead of str by @mambelli in #355
- Fixed TRUST_DOMAIN in setup_x509 by @mambelli in #359
- GLIDEIN_SINGULARITY_BINARY_OVERRIDE set as parameter by @mambelli in #360
- Fix Factory reconfig/upgrade by @mambelli in #356
New Contributors
Full Changelog: v3_10_3...v3_10_5
Contributors
Assets 2
GlideinWMS 3.10.3
Released on 9/11/2023
GlideinWMS v3_10_3 has been released.
It is available in the osg-development repository for RHEL 7, RHEL 8, and RHEL 9 architectures and is undergoing testing.
This release has been extensively tested on EL9 and Python 3.9. It is the first one we recommend with this platform for production. It removes Python 3.9 deprecated code and fixes bugs including a workaround for a bug in the rrdtools library.
The full release notes are available here and a CHANGELOG is in the repository.
GIT: git checkout v3_10_3
RPMS: Available in the osg-development repo and Koji:
DOCUMENTATION: http://glideinwms.fnal.gov/doc/
IMPORTANT
- The rebuilding of cvmfsexec distributions is now disabled by default, it must be enabled via the new factory knob. If you are upgrading from v3.9.5 or later, check that the cvmfsexec script is no more in the pre-reconfigure hooks directory.
- When using on-demand CVMFS, all Glideins after the first one on a node are failing. To avoid this we recommend using whole-node Glideins when using on-demand CVMFS.
- cvmfsexec builds for EL9 are incompatible with EL7. If you have an EL7 factory and use cvmfsexec, make sure the EL9 is disabled
What's Changed
- ready for release v3_10_2 by @mambelli in #288
- Deprecation of getchildren() in ElementTree by @namrathaurs in #302
- Removing wrong classad Python module from requirements.txt by @mambelli in #301
- imp module deprecated in favor of importlib module by @namrathaurs in #303
- Pupulate Grid rc also for arc by @mmascher in #304
- Avoid invocation of cvmfsexec distros building if not necessary by @mmascher in #309
- ready for release 3_10_3_rc1 by @mambelli in #310
- Fixed encoding in token files (text files) and token utils by @mambelli in #311
- Allow OSG_autoconf to skip sites or CE that are not present in the OSG collector by @mmascher in #315
- Minor updates for on-demand cvmfsexec by @namrathaurs in #312
- Added debian11 as possible WN OS by @mambelli in #320
- Fixed idtoken error by @mambelli in #322
- Fix argmuent parsing for gwms-factory upgrade by @BrunoCoimbra in #323
- Web pages security and code fixes plus pre-commit update by @mambelli in #328
- Updated github actions versions and fixed some references by @mambelli in #329
- rrd improvements by @mambelli in #326
- Add idtokens to manual_glidein_submit by @mmascher in #284
- Sort VOs to make sure generated xml is always the same in OSG_autoconf by @mmascher in #317
- Add option to set xml output directory in OSG_autoconf. Fixes #318 by @mmascher in #319
- Ensure that credentials have absfname or generator attributes by @BrunoCoimbra in #336
- Fixed node validation and added case insensitive gconfig_get by @mambelli in #335
- Updated CI to use requirements.txt and fix flagged issues by @mambelli in #337
- Added support for timeout, encoding and logging to iexe_cmd() by @mambelli in #334
- Cleaned up exception handling by @mambelli in #330
- Added support for structured logs by @mambelli in #333
- Fix for rrdtool bindings crash when file is missing in EL9 by @namrathaurs in #339
- ready for release v3_10_3_rc2 by @mambelli in #340
- Add option to set num_factories in OSG_autoconf by @mmascher in #345
- Add an option to clear conflicting variables in containers environment by @mmascher in #342
- Added fallback config w/o CallsiteParameterAdder by @mambelli in #346
Full Changelog: v3_10_2...v3_10_3
Contributors
Assets 2
GlideinWMS 3.10.2
Released on 5/10/2023
GlideinWMS v3_10_2 has been released.
It is available in the osg-development repository for RHEL 7, RHEL 8, and RHEL 9 architectures and is undergoing testing.
The full release notes are available here and a CHANGELOG is in the repository.
GIT: git checkout v3_10_2
RPMS: Available in the osg-development repo and Koji:
DOCUMENTATION: http://glideinwms.fnal.gov/doc/
IMPORTANT
- The rebuilding of cvmfsexec distributions is now disabled by default, it must be enabled via the new factory knob. If you are upgrading from v3.9.5 or later, check that the cvmfsexec script is no more in the pre-reconfigure hooks directory.
- When using on-demand CVMFS, all Glideins after the first one on a node are failing. To avoid this we recommend using whole-node Glideins when using on-demand CVMFS.
What's Changed
- ready for release v3_10_1 by @mambelli in #257
- Substitution of deprecated grep command/add_config_line for reading from/writing to glidein_config by @namrathaurs in #238
- Use correct variable for $exit_code in singularity_exec_simple by @matyasselmeci in #259
- Make sure default value does not overwrite global one for idtoken_lif… by @mmascher in #263
- Make sure to call add_identity_credential for scitoken. Fixes #264 by @mmascher in #265
- Improved docstrings by @mambelli in #266
- Disabling TRACK and TRACE in the web server configuration by @mambelli in #260
- Add missing error code for BEST_FIT in OSG_autoconf by @mmascher in #269
- Add parameter to modify idtokens keyname by @mmascher in #268
- Protect OSG_autoconf from OSG collector unavailability. Fixes #273 by @mmascher in #276
- Improve the generation of JDL file for ARC CE REST interface by @mmascher in #271
- Fix nullglob problems by @mambelli in #279
- adding hepcloud as stakeholder by @mambelli in #277
- Factory Knob for Controlling Cvmfsexec Distributions by @namrathaurs in #272
- ARC-CEs REST interface has a new QUEUING state by @mmascher in #286
- Add rhel9 to condor_platform_select.sh by @mmascher in #285
Full Changelog: v3_10_1...v3_10_2
Contributors
Assets 2
GlideinWMS 3.10.1
Released on 12/13/2022
GlideinWMS v3_10_1 has been released.
It is available in the osg-development repository for RHEL 7 and RHEL 8 architectures and is undergoing testing.
GlideinWMS 3.10.1 is also in OSG 3.5 development but is not officially supported there.
This release is considered a production release completing token support and Python 3.
It is very similar to 3.9.6 with a few bug fixes and a python module to help with Python custom scripts.
The full release notes are available here and a CHANGELOG is in the repository.
GIT: git checkout v3_10_1
RPMS: Available in the osg-development repo and Koji:
DOCUMENTATION: http://glideinwms.fnal.gov/doc/
IMPORTANT. If you write setup (custom) scripts please change the use of glidein_config:
- custom scripts should always read values via
gconfig_get(). The only exception is the parsing of the line to get the add_config_line source file add_config_lineis deprecated in favor ofgconfig_add. Will be removed from future versionsadd_config_line_safeis deprecated in favor ofgconfig_add_safe.gconfig_addis the recommended method to use and it is safe also in concurrent scripts- custom script in Python should import gconfig.py (compatible w/ both Python 2 and 3) and use the provided functions or classes:
gconfig_get,gconfig_add, ...
What's Changed
- Release v3 9 6 by @mambelli in #230
- Conditionally include --pid based on environment (SOFTWARE-5340) by @edquist in #232
- Adding the build of the release documentation tarball to release.py by @mambelli in #231
- fix broken uses of PATH (SOFTWARE-5340) by @edquist in #233
- Updated singularity_lib.sh to use gconfig_ functions and bug fixes and hardenings by @mambelli in #235
- Added license_notice to the htchirp files by @mambelli in #236
- Added autodetect for aarch64 (ARM) architecture in condor platforms by @mambelli in #241
- Added credential ID when using scitokens by @mambelli in #243
- Improved regexes for issue labeler workflow by @namrathaurs in #246
- Added python utilities to read and write glidien_config by @mambelli in #237
- Release v3.10.0 rc1 by @mambelli in #248
- Fixed tags not to have " #" which starts a comment in yaml by @mambelli in #251
- Release v3_10_0 by @mambelli in #253
- Fix for failing GWMS rpms build jobs by @namrathaurs in #252
- Fixed status reporting for gconfig.py and added error_gen for python scripts by @mambelli in #254
- ready for release v3_10_1_rc1 by @mambelli in #255
Full Changelog: v3_9_6...v3_10_1
Contributors
Assets 2
GlideinWMS 3.9.6
Released on 10/27/2022
GlideinWMS v3_9_6 has been released.
It is available in the osg-development repository for RHEL 7 and RHEL 8 architectures and is undergoing testing.
GlideinWMS 3.9.6 is also in OSG 3.5 development but is not officially supported there.
This release adds token (and hybrid) support for Clouds (AWS/GCE) which completes the token support in GlideinWMS.
A big thank you to @ddbox for pushing forward and adding all the token support to GlideinWMS.
The release has also many bug fixes, including: glidien_config no more corrupted by setup scripts, fixed DC_DAEMON_LIST, one Python 2 leftover code updated, and setup_x509.sh no more writing messages in the classad.
The full release notes are available here and a CHANGELOG is in the repository.
GIT: git checkout v3_9_6
RPMS: Available in the osg-development repo and Koji:
DOCUMENTATION: http://glideinwms.fnal.gov/doc/
IMPORTANT. If you write setup (custom) scripts please change the use of glidein_config:
- custom scripts should always read values via
gconfig_get(). The only exception is the parsing of the line to get the add_config_line source file add_config_lineis deprecated in favor ofgconfig_add. Will be removed from future versionsadd_config_line_safeis deprecated in favor ofgconfig_add_safe.gconfig_addis the recommended method to use and it is safe also in concurrent scripts
What's Changed
- Fix RFC Proxy Regenration by @efajardo in #26
- Create codeql-analysis.yml by @mambelli in #31
- Add the ability to set the X509 proxies strength by @efajardo in #32
- Add missing filename to grep line by @matyasselmeci in #33
- For default make RFC proxies for direct proxies too by @efajardo in #29
- fix replace_range function to allow hyphens in the hostname by @ddavila0 in #38
- Adding a httpd subpackage by @efajardo in #37
- Disable LD_PRELOAD during container execution by @rynge in #39
- Cleaned up GPU implementation. We no longer do direct binding to /hos… by @rynge in #40
- Call 'od' to limit data from /dev/urandom by @brianhlin in #43
- Adding condor_gpu_utilization to the tarball by @efajardo in #42
- Don't set LD_LIBRARY_PATH in condor_startup.sh for job or daemons by @GregThain in #54
- Bump VOMS proxy key bitsize to 2048 by @brianhlin in #62
- Fix OSG_autoconf for compatibility with Python 3 by @BrunoCoimbra in #69
- Fixing end of files and whitespaces by @mambelli in #70
- Addin basic python pre-commit tests by @mambelli in #71
- Some changes to make manual_glidein_submit python3 compatible by @mmascher in #72
- Do not clear LD_LIBRARY_PATH before starting condor. Fixes #73 by @mmascher in #74
- [v3_7_6] Do not clear LD_LIBRARY_PATH before starting condor. Fixes #73 by @mmascher in #75
- Enable submit attrs for ARC REST interface by @mmascher in #81
- Added anchor to refer directly a version in the history file by @mambelli in #83
- Strip newline at the end of scitoken. Fixes #77 by @mmascher in #78
- Make sure submit attributes are set in the OSG.yaml file. Fixes #79 by @mmascher in #80
- issue 65 imporove SHARED_PORT defaults by @ddbox in #85
- Precommit auto formatting w settings by @mambelli in #87
- prettier modification and fixes by @mambelli in #88
- Fix scitoken transfer in glideFactoryEntry.py by @BrunoCoimbra in #92
- fix for issue 89 by @ddbox in #90
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #96
- Improved fe logging by @rynge in #94
- Fix OS detection fot CentOS Stream 8 by @mmascher in #98
- Downgrade pyupgrade version on pre-commit config by @BrunoCoimbra in #102
- Adding issue templates by @mambelli in #93
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #110
- Supporting the new python3-rrdtool added in OSG 3.6 also for RHEL7 by @mambelli in #112
- Add dynamic creation of distribution, refactor and include documentation for on demand CVMFS feature by @namrathaurs in #107
- Special handling of multiline condor config values by @mambelli in #117
- Added Apptainer support by @mambelli in #109
- Advertising CONTAINERSW_FULL_VERSION and CONTAINERSW_ variables by @mambelli in #118
- Added changelog file and fixed the yaml log for releases by @mambelli in #120
- ensure any files ending in 'idtoken' are transferred to glidein by @ddbox in #119
- (26153) Version-based download/build of cvmfsexec distribution tar files by @namrathaurs in #113
- fix python3-rrdtool dependencies (SOFTWARE-5134) by @edquist in #125
- Add line terminator to the end of decoded SciToken by @BrunoCoimbra in #111
- Add the possibility to configure the lifetime of idtokens. Fixes #129 by @mmascher in #131
- fix of a bad merge by @ddbox in #136
- Issue 128 master by @ddbox in #133
- Fix bad merge by @ddbox in #137
- Integrate changes to support credential generators by @BrunoCoimbra in #108
- Update osg-release.sh by @BrunoCoimbra in #139
- Issue 128 fixed by @ddbox in #141
- Factory will not skip request on expired scitoken by @BrunoCoimbra in #143
- Fix condor tarball directory structure by @mmascher in #144
- Release 3.9.5.rc3 by @BrunoCoimbra in #146
- Added credential generators documentation by @mambelli in #147
- Release 3.9.5 by @BrunoCoimbra in #148
- lock pyupgrade version for py3.6 compatibility by @mambelli in #151
- Added clarification for credential generator by @mambelli in #150
- Fix pre-commit node version to 17.9.0, the last to support SL7. by @mambelli in #156
- Prevent collision when generating credentials by @BrunoCoimbra in #157
- Fix factory crash when excess removal is enabled in the frontend. Fix… by @mmascher in #159
- Adding GLIDEIN_PERIODIC_SCRIPT to env to notify user scripts by @mambelli in #161
- Allows token usage in manual_glidein_submit. Fixes #140 by @mmascher in #158
- Update DEVELOPMENT.md by @mmascher in #167
- make default glidein_config robust to parallel updates by @mambelli in #166
- Updated developers documentation by @mambelli in #168
- ready for release v3_9_6_rc1 by @mambelli in #172
- Removed pylint complain about test_time by @mambelli in #173
- Fix exception during completed stats computation by @mambelli in #176
- Fixing add_config_line alias and adding NFS-safe spin lock by @mambelli in #175
- Fix fair_split and support token proxy pairs by @mambelli in #178
- Use time.time when ServerTime is not available in condor classads. Fi… by @mmascher in #184
- Read...
Contributors
Assets 2
v3.7.6
GlideinWMS v3_7_6 has been released.
It is available in the production OSG yum repository for OSG 3.5.
This is a stable release using Python 2. It supports GSI and token (IDTOKEN, SciToken) authentications. User jobs can run in containers (Singularity) and use CVMFS also on resources where it is not installed on the system.
The full release notes are available here:
http://glideinwms.fnal.gov/doc.v3_7_6/history.html
GIT:
git checkout v3_7_6
RPMS:
Available in the OSG yum repo.
DOCUMENTATION:
http://glideinwms.fnal.gov/doc/
v3.6.5
GlideinWMS v3_6_5 has been released.
It is available in the production osg yum repository for both OSG 3.4 (to support also CREAM CEs) and 3.5.
This is a stable release using Python 2. It supports GSI authentication and both Factory and Frontend run as unprivileged services (the need for root-switchboard for the Factory has been removed).
The full release notes are available here:
http://glideinwms.fnal.gov/doc.v3_6_5/history.html
GIT:
git checkout v3_6_5
RPMS:
Available in the osg yum repo.
DOCUMENTATION:
http://glideinwms.fnal.gov/doc/
v3.4
Greetings,
GlideinWMS v3_4 has been released.
It is available in the osg-upcoming repository and will migrate in the future to the OSG production repository.
A lot of code changes for this release, about 10 times the usual, including code modernization to Python 2.7 idioms (Python 2.6 is still supported) and many new unit tests.
Please read the warnings below, some actions may be required.
This release includes all features of previous stable series 3.2 and development series 3.3.
New to 3.2 are the support for Google CE and the policy plugins.
Other new exciting features include:
- Glidein lifetime not based anymore on the length of the proxy
- Internal support of condor_switchboard (discontinued by HTCondor)
- New option to kill glideins when job requests decrease
- Estimate in advance the cores provided to glideins discovering cores automatically
- Add entry monitoring breakdown for metasites
- Review Factory and Frontend tools, especially glidien_off and manual_glidein_submit.py
- Singularity improvements to use PATH and module when SINGULARITY_BIN does not contain the correct path and to run in a separate session to support restricted-access CVMFS
WARNINGS, some action may be required for the following
All:
- GLIDEIN_ToDie is no more shortened depending on the X509 proxy lifetime, to get the old behavior set GLIDEIN_Ignore_X509_Duration to False
- The type of the GLIDEIN_CPU attr is String (to accommodate the keywords auto, slot, node). Documentation was reporting Int incorrectly. Make sure your configuration uses the correct type or you may get a reconfig/upgrade error.
Factory only:
3. If you use HTCondor 8.7.2 or bigger with the GlideinWMS Factory, you must install also glideinwms-switchboard, to have condor_root_switchboard, that was removed from HTCondor
4. 'entry_sets' should be considered an experimental feature: the implementation will change in 3.4.1 and there may be errors and manual interventions required when upgrading to new versions of GlideinWMS if metasites are configured
The full release notes are available here:
http://glideinwms.fnal.gov/doc.v3_4/history.html
GIT:
git checkout v3_4
TARBALL:
http://glideinwms.fnal.gov/glideinWMS_v3_4.tgz
http://glideinwms.fnal.gov/glideinWMS_v3_4_factory.tgz
http://glideinwms.fnal.gov/glideinWMS_v3_4_frontend.tgz
RPMS:
Available in the osg-upcoming repo.
DOCUMENTATION:
http://glideinwms.fnal.gov/doc/