fix: retirement pipeline pip version

[dyudyunov]

With the current version of the pip (19.0.3) it is impossible to install the cryptography package with a version higher than 39.X.X (starting from the palm release it is 40+, source: https://github.com/openedx/tubular/blob/open-release/palm.3/requirements/base.txt#L47)

The issue could be resolved by either installing the rustc to build the package or upgrading the pip version to use the prebuild wheel which is the recommendation from the cryptography setup guide: https://cryptography.io/en/latest/installation/#building-cryptography-on-linux

In this PR I increased the pip version using the version pinned in the tubular requirements:
https://github.com/openedx/tubular/blob/master/requirements/pip.txt#L11

The pip version installation described here
https://github.com/openedx/configuration/blob/master/playbooks/roles/user_retirement_pipeline/tasks/main.yml#L34

---

Notes

Installing requirements from the requirements/pip.txt file instead of pinning the pip version in the configuration seems to be a better solution for me, but I decided to go with minimal changes for now.

Here are the error logs for the issue:

TASK [user_retirement_pipeline : Install python requirements] ******************
fatal: [palmtest-dev]: FAILED! => changed=true 
  cmd:
  - pip
  - install
  - -r
  - requirements.txt
  delta: '0:00:11.107607'
  end: '2023-10-03 02:22:30.434814'
  msg: non-zero return code
  rc: 1
  start: '2023-10-03 02:22:19.327207'
  stderr: |2-
      Failed building wheel for cryptography
    Could not build wheels for cryptography which use PEP 517 and cannot be installed directly
    You are using pip version 19.0.3, however version 23.2.1 is available.
    You should consider upgrading via the 'pip install --upgrade pip' command.
  stderr_lines: <omitted>
  stdout: |-

 /tmp/pip-build-env-1tb3_fgi/overlay/lib/python3.8/site-packages/setuptools/command/build_py.py:204: _Warning: Package 'cryptography.hazmat.bindings._rust' is absent from the `packages` configuration.
      !!
  
              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'cryptography.hazmat.bindings._rust' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.
  
              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'cryptography.hazmat.bindings._rust' is explicitly added
              to the `packages` configuration field.
  
              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).
  
              You can read more about "package discovery" on setuptools documentation page:
  
              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
              If you don't want 'cryptography.hazmat.bindings._rust' to be distributed and are
              already explicitly excluding 'cryptography.hazmat.bindings._rust' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.
  
              You can read more about "package data files" on setuptools documentation page:
  
              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************
  
      !!
        check.warn(importable)
      /tmp/pip-build-env-1tb3_fgi/overlay/lib/python3.8/site-packages/setuptools/command/build_py.py:204: _Warning: Package 'cryptography.hazmat.bindings._rust.openssl' is absent from the `packages` configuration.
      !!
  
              ********************************************************************************
              ############################
              # Package would be ignored #
              ############################
              Python recognizes 'cryptography.hazmat.bindings._rust.openssl' as an importable package[^1],
              but it is absent from setuptools' `packages` configuration.
  
              This leads to an ambiguous overall configuration. If you want to distribute this
              package, please make sure that 'cryptography.hazmat.bindings._rust.openssl' is explicitly added
              to the `packages` configuration field.
  
              Alternatively, you can also rely on setuptools' discovery methods
              (for example by using `find_namespace_packages(...)`/`find_namespace:`
              instead of `find_packages(...)`/`find:`).
  
              You can read more about "package discovery" on setuptools documentation page:
  
              - https://setuptools.pypa.io/en/latest/userguide/package_discovery.html
  
              If you don't want 'cryptography.hazmat.bindings._rust.openssl' to be distributed and are
              already explicitly excluding 'cryptography.hazmat.bindings._rust.openssl' via
              `find_namespace_packages(...)/find_namespace` or `find_packages(...)/find`,
              you can try to use `exclude_package_data`, or `include-package-data=False` in
              combination with a more fine grained `package-data` configuration.
  
              You can read more about "package data files" on setuptools documentation page:
  
              - https://setuptools.pypa.io/en/latest/userguide/datafiles.html
  
  
              [^1]: For Python, any directory (with suitable naming) can be imported,
                    even if it does not contain any `.py` files.
                    On the other hand, currently there is no concept of package data
                    directory, all directories are treated like packages.
              ********************************************************************************
  
      !!
        check.warn(importable)
  
          =============================DEBUG ASSISTANCE=============================
          If you are seeing a compilation error please try the following steps to
          successfully install cryptography:
          1) Upgrade to the latest pip and try again. This will fix errors for most
             users. See: https://pip.pypa.io/en/stable/installing/#upgrading-pip
          2) Read https://cryptography.io/en/latest/installation/ for specific
             instructions for your platform.
          3) Check our frequently asked questions for more information:
             https://cryptography.io/en/latest/faq/
          4) Ensure you have a recent Rust toolchain installed:
             https://cryptography.io/en/latest/installation/#rust
  
          Python: 3.8.10
          platform: Linux-5.4.0-33-generic-x86_64-with-glibc2.29
          pip: n/a
          setuptools: 68.2.2
          setuptools_rust: 1.7.0
          rustc: n/a
          =============================DEBUG ASSISTANCE=============================
  
      error: can't find Rust compiler
  
      If you are using an outdated pip version, it is possible a prebuilt wheel is available for this package but pip is not able to install from it. Installing from the wheel would avoid the need for a Rust compiler.
  
      To update pip, run:
  
          pip install --upgrade pip
  
      and then retry package installation.
  
      If you did intend to build this package from source, try installing a Rust compiler from your system package manager and ensure it is on the PATH during installation. Alternatively, rustup (available at https://rustup.rs) is the recommended way to download and update the Rust compiler toolchain.
  
      This package requires Rust >=1.48.0.
  
      ----------------------------------------
      Running setup.py clean for cryptography
    Failed to build cryptography
  stdout_lines: <omitted>

[openedx-webhooks]

Thanks for the pull request, @dyudyunov! Please note that it may take us up to several weeks or months to complete a review and merge your PR.

Feel free to add as much of the following information to the ticket as you can:

• supporting documentation
• Open edX discussion forum threads
• timeline information ("this must be merged by XX date", and why that is)
• partner information ("this is a course on edx.org")
• any other information that can help Product understand the context for the PR

All technical communication about the code itself will be done via the GitHub pull request interface. As a reminder, our process documentation is here.

Please let us know once your PR is ready for our review and all tests are green.

[mphilbrick211]

Hi @edx-secure @adzuci! Would someone be able to please review / merge this for us? There is also a backport.

[mphilbrick211]

Hi @edx-secure @adzuci! Would someone be able to please review / merge this for us? There is also a backport.

Hi @edx-secure @adzuci! Following up on this. :)

[pomegranited]

@Agrendalath @xitij2000 Have we seen this error before? Any concerns about this solution?

[Agrendalath]

@pomegranited, yes, we're just pinning retirement_service_pip_version: 21.2.1, which matches the version used here. If the newer version works, then it's even better. 👍🏻 from me.

[pomegranited]

👍

• I tested this by manually running the playbook steps to ensure dependencies will be installed:

  git clone https://github.com/openedx/tubular/
  cd tubular
  
  python3 -m venv venv
  source venv/bin/activate
  
  # https://github.com/dyudyunov/configuration/blob/fix-retirement-pipeline/playbooks/roles/user_retirement_pipeline/tasks/main.yml#L34-L39
  pip install pip==21.2.1 # COMMON_PIP_VERSION 
  
  # https://github.com/dyudyunov/configuration/blob/fix-retirement-pipeline/playbooks/roles/user_retirement_pipeline/tasks/main.yml#L41-L49
  pip install -r requirements.txt
  pip freeze | grep cryptography
  # cryptography==41.0.4
  

• I read through the code
• I checked for accessibility issues N/A
• Includes documentation N/A
• User-facing strings are extracted for translation N/A

[openedx-webhooks]

@dyudyunov 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.