Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Poetry #179

Merged
merged 16 commits into from
Jan 29, 2025
Merged

Poetry #179

merged 16 commits into from
Jan 29, 2025

Conversation

peppelinux
Copy link
Member

this PR switches from virtualenv and pip to poetry

@peppelinux peppelinux requested a review from MdreW January 24, 2025 16:24
@MdreW
Copy link
Collaborator

MdreW commented Jan 27, 2025

Hi @peppelinux , the Dockerfile don't install poetry, work with old method:

RUN python3 -m venv .venv && . .venv/bin/activate && pip3 install --upgrade pip setuptools \
&& pip3 install -r requirements.txt --ignore-installed --root-user-action=ignore && mkdir $BASEDIR \
&& addgroup -S satosa && adduser -S satosa -G satosa \
&& chown satosa:satosa $BASEDIR

Could you have missed a commit?

@MdreW
Copy link
Collaborator

MdreW commented Jan 27, 2025

@peppelinux other question, with env in poetry I have some problem:

poetry env is readed

[ranaldi@andrea-ranaldi-p Docker-compose]$ poetry env activate
[ranaldi@andrea-ranaldi-p Docker-compose]$ poetry env info

Virtualenv
Python:         3.13.1
Implementation: CPython
Path:           /home/ranaldi/.cache/pypoetry/virtualenvs/iam-proxy-italia-4RdGUjsM-py3.13
Executable:     /home/ranaldi/.cache/pypoetry/virtualenvs/iam-proxy-italia-4RdGUjsM-py3.13/bin/python
Valid:          True

Base
Platform:   linux
OS:         posix
Python:     3.13.1
Path:       /usr
Executable: /usr/bin/python3.13

But fail the poetry dependance installation:

[ranaldi@andrea-ranaldi-p Docker-compose]$ poetry install
Installing dependencies from lock file

Package operations: 54 installs, 0 updates, 0 removals

  - Installing pydantic-core (2.4.0): Failed

PEP517 build of a dependency failed

Backend subprocess exited when trying to invoke build_wheel

    | Command '['/tmp/tmp8m0p5yy0/.venv/bin/python', '/home/ranaldi/.local/share/pipx/venvs/poetry/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py', 'build_wheel', '/tmp/tmpy_psdwcz']' returned non-zero exit status 1.
    |
    | Running `maturin pep517 build-wheel -i /tmp/tmp8m0p5yy0/.venv/bin/python --compatibility off`
    | 📦 Including license file "/tmp/tmpyqm8ny82/pydantic_core-2.4.0/LICENSE"
    | 🍹 Building a mixed python/rust project
    | 🔗 Found pyo3 bindings
    | 🐍 Found CPython 3.13 at /tmp/tmp8m0p5yy0/.venv/bin/python
    | 📡 Using build options features, bindings from pyproject.toml
    |    Compiling autocfg v1.1.0
    |    Compiling target-lexicon v0.12.9
    |    Compiling proc-macro2 v1.0.64
    |    Compiling python3-dll-a v0.2.9
    |    Compiling unicode-ident v1.0.10
    |    Compiling quote v1.0.29
    |    Compiling libc v0.2.147
    |    Compiling once_cell v1.18.0
    |    Compiling rustversion v1.0.13
    |    Compiling syn v1.0.109
    |    Compiling version_check v0.9.4
    |    Compiling memchr v2.5.0
    |    Compiling num-traits v0.2.15
    |    Compiling cfg-if v1.0.0
    |    Compiling lock_api v0.4.10
    |    Compiling num-integer v0.1.45
    |    Compiling parking_lot_core v0.9.8
    |    Compiling cc v1.0.79
    |    Compiling tinyvec_macros v0.1.1
    |    Compiling tinyvec v1.6.0
    |    Compiling pyo3-build-config v0.19.1
    |    Compiling syn v2.0.25
    |    Compiling num-bigint v0.4.3
    |    Compiling libmimalloc-sys v0.1.33
    |    Compiling memoffset v0.9.0
    |    Compiling smallvec v1.11.0
    |    Compiling serde v1.0.175
    |    Compiling heck v0.4.1
    |    Compiling scopeguard v1.1.0
    |    Compiling unicode-normalization v0.1.22
    |    Compiling aho-corasick v1.0.2
    |    Compiling ahash v0.8.3
    |    Compiling pyo3-ffi v0.19.1
    |    Compiling pyo3 v0.19.1
    |    Compiling regex-syntax v0.7.4
    |    Compiling hashbrown v0.14.0
    |    Compiling unicode-bidi v0.3.13
    |    Compiling equivalent v1.0.1
    |    Compiling serde_json v1.0.103
    |    Compiling percent-encoding v2.3.0
    |    Compiling serde_derive v1.0.175
    |    Compiling form_urlencoded v1.2.0
    |    Compiling pyo3-macros-backend v0.19.1
    |    Compiling indexmap v2.0.0
    |    Compiling regex-automata v0.3.2
    |    Compiling idna v0.4.0
    |    Compiling strum_macros v0.25.1
    |    Compiling pydantic-core v2.4.0 (/tmp/tmpyqm8ny82/pydantic_core-2.4.0)
    |    Compiling parking_lot v0.12.1
    |    Compiling getrandom v0.2.10
    |    Compiling unindent v0.1.11
    |    Compiling indoc v1.0.9
    |    Compiling ryu v1.0.14
    |    Compiling pyo3-macros v0.19.1
    |    Compiling itoa v1.0.8
    |    Compiling regex v1.9.1
    | error: failed to run custom build command for `pydantic-core v2.4.0 (/tmp/tmpyqm8ny82/pydantic_core-2.4.0)`
    |
    | Caused by:
    |   process didn't exit successfully: `/tmp/tmpyqm8ny82/pydantic_core-2.4.0/target/release/build/pydantic-core-06828f5d5abcd2c7/build-script-build` (exit status: 101)
    |   --- stdout
    |   cargo:rustc-cfg=Py_3_6
    |   cargo:rustc-cfg=Py_3_7
    |   cargo:rustc-cfg=Py_3_8
    |   cargo:rustc-cfg=Py_3_9
    |   cargo:rustc-cfg=Py_3_10
    |   cargo:rustc-cfg=Py_3_11
    |   cargo:rustc-cfg=Py_3_12
    |   cargo:rustc-cfg=Py_3_13
    |   cargo:rerun-if-changed=python/pydantic_core/core_schema.py
    |   cargo:rerun-if-changed=generate_self_schema.py
    |
    |   --- stderr
    |   Traceback (most recent call last):
    |     File "/tmp/tmpyqm8ny82/pydantic_core-2.4.0/generate_self_schema.py", line 176, in eval_forward_ref
    |       return type_._evaluate(core_schema.__dict__, None, set())
    |              ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   TypeError: ForwardRef._evaluate() missing 1 required keyword-only argument: 'recursive_guard'
    |
    |   During handling of the above exception, another exception occurred:
    |
    |   Traceback (most recent call last):
    |     File "/tmp/tmpyqm8ny82/pydantic_core-2.4.0/generate_self_schema.py", line 216, in <module>
    |       main()
    |       ~~~~^^
    |     File "/tmp/tmpyqm8ny82/pydantic_core-2.4.0/generate_self_schema.py", line 192, in main
    |       value = get_schema(s)
    |     File "/tmp/tmpyqm8ny82/pydantic_core-2.4.0/generate_self_schema.py", line 52, in get_schema
    |       return type_dict_schema(obj)
    |     File "/tmp/tmpyqm8ny82/pydantic_core-2.4.0/generate_self_schema.py", line 137, in type_dict_schema
    |       field_type = eval_forward_ref(field_type)
    |     File "/tmp/tmpyqm8ny82/pydantic_core-2.4.0/generate_self_schema.py", line 179, in eval_forward_ref
    |       return type_._evaluate(core_schema.__dict__, None)
    |              ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   TypeError: ForwardRef._evaluate() missing 1 required keyword-only argument: 'recursive_guard'
    |   thread 'main' panicked at 'generate_self_schema.py failed with exit status: 1', build.rs:29:9
    |   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    | warning: build failed, waiting for other jobs to finish...
    | 💥 maturin failed
    |   Caused by: Failed to build a native library through cargo
    |   Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.13-64bit" PYO3_PYTHON="/tmp/tmp8m0p5yy0/.venv/bin/python" PYTHON_SYS_EXECUTABLE="/tmp/tmp8m0p5yy0/.venv/bin/python" "cargo" "rustc" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/tmpyqm8ny82/pydantic_core-2.4.0/Cargo.toml" "--release" "--lib" "--crate-type" "cdylib"`
    | Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/tmp/tmp8m0p5yy0/.venv/bin/python', '--compatibility', 'off'] returned non-zero exit status 1

Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with pydantic-core (2.4.0)

  - not supporting PEP 517 builds
  - not specifying PEP 517 build requirements correctly
  - the build requirements are incompatible with your operating system or Python version
  - the build requirements are missing system dependencies (eg: compilers, libraries, headers).

You can verify this by running pip wheel --no-cache-dir --use-pep517 "pydantic-core (==2.4.0)".

This can be a system problem, I need to study this problem

@peppelinux
Copy link
Member Author

yeah, weird

@MdreW
Copy link
Collaborator

MdreW commented Jan 27, 2025

@peppelinux , simplified poetry dockerfile (start from python-xx-alpine):

FROM python:3.13-alpine

# Metadata params
ARG BUILD_DATE
ARG VERSION
ARG VCS_URL="https://github.com/italia/iam-proxy-italia.git"
ARG VCS_REF
ARG AUTHORS
ARG VENDOR

# Metadata : https://github.com/opencontainers/image-spec/blob/main/annotations.md
LABEL org.opencontainers.image.authors=$AUTHORS \
      org.opencontainers.image.vendor=$VENDOR \
      org.opencontainers.image.title="iam-proxy-italia" \
      org.opencontainers.image.created=$BUILD_DATE \
      org.opencontainers.image.version=$VERSION \
      org.opencontainers.image.source=$VCS_URL \
      org.opencontainers.image.revision=$VCS_REF \
      org.opencontainers.image.description="Docker Image di iam-proxy-italia."

RUN apk add --update --no-cache tzdata mailcap poetry

WORKDIR /satosa_proxy
ADD poetry.lock pyproject.toml ./
RUN poetry lock && poetry install

P.S. must be add !poetry.lock and !pyproject.toml in .dockerignore file

@peppelinux
Copy link
Member Author

This PR must use a static venv path, according to the requirement shown below
https://github.com/italia/iam-proxy-italia/pull/181/files#r1930443551

@MdreW
Copy link
Collaborator

MdreW commented Jan 27, 2025

This PR must use a static venv path, according to the requirement shown below https://github.com/italia/iam-proxy-italia/pull/181/files#r1930443551

We can add poetry config virtualenvs.in-project true to create the virtual env directory in the project/.env path and move poetry.lock, pyproject.toml in root directory to make root as project directory.

FROM python:3.13-alpine

# Metadata params
ARG BUILD_DATE
ARG VERSION
ARG VCS_URL="https://github.com/italia/iam-proxy-italia.git"
ARG VCS_REF
ARG AUTHORS
ARG VENDOR

# Metadata : https://github.com/opencontainers/image-spec/blob/main/annotations.md
LABEL org.opencontainers.image.authors=$AUTHORS \
      org.opencontainers.image.vendor=$VENDOR \
      org.opencontainers.image.title="iam-proxy-italia" \
      org.opencontainers.image.created=$BUILD_DATE \
      org.opencontainers.image.version=$VERSION \
      org.opencontainers.image.source=$VCS_URL \
      org.opencontainers.image.revision=$VCS_REF \
      org.opencontainers.image.description="Docker Image di iam-proxy-italia."

RUN apk add --update --no-cache tzdata mailcap poetry
RUN poetry config virtualenvs.in-project true
ADD poetry.lock pyproject.toml /
RUN poetry lock && poetry install
WORKDIR /satosa_proxy

Now I'm in train, when have a good internet connection I try

@peppelinux
Copy link
Member Author

@MdreW static env path here: 9087786

@peppelinux
Copy link
Member Author

@MdreW dockerfile updated here: ed63635

@peppelinux peppelinux marked this pull request as draft January 28, 2025 16:20
@peppelinux peppelinux marked this pull request as ready for review January 28, 2025 18:30
@peppelinux
Copy link
Member Author

@MdreW I have fixed all the issues, including the one about poetry and several other deprecations and stuffs foudn in a total state of regression

@peppelinux peppelinux merged commit 4ec113a into master Jan 29, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants