Comments (12)
Line 16 in a017bd9
You might have some luck setting the PYTHON
env var., or looking at the PR that changed that code.
My guess is the poetry is supposed to set the env var, or that's what we assumed.
I think we should probably build the core schema python file and include it in git or sdist, and just check it's right on pre-commit.
from pydantic-core.
You shouldn't need to install typing-extensions first, it's a build requirement
Line 4 in a017bd9
from pydantic-core.
typing-extensions is correctly installed in the isolated build environment per PEP517, but then the pydantic-core build chooses to use a different python to execute that script.
from pydantic-core.
per python-poetry/poetry#9518 (comment), this isn't a poetry thing, you can reproduce the same with python -m build
from pydantic-core.
Yup, like the link I sent above, we use the default python3
executable, or the value of the PYTHON
env var to run the script that builds the core schema python definitions.
Happy to accept a PR to improve that logic, or store that definition in version control, or pre-build it for sdist.
from pydantic-core.
you do not use the default python3 executable, that would be the one in the isolated build virtual environment.
I assume that pyo3_build_config::get().executable
is somehow giving an undesirable answer
from pydantic-core.
You shouldn't need to install typing-extensions first, it's a build requirement
I know, only showed that build order issue is not exists.
Build with PYTHON
env variable works.
$ poetry install --no-root
Installing dependencies from lock file
Package operations: 4 installs, 0 updates, 0 removals
- Installing typing-extensions (4.12.2)
- Installing annotated-types (0.7.0)
- Installing pydantic-core (2.14.5): Failed
ChefBuildError
Backend subprocess exited when trying to invoke build_wheel
Running `maturin pep517 build-wheel -i /usr/bin/python3.11 --compatibility off`
Updating crates.io index
Downloading crates ...
Downloaded equivalent v1.0.1
Downloaded unindent v0.2.3
Downloaded strum v0.25.0
Downloaded tinyvec_macros v0.1.1
Downloaded pyo3-macros v0.20.0
Downloaded windows-targets v0.48.1
Downloaded speedate v0.13.0
Downloaded once_cell v1.18.0
Downloaded num-integer v0.1.45
Downloaded autocfg v1.1.0
Downloaded cfg-if v1.0.0
Downloaded indoc v2.0.4
Downloaded heck v0.4.1
Downloaded form_urlencoded v1.2.0
Downloaded memoffset v0.9.0
Downloaded static_assertions v1.1.0
Downloaded rustversion v1.0.13
Downloaded percent-encoding v2.3.0
Downloaded itoa v1.0.8
Downloaded redox_syscall v0.3.5
Downloaded strum_macros v0.25.3
Downloaded version_check v0.9.4
Downloaded quote v1.0.29
Downloaded parking_lot_core v0.9.8
Downloaded bitflags v1.3.2
Downloaded wasi v0.11.0+wasi-snapshot-preview1
Downloaded lexical-parse-integer v0.8.6
Downloaded smallvec v1.11.1
Downloaded enum_dispatch v0.3.12
Downloaded pyo3-build-config v0.20.0
Downloaded target-lexicon v0.12.9
Downloaded scopeguard v1.1.0
Downloaded ahash v0.8.6
Downloaded lexical-core v0.8.5
Downloaded getrandom v0.2.10
Downloaded ryu v1.0.14
Downloaded proc-macro2 v1.0.69
Downloaded uuid v1.5.0
Downloaded unicode-ident v1.0.10
Downloaded lock_api v0.4.10
Downloaded parking_lot v0.12.1
Downloaded serde_derive v1.0.190
Downloaded indexmap v2.0.0
Downloaded pyo3-macros-backend v0.20.0
Downloaded num-traits v0.2.16
Downloaded lexical-write-integer v0.8.5
Downloaded unicode-bidi v0.3.13
Downloaded cc v1.0.79
Downloaded base64 v0.21.5
Downloaded tinyvec v1.6.0
Downloaded python3-dll-a v0.2.9
Downloaded serde v1.0.190
Downloaded pyo3-ffi v0.20.0
Downloaded jiter v0.0.4
Downloaded lexical-util v0.8.5
Downloaded url v2.4.1
Downloaded lexical-write-float v0.8.5
Downloaded num-bigint v0.4.4
Downloaded memchr v2.6.3
Downloaded hashbrown v0.14.0
Downloaded zerocopy v0.7.20
Downloaded unicode-normalization v0.1.22
Downloaded aho-corasick v1.0.2
Downloaded serde_json v1.0.108
Downloaded lexical-parse-float v0.8.5
Downloaded syn v2.0.38
Downloaded idna v0.4.0
Downloaded regex v1.10.2
Downloaded regex-syntax v0.8.2
Downloaded windows_x86_64_gnullvm v0.48.0
Downloaded windows_aarch64_gnullvm v0.48.0
Downloaded pyo3 v0.20.0
Downloaded zerocopy-derive v0.7.20
Downloaded windows_x86_64_msvc v0.48.0
Downloaded windows_aarch64_msvc v0.48.0
Downloaded windows_i686_msvc v0.48.0
Downloaded windows_i686_gnu v0.48.0
Downloaded libc v0.2.147
Downloaded windows_x86_64_gnu v0.48.0
Downloaded regex-automata v0.4.3
📦 Including license file "/tmp/tmph7u3k8qd/pydantic_core-2.14.5/LICENSE"
🍹 Building a mixed python/rust project
🔗 Found pyo3 bindings
🐍 Found CPython 3.11 at /usr/bin/python3.11
📡 Using build options features, bindings from pyproject.toml
Compiling autocfg v1.1.0
Compiling proc-macro2 v1.0.69
Compiling quote v1.0.29
Compiling target-lexicon v0.12.9
Compiling unicode-ident v1.0.10
Compiling python3-dll-a v0.2.9
Compiling once_cell v1.18.0
Compiling libc v0.2.147
Compiling heck v0.4.1
Compiling static_assertions v1.1.0
Compiling version_check v0.9.4
Compiling cfg-if v1.0.0
Compiling rustversion v1.0.13
Compiling parking_lot_core v0.9.8
Compiling lexical-util v0.8.5
Compiling tinyvec_macros v0.1.1
Compiling scopeguard v1.1.0
Compiling smallvec v1.11.1
Compiling tinyvec v1.6.0
Compiling serde v1.0.190
Compiling memchr v2.6.3
Compiling unicode-bidi v0.3.13
Compiling regex-syntax v0.8.2
Compiling hashbrown v0.14.0
Compiling serde_json v1.0.108
Compiling num-traits v0.2.16
Compiling num-integer v0.1.45
Compiling lock_api v0.4.10
Compiling memoffset v0.9.0
Compiling num-bigint v0.4.4
Compiling pyo3-build-config v0.20.0
Compiling syn v2.0.38
Compiling ahash v0.8.6
Compiling getrandom v0.2.10
Compiling aho-corasick v1.0.2
Compiling parking_lot v0.12.1
Compiling lexical-write-integer v0.8.5
Compiling lexical-parse-integer v0.8.6
Compiling zerocopy v0.7.20
Compiling indoc v2.0.4
Compiling percent-encoding v2.3.0
Compiling equivalent v1.0.1
Compiling unindent v0.2.3
Compiling unicode-normalization v0.1.22
Compiling lexical-write-float v0.8.5
Compiling lexical-parse-float v0.8.5
Compiling form_urlencoded v1.2.0
Compiling indexmap v2.0.0
Compiling ryu v1.0.14
Compiling itoa v1.0.8
Compiling uuid v1.5.0
Compiling base64 v0.21.5
Compiling idna v0.4.0
Compiling lexical-core v0.8.5
Compiling url v2.4.1
Compiling regex-automata v0.4.3
Compiling pyo3-ffi v0.20.0
Compiling pyo3 v0.20.0
Compiling pydantic-core v2.14.5 (/tmp/tmph7u3k8qd/pydantic_core-2.14.5)
error: failed to run custom build command for `pydantic-core v2.14.5 (/tmp/tmph7u3k8qd/pydantic_core-2.14.5)`
Caused by:
process didn't exit successfully: `/tmp/tmph7u3k8qd/pydantic_core-2.14.5/target/release/build/pydantic-core-5e5a4926f685a323/build-script-build` (exit status: 101)
--- stdout
cargo:rustc-cfg=Py_3_6
cargo:rustc-cfg=Py_3_7
cargo:rustc-cfg=Py_3_8
cargo:rustc-cfg=Py_3_9
cargo:rustc-cfg=Py_3_10
cargo:rustc-cfg=Py_3_11
cargo:rerun-if-changed=python/pydantic_core/core_schema.py
cargo:rerun-if-changed=generate_self_schema.py
--- stderr
Traceback (most recent call last):
File "/tmp/tmph7u3k8qd/pydantic_core-2.14.5/generate_self_schema.py", line 17, in <module>
from typing_extensions import TypedDict, get_args, get_origin, is_typeddict
ModuleNotFoundError: No module named 'typing_extensions'
thread 'main' panicked at build.rs:29:9:
generate_self_schema.py failed with exit status: 1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
Caused by: Failed to build a native library through cargo
Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.11-64bit" PYO3_PYTHON="/usr/bin/python3.11" PYTHON_SYS_EXECUTABLE="/usr/bin/python3.11" "cargo" "rustc" "--features" "pyo3/extension-module" "--message-format" "json-render-diagnostics" "--manifest-path" "/tmp/tmph7u3k8qd/pydantic_core-2.14.5/Cargo.toml" "--release" "--lib" "--crate-type" "cdylib"`
Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/usr/bin/python3.11', '--compatibility', 'off'] returned non-zero exit status 1
at /opt/poetry/venv/lib/python3.11/site-packages/poetry/installation/chef.py:164 in _prepare
160│
161│ error = ChefBuildError("\n\n".join(message_parts))
162│
163│ if error is not None:
→ 164│ raise error from None
165│
166│ return path
167│
168│ def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:
Note: This error originates from the build backend, and is likely not a problem with poetry but with pydantic-core (2.14.5) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "pydantic-core (==2.14.5)"'.
$ which python3
/venv/bin/python3
$ PYTHON=/venv/bin/python3 poetry install --no-root
Installing dependencies from lock file
Package operations: 2 installs, 0 updates, 0 removals
- Installing pydantic-core (2.14.5)
- Installing pydantic (2.5.2)
$ poetry update --dry-run
Updating dependencies
Resolving dependencies... (0.3s)
Package operations: 0 installs, 0 updates, 0 removals, 4 skipped
- Installing annotated-types (0.7.0): Skipped for the following reason: Already installed
- Installing pydantic (2.5.2): Skipped for the following reason: Already installed
- Installing pydantic-core (2.14.5): Skipped for the following reason: Already installed
- Installing typing-extensions (4.12.2): Skipped for the following reason: Already installed
from pydantic-core.
It sounds like the issue is with pyo3_build_config::get().executable
then.
Related to #674. @davidhewitt is on pat leave right now, but might have some thoughts when he's back. You could perhaps create an issue on https://github.com/pyo3/pyo3, although I know you've already going quite far down the rabbit whole on this one.
I think that the only fix open to pydantic-core is to pre-build self_schema.py
, and thereby avoid the need to involve python at install time at all.
from pydantic-core.
maybe these two tries at finding a python should simply be swapped so that the python in the path is found first
Lines 18 to 19 in a017bd9
edit: not quite so simple as "simply swapped" of course, would need to check that looking for a python in the path did find something
from pydantic-core.
I think pyo3_build_config::get().executable
will be right more often than just guessing at 'python3'
.
Given that #764 was introduced a year ago, and AFAIK this is the first issue reported, I suspect what we have is working pretty well.
I don't want to introduce more and more logic to find the right python executable, let's either fix pyo3_build_config::get().executable
or avoid the problem completely by pre-building self_schema.py
.
from pydantic-core.
I believe it is PyO3/maturin#2094 at the bottom of the rabbit-hole. maturin
deliberately looks for the base python interpreter, rather than the one in the virtual environment.
That propagates via PYO3_PYTHON
through pyo3
and then into the pydantic-core
build script.
My feeling is that the maturin pull request was simply wrong - it directly contradicts this comment that it left unchanged, and seems rather to miss the point of setting up an isolated build environment with specified requirements.
Alternatively perhaps maturin is implicitly requiring that projects do not do as pydantic-core is doing ie actually use that python interpreter. But then they seem to be going to a lot of trouble to pass around an interpreter that should not be used...
from pydantic-core.
Yep this has now been fixed in maturin
.
from pydantic-core.
Related Issues (20)
- fail-fast list schema configuration
- Make better union validation decisions based on `extra` behavior HOT 6
- Specify value in UserWarning "serialized value may not be as expected" HOT 2
- Proposal for mapping Python types to CombinedValidator HOT 13
- Numerically constrainted fields don't generate proper error on non-Number fields HOT 1
- Making `ObType` a separate crate
- ValidationError missing reference to model being validated
- Fail-fast feature HOT 2
- [pydantic #9683] @field_serializer("*") is not compatible with @computed_field when expecting FieldSerializationInfo
- SIGSEGV Linux arm64, python3.13-nogil HOT 2
- could not compile `thiserror` (lib) due to 1 previous error HOT 17
- Creating Pydantic objects in Rust and passing to the interpreter. HOT 4
- Build directory (as a string) injected into _pydantic-core.*.so (which will cause non-reproducible builds) HOT 5
- Pydantic-core 2.10.1 installation is failing in Python 3.12.4 and Rustc 1.79.0 HOT 1
- Inconsistent Behavior in Pattern Matching String HOT 1
- Improvement in generic schema generation HOT 1
- Field pattern regex validation fails inconsistent to constr HOT 1
- Customize Pydantic Error Type (AKA subclassing PydanticCustomError) HOT 1
- Improved Documentation
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pydantic-core.