pdm-project / pdm-backend Goto Github PK
View Code? Open in Web Editor NEWThe build backend used by PDM that supports latest packaging standards.
Home Page: https://backend.pdm-project.org
License: MIT License
The build backend used by PDM that supports latest packaging standards.
Home Page: https://backend.pdm-project.org
License: MIT License
I didn't see any effort into supporting PEP 660 in setuptools. So I will provide one in pdm.pep517
, adapt from https://github.com/dholth/setuptools_pep660/
Currently the sdist PKG_INFO produced by pdm517
carries the whole long description in the header Description
.
Given that it support version 2.1 of the metadata format, wouldn't it be reasonable to use the new way of enclosing it? It would save some bytes (the " " * 8
indent) if anything else, and looks better.
Our infra uses a top level build/
directory for our internal build artifacts and metadata. Apparently WheelBuilder
essentially hard codes <cwd>/build/
for its build directory too. What makes this a fatal problem is this code in wheel.py
:
def _build(self) -> None:
build_dir = self.location / "build"
if build_dir.exists():
shutil.rmtree(str(build_dir))
The effect of this is that if I do pdm build --no-wheel
my build works fine because the necessary metadata in build/
is preserved. But if I do build the wheel (i.e. no arg, or --no-sdist
, my build/
directory gets blown away and the metadata that my pre_build
script creates won't exist by the time the wheel is built, thus failing the build.
I can't find a configuration or pyproject.toml
setting to change the wheel build directory.
Similar to setuptools-scm
's write_to
, it would be great to be able to write the generated version to a file.
[build-system]
requires = ["pdm-pep517"]
build-backend = "pdm.pep517.api"
My project is for a company, and the package that I try to install has a proprietary license. According to PEP 621, the license field can be text
or file
. I used to have a text that was working about a couple of weeks ago, but last week I noted there were problems to do a pdm install
of my package.
I checked the log and the pdm-pep517 was unable to recognize the license that I have. The license is similar to the following expression:
license = {text = "(C) Copyright 2016-2022 My Great Company. All rights reserved."}
There was absolutely no text value that I could use to appease the license validator. It seems that this module is trying to enforce PEP-639, which is still a draft.
If I remove the license field, pdm install
would work. It would also work if I use a file instead of a text:
license = {file = "LICENSE"}
I noted that I can use PEP 639, but doing pdm show
after pdm install
shows None for license
. I used the following fields for trying PEP-639 instead of the use of the license
field:
license-expression = "LicenseRef-Proprietary"
license-files = {paths = ["LICENSE"]}
I think PEP-621 should be permitted as-is while PEP-639 becomes final.
Any way, I can continue working on my project, and I resolved this issue for the time being. But, perhaps, other people may find the same issue that I document here.
Thanks for the great work!
The toml
package is no longer maintained (no activity since Nov 2020) and does not conform to TOML 1.0.0 specification (it still uses 0.5.0). Please consider replacing it with a maintained TOML library. tomli
seems to be a common choice among other projects.
I accidentally put a source-includes
field in the [project]
section of my pyproject.toml instead of [tool.pdm]
. For versions of pdm-pep517 prior to 0.11.2 this was fine, but 0.12.0 up to the current 0.12.3, PDM gives the error below.
Would it be possible to print line number and offending line (on stdout instead of in the log) when this happens? Even better, if we know that a field is correct but in the wrong section, the error could mention that.
Collecting pdm-pep517
Using cached pdm_pep517-0.12.3-py3-none-any.whl (302 kB)
Installing collected packages: pdm-pep517
Successfully installed pdm-pep517-0.12.3
Traceback (most recent call last):
File "/Users/dlipsitt/Library/Application Support/pdm/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/Users/dlipsitt/Library/Application Support/pdm/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/Users/dlipsitt/Library/Application Support/pdm/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 283, in build_editable
return hook(wheel_directory, config_settings, metadata_directory)
File "/var/folders/m9/d2t2rmwx7rz10l9dvlsb35540000gq/T/pdm-build-env-eluxduco-shared/lib/python3.9/site-packages/pdm/pep517/api.py", line 97, in build_editable
with EditableBuilder(Path.cwd(), config_settings) as builder:
File "/var/folders/m9/d2t2rmwx7rz10l9dvlsb35540000gq/T/pdm-build-env-eluxduco-shared/lib/python3.9/site-packages/pdm/pep517/editable.py", line 75, in __init__
to_filename(self.meta.project_name), self.location.as_posix()
File "/var/folders/m9/d2t2rmwx7rz10l9dvlsb35540000gq/T/pdm-build-env-eluxduco-shared/lib/python3.9/site-packages/pdm/pep517/base.py", line 144, in meta
self._meta.validate(True)
File "/var/folders/m9/d2t2rmwx7rz10l9dvlsb35540000gq/T/pdm-build-env-eluxduco-shared/lib/python3.9/site-packages/pdm/pep517/metadata.py", line 87, in validate
return validate_pep621(self._metadata, raising)
File "/var/folders/m9/d2t2rmwx7rz10l9dvlsb35540000gq/T/pdm-build-env-eluxduco-shared/lib/python3.9/site-packages/pdm/pep517/validator.py", line 85, in validate_pep621
raise PEP621ValidationError(validator.errors)
pdm.pep517.exceptions.PEP621ValidationError: {'source-includes': ['unknown field']}
When building extension modules, it's a common idiom to create a Python wrapper with the public name, and name the extension module with a leading underscore. For example, _extension.c
produces an extension module named _extension
but you expose this low-level API through an extension.py
module, which is the public interface.
If you use editable-backend = 'editables'
(the default), this breaks because editables creates a proxy for the public module name, prepending _
so when you try to import _extension
, you actually get the _extension.py
and not the `_extension-blah.so extension module.
Changing editable-backend = 'path'
fixes this.
Create a package with an extension.py
file that imports _extension
. Create your extension module and call it _extension.c
. Build with pdm where editable-backend = 'editables'
and then try to import _extension
. You'll get the editables crafted _extension.py
file instead of the _extension
extension module.
I don't think pdm
actually should change, as I think this is a behavior of editables
. I don't know if that package has a workaround, but a quick glance at its tracker doesn't indicate that this issue has been reported before. You or I could certainly do that.
I think pdm
should just mention this in the editable-backend
documentation. It already mentions that 'path' is required if you're using namespace packages, so it should just also note that 'path' is required when you have this kind of extension module idiom.
# Paste the output of `pdm info && pdm info --env` below:
setuptools 61.0.0 introduced pyproject.toml
support with some validation on top. As a result, a few tests are now failing due to pyproject.toml
validation errors:
$ pdm run pytest tests
========================================================= test session starts =========================================================
platform linux -- Python 3.10.4, pytest-7.1.1, pluggy-1.0.0
rootdir: /tmp/pdm-pep517, configfile: pyproject.toml
plugins: forked-1.4.0, anyio-3.5.0, xprocess-0.18.1, xdist-2.5.0, pkgcore-0.12.10
collected 67 items
tests/test_api.py .......x.FF....F. [ 25%]
tests/test_file_finder.py ................ [ 49%]
tests/test_metadata.py ..................... [ 80%]
tests/test_validator.py ........... [ 97%]
tests/test_wheel.py .. [100%]
============================================================== FAILURES ===============================================================
_____________________________________________________ test_build_with_cextension ______________________________________________________
self = <pdm.pep517.wheel.WheelBuilder object at 0x7f7f6954f430>, wheel = <zipfile.ZipFile [closed]>
def _build(self, wheel: zipfile.ZipFile) -> None:
if not self.meta.build:
return
setup_py = self.ensure_setup_py()
build_args = [
sys.executable,
str(setup_py),
"build",
"-b",
str(self.location / "build"),
]
try:
> subprocess.check_call(build_args)
pdm/pep517/wheel.py:194:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
popenargs = (['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/build'],)
kwargs = {}, retcode = 1
cmd = ['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/build']
def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the call function. Example:
check_call(["ls", "-l"])
"""
retcode = call(*popenargs, **kwargs)
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
> raise CalledProcessError(retcode, cmd)
E subprocess.CalledProcessError: Command '['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/build']' returned non-zero exit status 1.
/usr/lib/python3.10/subprocess.py:369: CalledProcessError
During handling of the above exception, another exception occurred:
tmp_path = PosixPath('/tmp/pytest-of-mgorny/pytest-1/test_build_with_cextension0')
def test_build_with_cextension(tmp_path: Path) -> None:
with build_fixture_project("demo-cextension"):
> wheel_name = api.build_wheel(tmp_path.as_posix())
tests/test_api.py:173:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pdm/pep517/api.py:69: in build_wheel
return Path(builder.build(wheel_directory)).name
pdm/pep517/wheel.py:76: in build
self._build(zip_file)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pdm.pep517.wheel.WheelBuilder object at 0x7f7f6954f430>, wheel = <zipfile.ZipFile [closed]>
def _build(self, wheel: zipfile.ZipFile) -> None:
if not self.meta.build:
return
setup_py = self.ensure_setup_py()
build_args = [
sys.executable,
str(setup_py),
"build",
"-b",
str(self.location / "build"),
]
try:
subprocess.check_call(build_args)
except subprocess.CalledProcessError as e:
> raise BuildError(f"Error occurs when running {build_args}:\n{e}")
E pdm.pep517.exceptions.BuildError: Error occurs when running ['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/build']:
E Command '['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/build']' returned non-zero exit status 1.
pdm/pep517/wheel.py:196: BuildError
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
- Adding my_package/__init__.py
GIVEN VALUE:
""
OFFENDING RULE: 'format'
DEFINITION:
{
"type": "string",
"format": "url"
}
-------------------------------------------------------- Captured stderr call ---------------------------------------------------------
/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:102: _ExperimentalProjectMetadata: Support for project metadata in `pyproject.toml` is still experimental and may be removed (or change) in future releases.
warnings.warn(msg, _ExperimentalProjectMetadata)
For maximum compatibility please make sure to include a `scheme` prefix in your URL (e.g. 'http://'). Given value:
configuration error: `project.urls.homepage` must be url
Traceback (most recent call last):
File "/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension/setup.py", line 29, in <module>
setup(**setup_kwargs)
File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 122, in setup
dist.parse_config_files()
File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 854, in parse_config_files
pyprojecttoml.apply_configuration(self, filename, ignore_option_errors)
File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 54, in apply_configuration
config = read_configuration(filepath, True, ignore_option_errors, dist)
File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 120, in read_configuration
validate(subset, filepath)
File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 43, in validate
raise error from None
ValueError: invalid pyproject.toml config: `project.urls.homepage`
__________________________________________________ test_build_with_cextension_in_src __________________________________________________
self = <pdm.pep517.wheel.WheelBuilder object at 0x7f7f695e0f40>, wheel = <zipfile.ZipFile [closed]>
def _build(self, wheel: zipfile.ZipFile) -> None:
if not self.meta.build:
return
setup_py = self.ensure_setup_py()
build_args = [
sys.executable,
str(setup_py),
"build",
"-b",
str(self.location / "build"),
]
try:
> subprocess.check_call(build_args)
pdm/pep517/wheel.py:194:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
popenargs = (['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/build'],)
kwargs = {}, retcode = 1
cmd = ['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/build']
def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the call function. Example:
check_call(["ls", "-l"])
"""
retcode = call(*popenargs, **kwargs)
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
> raise CalledProcessError(retcode, cmd)
E subprocess.CalledProcessError: Command '['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/build']' returned non-zero exit status 1.
/usr/lib/python3.10/subprocess.py:369: CalledProcessError
During handling of the above exception, another exception occurred:
tmp_path = PosixPath('/tmp/pytest-of-mgorny/pytest-1/test_build_with_cextension_in_0')
def test_build_with_cextension_in_src(tmp_path: Path) -> None:
with build_fixture_project("demo-cextension-in-src"):
> wheel_name = api.build_wheel(tmp_path.as_posix())
tests/test_api.py:196:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pdm/pep517/api.py:69: in build_wheel
return Path(builder.build(wheel_directory)).name
pdm/pep517/wheel.py:76: in build
self._build(zip_file)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pdm.pep517.wheel.WheelBuilder object at 0x7f7f695e0f40>, wheel = <zipfile.ZipFile [closed]>
def _build(self, wheel: zipfile.ZipFile) -> None:
if not self.meta.build:
return
setup_py = self.ensure_setup_py()
build_args = [
sys.executable,
str(setup_py),
"build",
"-b",
str(self.location / "build"),
]
try:
subprocess.check_call(build_args)
except subprocess.CalledProcessError as e:
> raise BuildError(f"Error occurs when running {build_args}:\n{e}")
E pdm.pep517.exceptions.BuildError: Error occurs when running ['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/build']:
E Command '['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/build']' returned non-zero exit status 1.
pdm/pep517/wheel.py:196: BuildError
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
- Adding my_package/__init__.py
GIVEN VALUE:
""
OFFENDING RULE: 'format'
DEFINITION:
{
"type": "string",
"format": "url"
}
-------------------------------------------------------- Captured stderr call ---------------------------------------------------------
/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:102: _ExperimentalProjectMetadata: Support for project metadata in `pyproject.toml` is still experimental and may be removed (or change) in future releases.
warnings.warn(msg, _ExperimentalProjectMetadata)
For maximum compatibility please make sure to include a `scheme` prefix in your URL (e.g. 'http://'). Given value:
configuration error: `project.urls.homepage` must be url
Traceback (most recent call last):
File "/tmp/pdm-pep517/tests/fixtures/projects/demo-cextension-in-src/setup.py", line 30, in <module>
setup(**setup_kwargs)
File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 122, in setup
dist.parse_config_files()
File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 854, in parse_config_files
pyprojecttoml.apply_configuration(self, filename, ignore_option_errors)
File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 54, in apply_configuration
config = read_configuration(filepath, True, ignore_option_errors, dist)
File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 120, in read_configuration
validate(subset, filepath)
File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 43, in validate
raise error from None
ValueError: invalid pyproject.toml config: `project.urls.homepage`
________________________________________________ test_build_purelib_project_with_build ________________________________________________
self = <pdm.pep517.wheel.WheelBuilder object at 0x7f7f696d8b20>, wheel = <zipfile.ZipFile [closed]>
def _build(self, wheel: zipfile.ZipFile) -> None:
if not self.meta.build:
return
setup_py = self.ensure_setup_py()
build_args = [
sys.executable,
str(setup_py),
"build",
"-b",
str(self.location / "build"),
]
try:
> subprocess.check_call(build_args)
pdm/pep517/wheel.py:194:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
popenargs = (['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/build'],)
kwargs = {}, retcode = 1
cmd = ['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/build']
def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the call function. Example:
check_call(["ls", "-l"])
"""
retcode = call(*popenargs, **kwargs)
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
> raise CalledProcessError(retcode, cmd)
E subprocess.CalledProcessError: Command '['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/build']' returned non-zero exit status 1.
/usr/lib/python3.10/subprocess.py:369: CalledProcessError
During handling of the above exception, another exception occurred:
tmp_path = PosixPath('/tmp/pytest-of-mgorny/pytest-1/test_build_purelib_project_wit0')
def test_build_purelib_project_with_build(tmp_path: Path) -> None:
with build_fixture_project("demo-purelib-with-build"):
> wheel_name = api.build_wheel(tmp_path.as_posix())
tests/test_api.py:306:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pdm/pep517/api.py:69: in build_wheel
return Path(builder.build(wheel_directory)).name
pdm/pep517/wheel.py:76: in build
self._build(zip_file)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <pdm.pep517.wheel.WheelBuilder object at 0x7f7f696d8b20>, wheel = <zipfile.ZipFile [closed]>
def _build(self, wheel: zipfile.ZipFile) -> None:
if not self.meta.build:
return
setup_py = self.ensure_setup_py()
build_args = [
sys.executable,
str(setup_py),
"build",
"-b",
str(self.location / "build"),
]
try:
subprocess.check_call(build_args)
except subprocess.CalledProcessError as e:
> raise BuildError(f"Error occurs when running {build_args}:\n{e}")
E pdm.pep517.exceptions.BuildError: Error occurs when running ['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/build']:
E Command '['/usr/bin/python3.10', '/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/setup.py', 'build', '-b', '/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/build']' returned non-zero exit status 1.
pdm/pep517/wheel.py:196: BuildError
-------------------------------------------------------- Captured stdout call ---------------------------------------------------------
- Adding my_package/__init__.py
GIVEN VALUE:
""
OFFENDING RULE: 'format'
DEFINITION:
{
"type": "string",
"format": "url"
}
-------------------------------------------------------- Captured stderr call ---------------------------------------------------------
/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:102: _ExperimentalProjectMetadata: Support for project metadata in `pyproject.toml` is still experimental and may be removed (or change) in future releases.
warnings.warn(msg, _ExperimentalProjectMetadata)
For maximum compatibility please make sure to include a `scheme` prefix in your URL (e.g. 'http://'). Given value:
configuration error: `project.urls.homepage` must be url
Traceback (most recent call last):
File "/tmp/pdm-pep517/tests/fixtures/projects/demo-purelib-with-build/setup.py", line 29, in <module>
setup(**setup_kwargs)
File "/usr/lib/python3.10/site-packages/setuptools/__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 122, in setup
dist.parse_config_files()
File "/usr/lib/python3.10/site-packages/setuptools/dist.py", line 854, in parse_config_files
pyprojecttoml.apply_configuration(self, filename, ignore_option_errors)
File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 54, in apply_configuration
config = read_configuration(filepath, True, ignore_option_errors, dist)
File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 120, in read_configuration
validate(subset, filepath)
File "/usr/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py", line 43, in validate
raise error from None
ValueError: invalid pyproject.toml config: `project.urls.homepage`
======================================================= short test summary info =======================================================
FAILED tests/test_api.py::test_build_with_cextension - pdm.pep517.exceptions.BuildError: Error occurs when running ['/usr/bin/python...
FAILED tests/test_api.py::test_build_with_cextension_in_src - pdm.pep517.exceptions.BuildError: Error occurs when running ['/usr/bin...
FAILED tests/test_api.py::test_build_purelib_project_with_build - pdm.pep517.exceptions.BuildError: Error occurs when running ['/usr...
=============================================== 3 failed, 63 passed, 1 xfailed in 7.85s ===============================================
pytest-xprocess reminder::Be sure to terminate the started process by running 'pytest --xkill' if you have not explicitly done so in your fixture with 'xprocess.getinfo(<process_name>).terminate()'.
Hi,
I am trying to convert an entrypoint like this:
https://github.com/cockroachdb/sqlalchemy-cockroachdb/blob/master/setup.py#L41-L46
to pdm:
[project.entry-points.sqlalchemy.dialects]
cockroachdb = "sqlalchemy_cockroachdb.psycopg2:CockroachDBDialect_psycopg2"
Building this fails horribly with:
Successfully installed pdm-pep517-0.12.3
Traceback (most recent call last):
File "/home/florian/.local/pipx/venvs/pdm/lib64/python3.10/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/home/florian/.local/pipx/venvs/pdm/lib64/python3.10/site-packages/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/florian/.local/pipx/venvs/pdm/lib64/python3.10/site-packages/pep517/in_process/_in_process.py", line 314, in build_sdist
return backend.build_sdist(sdist_directory, config_settings)
File "/tmp/pdm-build-env-xdmskv1l-shared/lib/python3.8/site-packages/pdm/pep517/api.py", line 77, in build_sdist
return Path(builder.build(sdist_directory)).name
File "/tmp/pdm-build-env-xdmskv1l-shared/lib/python3.8/site-packages/pdm/pep517/sdist.py", line 60, in build
version = self.meta_version
File "/tmp/pdm-build-env-xdmskv1l-shared/lib/python3.8/site-packages/pdm/pep517/base.py", line 149, in meta_version
meta_version = self.meta.version
File "/tmp/pdm-build-env-xdmskv1l-shared/lib/python3.8/site-packages/pdm/pep517/base.py", line 144, in meta
self._meta.validate(True)
File "/tmp/pdm-build-env-xdmskv1l-shared/lib/python3.8/site-packages/pdm/pep517/metadata.py", line 87, in validate
return validate_pep621(self._metadata, raising)
File "/tmp/pdm-build-env-xdmskv1l-shared/lib/python3.8/site-packages/pdm/pep517/validator.py", line 85, in validate_pep621
raise PEP621ValidationError(validator.errors)
pdm.pep517.exceptions.PEP621ValidationError: {'entry-points': [{'sqlalchemy': [{'dialects': ['must be of string type']}]}]}
I wonder if pdm splits the entrypoints to eagerly here.
According to https://peps.python.org/pep-0517/#in-tree-build-backends:
Projects can specify that their backend code is hosted in-tree by including the backend-path key in pyproject.toml. This key contains a list of directories, which the frontend will add to the start of sys.path when loading the backend, and running the backend hooks.
backend-path
should be a list of strings, not a string.
This results in the build fails, e.g.:
[builder@localhost pdm-pep517]$ python3 -m build .
ERROR Failed to validate `build-system` in pyproject.toml: `backend-path` must be an array of strings
pdm build
fails with IsADirectoryError
if a project complies with the reuse spec.
The spec says
Each License File MUST be placed in the LICENSES/ directory in the root of the Project. The name of the License File MUST be the SPDX License Identifier of the license followed by an appropriate file extension (example: LICENSES/GPL-3.0-or-later.txt). The License File MUST be in plain text format.
With just the default pyproject.toml with a
[project]
license = {text = "MPL-2.0"}
and a MPL-2.0.txt
in a LICENSES
directory at the root, the step the adds the licenses to the wheel tries the LICEN[CS]E*
pattern as a file. But it is a directory.
What is correct here? Should reuse projects just use this:
[project]
license-files = {globs = ["LICENSES/*"]}
On the other hand, hard failing over a filetype issue with an exception is definitely not correct... I'll write a fix for that, and then might as well add the reuse glob pattern to the default ones to try.
I had a malformed requirement of
git+ssh://[email protected]/me/my-project@my-branch
which is a malformed requirement (and should be)
my-project @ git+ssh://[email protected]/me/my-project@my-branch
and an error message of
Traceback (most recent call last):
File "/tmp/pdm-build-env-_3xcxqno-shared/lib/python3.8/site-packages/pdm/pep517/_vendor/packaging/requirements.py", line 102, in __init__
req = REQUIREMENT.parseString(requirement_string)
File "/tmp/pdm-build-env-_3xcxqno-shared/lib/python3.8/site-packages/pdm/pep517/_vendor/pyparsing/core.py", line 1134, in parse_string
raise exc.with_traceback(None)
pdm.pep517._vendor.pyparsing.exceptions.ParseException: Expected string_end, found '+' (at char 3), (line:1, col:4)
A better error message that says something like "invalid requirement at location (location) or something "
I noticed that the sdist PKG_INFO
file does not contain any of the dependencies of the package, neither the mandatory ones nor the optional ones, although it carries the Provides-Extra
field.
Is there any reason this difference between sdist and wheel?
Most PDM commands fail with an assertion failure if the most recent git tag ends in a letter, for example v1.0.0b
.
Here is the source of the error:
https://github.com/pdm-project/pdm-pep517/blob/fa7ef3b7289badda3379fe537fb3436608552075/pdm/pep517/scm.py#L264-L266
I see that the example string is not PEP440 compliant, but I don't think it's reasonable to expect that git tags in general would be. It might be reasonable to try to normalize the version string before bumping it, although the results might be unexpected. That would probably still be better than crashing, though. Alternatively, a better error message would be good.
I found a library called parver that can do version normalization:
>>> parver.Version.parse('v1.1.1beta').normalize()
<Version '1.1.1b0'>
It still wouldn't handle arbitrary strings which means we still need some kind of fallback behavior, but it's an improvement:
>>> parver.Version.parse('beta').normalize()
ParseError: Expected v or int at position (1, 1) => '*beta'.
https://github.com/abersheeran/baize/pull/17/checks?check_run_id=3811820806
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 349, in <module>
main()
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 331, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 270, in build_editable
return hook(wheel_directory, config_settings, metadata_directory)
File "/tmp/pdm-build-env-ix8_99sb-shared/lib/python3.10/site-packages/pdm/pep517/api.py", line 86, in build_editable
return Path(builder.build(wheel_directory)).name
File "/tmp/pdm-build-env-ix8_99sb-shared/lib/python3.10/site-packages/pdm/pep517/wheel.py", line 76, in build
self._write_metadata(zip_file)
File "/tmp/pdm-build-env-ix8_99sb-shared/lib/python3.10/site-packages/pdm/pep517/editable.py", line 155, in _write_metadata
return super()._write_metadata(wheel)
File "/tmp/pdm-build-env-ix8_99sb-shared/lib/python3.10/site-packages/pdm/pep517/wheel.py", line 144, in _write_metadata
self._write_wheel_file(f)
File "/tmp/pdm-build-env-ix8_99sb-shared/lib/python3.10/site-packages/pdm/pep517/wheel.py", line 265, in _write_wheel_file
tag=self.tag,
File "/tmp/pdm-build-env-ix8_99sb-shared/lib/python3.10/site-packages/pdm/pep517/wheel.py", line 121, in tag
assert (
AssertionError: would build wheel with unsupported tag ('cp310', 'cp310', 'linux_x86_64')
Install baize 0.11.0 failed
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/builders/base.py", line 78, in log_subprocessor
subprocess.check_call(
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/subprocess.py", line 364, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/opt/hostedtoolcache/Python/3.10.0/x64/bin/python3.10', '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pep517/in_process/_in_process.py', 'build_editable', '/tmp/tmp1_wbu8wi']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/hostedtoolcache/Python/3.8.12/x64/bin/pdm", line 8, in <module>
sys.exit(main())
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/core.py", line 198, in main
return Core().main(args)
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/core.py", line 153, in main
raise cast(Exception, err).with_traceback(traceback)
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/core.py", line 148, in main
f(options.project, options)
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/cli/commands/install.py", line 45, in handle
actions.do_sync(
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/cli/actions.py", line 169, in do_sync
handler.synchronize()
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/installers/synchronizers.py", line 372, in synchronize
self.install_candidate(self_key)
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/installers/synchronizers.py", line 190, in install_candidate
self.manager.install(can)
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/installers/manager.py", line 38, in install
installer(candidate.build(), self.environment, candidate.direct_url())
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/models/candidates.py", line 307, in build
self.wheel = builder.build(build_dir, metadata_directory=self._metadata_dir)
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/builders/editable.py", line 52, in build
filename = self._hook.build_editable(
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pep517/wrappers.py", line 257, in build_editable
return self._call_hook('build_editable', {
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pep517/wrappers.py", line 318, in _call_hook
self._subprocess_runner(
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/builders/base.py", line 235, in subprocess_runner
return log_subprocessor(cmd, cwd, extra_environ=env)
File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pdm/builders/base.py", line 86, in log_subprocessor
raise BuildError(f"Call command {cmd} return non-zero status.")
pdm.exceptions.BuildError: Call command ['/opt/hostedtoolcache/Python/3.10.0/x64/bin/python3.10', '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/pep517/in_process/_in_process.py', 'build_editable', '/tmp/tmp1_wbu8wi'] return non-zero status.
See discussion in pdm-project/pdm#453.
pdm-pep517 should add the Python 3.6 classifier even when Python 3.6.0 is not contained within the range described by the python-requires
field of the metadata (but others 3.6.x are of course).
I am always trying to bring the bleeding edge of the Python packaging ecosystem to PDM projects. pdm-pep517 is suffering from lacking standards of installing editable packages from PEP 517 and now it is supplemented by PEP 660, as the time of writing, it is still in Draft status.
It is time to implement the interface described in PEP 660, where it suggests two approaches to achieve this:
The first approach is easier and more straight-forward, but is not so accurate, while the second approach requires runtime dependencies
Make sure you run commands with -v
flag before pasting the output.
I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.
python3 -sBm build -w --no-isolation
pep517 build peorduces .whl with 0.0.0
version.
+ /usr/bin/python3 -sBm build -w --no-isolation
* Getting dependencies for wheel...
* Building wheel...
- Adding pdm/__init__.py
- Adding pdm/__main__.py
- Adding pdm/_types.py
- Adding pdm/_vendor/__init__.py
- Adding pdm/_vendor/colorama/LICENSE.txt
- Adding pdm/_vendor/colorama/__init__.py
- Adding pdm/_vendor/colorama/ansi.py
- Adding pdm/_vendor/colorama/ansitowin32.py
- Adding pdm/_vendor/colorama/initialise.py
- Adding pdm/_vendor/colorama/win32.py
- Adding pdm/_vendor/colorama/winterm.py
- Adding pdm/_vendor/halo/LICENSE
- Adding pdm/_vendor/halo/__init__.py
- Adding pdm/_vendor/halo/_utils.py
- Adding pdm/_vendor/halo/cursor.py
- Adding pdm/_vendor/halo/halo.py
- Adding pdm/_vendor/halo/halo_notebook.py
- Adding pdm/_vendor/log_symbols/LICENSE
- Adding pdm/_vendor/log_symbols/__init__.py
- Adding pdm/_vendor/log_symbols/symbols.py
- Adding pdm/_vendor/spinners/LICENSE
- Adding pdm/_vendor/spinners/__init__.py
- Adding pdm/_vendor/spinners/spinners.py
- Adding pdm/_vendor/termcolor.COPYING.txt
- Adding pdm/_vendor/termcolor.py
- Adding pdm/_vendor/vendors.txt
- Adding pdm/builders/__init__.py
- Adding pdm/builders/base.py
- Adding pdm/builders/editable.py
- Adding pdm/builders/sdist.py
- Adding pdm/builders/wheel.py
- Adding pdm/cli/__init__.py
- Adding pdm/cli/actions.py
- Adding pdm/cli/commands/__init__.py
- Adding pdm/cli/commands/add.py
- Adding pdm/cli/commands/base.py
- Adding pdm/cli/commands/build.py
- Adding pdm/cli/commands/cache.py
- Adding pdm/cli/commands/completion.py
- Adding pdm/cli/commands/config.py
- Adding pdm/cli/commands/export.py
- Adding pdm/cli/commands/import_cmd.py
- Adding pdm/cli/commands/info.py
- Adding pdm/cli/commands/init.py
- Adding pdm/cli/commands/install.py
- Adding pdm/cli/commands/list.py
- Adding pdm/cli/commands/lock.py
- Adding pdm/cli/commands/plugin.py
- Adding pdm/cli/commands/remove.py
- Adding pdm/cli/commands/run.py
- Adding pdm/cli/commands/search.py
- Adding pdm/cli/commands/show.py
- Adding pdm/cli/commands/sync.py
- Adding pdm/cli/commands/update.py
- Adding pdm/cli/commands/use.py
- Adding pdm/cli/completions/__init__.py
- Adding pdm/cli/completions/pdm.bash
- Adding pdm/cli/completions/pdm.fish
- Adding pdm/cli/completions/pdm.ps1
- Adding pdm/cli/completions/pdm.zsh
- Adding pdm/cli/options.py
- Adding pdm/cli/utils.py
- Adding pdm/core.py
- Adding pdm/exceptions.py
- Adding pdm/formats/__init__.py
- Adding pdm/formats/base.py
- Adding pdm/formats/flit.py
- Adding pdm/formats/legacy.py
- Adding pdm/formats/pipfile.py
- Adding pdm/formats/poetry.py
- Adding pdm/formats/requirements.py
- Adding pdm/formats/setup_py.py
- Adding pdm/installers/__init__.py
- Adding pdm/installers/installers.py
- Adding pdm/installers/manager.py
- Adding pdm/installers/packages.py
- Adding pdm/installers/synchronizers.py
- Adding pdm/installers/uninstallers.py
- Adding pdm/models/__init__.py
- Adding pdm/models/auth.py
- Adding pdm/models/caches.py
- Adding pdm/models/candidates.py
- Adding pdm/models/environment.py
- Adding pdm/models/in_process/__init__.py
- Adding pdm/models/in_process/get_abi_tag.py
- Adding pdm/models/in_process/pep508.py
- Adding pdm/models/in_process/sysconfig_get_paths.py
- Adding pdm/models/markers.py
- Adding pdm/models/pip_shims.py
- Adding pdm/models/project_info.py
- Adding pdm/models/python.py
- Adding pdm/models/python_max_versions.json
- Adding pdm/models/repositories.py
- Adding pdm/models/requirements.py
- Adding pdm/models/setup.py
- Adding pdm/models/specifiers.py
- Adding pdm/models/versions.py
- Adding pdm/models/working_set.py
- Adding pdm/pep582/sitecustomize.py
- Adding pdm/project/__init__.py
- Adding pdm/project/config.py
- Adding pdm/project/core.py
- Adding pdm/project/metadata.py
- Adding pdm/py.typed
- Adding pdm/resolver/__init__.py
- Adding pdm/resolver/core.py
- Adding pdm/resolver/providers.py
- Adding pdm/resolver/python.py
- Adding pdm/resolver/reporters.py
- Adding pdm/signals.py
- Adding pdm/termui.py
- Adding pdm/utils.py
- Adding pdm-0.0.0.dist-info/entry_points.txt
- Adding pdm-0.0.0.dist-info/WHEEL
- Adding pdm-0.0.0.dist-info/METADATA
- Adding pdm-0.0.0.dist-info/license_files/LICENSE
- Adding pdm-0.0.0.dist-info/RECORD
Successfully built pdm-0.0.0-py3-none-any.whl
List of modules installed in build env
Package Version
--------------- --------------
attrs 21.4.0
blinker 1.4
Brlapi 0.8.3
build 0.7.0
click 8.1.2
codespell 2.1.0
cycler 0.11.0
distro 1.7.0
extras 1.0.0
findpython 0.1.6
fixtures 4.0.0
fonttools 4.33.3
gpg 1.17.1-unknown
iniconfig 1.1.1
installer 0.5.1
kiwisolver 1.3.2
libcomps 0.1.18
louis 3.21.0
matplotlib 3.5.1
numpy 1.22.3
olefile 0.46
packaging 21.3
pbr 5.8.1
pdm-pep517 0.12.4
pep517 0.12.0
Pillow 9.1.0
pip 22.0.4
platformdirs 2.5.2
pluggy 1.0.0
py 1.11.0
PyGObject 3.42.1
pyparsing 3.0.8
pytest 7.1.2
pytest-mock 3.7.0
python-dateutil 2.8.2
resolvelib 0.8.1
rpm 4.17.0
setuptools 62.0.0
six 1.16.0
testtools 2.5.0
tomli 2.0.1
tomlkit 0.10.2
wheel 0.37.1
This issue is tranferred from pdm-project/pdm#332
I'd like to get your take on a use case. version
can be calculated dynamically, which is great for reducing repetition. Currently only 'file'
and 'scm'
are supported sources of dynamic version information.
I'm experimenting with integrating pdm
with the Python infra at $work, where we calculate version numbers of packages quite differently. I've played with trying to use a pre_build
hook to talk to our infra, and then saving the version number in a file. I think I can make that work, but it's not ideal (e.g. for several reasons, but f.e. I have to git ignore that file because overwritten during the build process).
What if pdm
added a 'call'
source for calculating the dynamic version, with the same semantics as say the pre_build
hook? Then I could do something like this in my pyproject.toml
file:
version = { call = 'getversion:main()' }
where getversion.py
is a script I can write whose main()
makes the external infra calls to get the next version number correctly. Seems like it wouldn't be too hard to add to the DynamicVersion
class.
WDYT?
$ pypy3 -m pytest
========================================================= test session starts =========================================================
platform linux -- Python 3.8.12[pypy-7.3.7-final], pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /tmp/pdm-pep517
collected 50 items
tests/test_api.py .......x........ [ 32%]
tests/test_file_finder.py ............ [ 56%]
tests/test_metadata.py ............ [ 80%]
tests/test_validator.py ........ [ 96%]
tests/test_wheel.py F. [100%]
============================================================== FAILURES ===============================================================
________________________________________________ test_override_tags_in_wheel_filename _________________________________________________
def test_override_tags_in_wheel_filename() -> None:
project = FIXTURES / "projects/demo-cextension"
builder = wheel.WheelBuilder(
project,
config_settings={"--py-limited-api": "cp36", "--plat-name": "win_amd64"},
)
> assert builder.tag == "cp36-abi3-win_amd64"
E AssertionError: assert 'pp38-pypy38_pp73-win_amd64' == 'cp36-abi3-win_amd64'
E - cp36-abi3-win_amd64
E + pp38-pypy38_pp73-win_amd64
tests/test_wheel.py:11: AssertionError
========================================================== warnings summary ===========================================================
pdm/pep517/_vendor/cerberus/errors.py:156
/tmp/pdm-pep517/pdm/pep517/_vendor/cerberus/errors.py:156: DeprecationWarning: invalid escape sequence '*'
"""
pdm/pep517/_vendor/cerberus/errors.py:184
/tmp/pdm-pep517/pdm/pep517/_vendor/cerberus/errors.py:184: DeprecationWarning: invalid escape sequence '*'
"""
tests/test_api.py::test_build_single_module
/tmp/pdm-pep517/pdm/pep517/base.py:152: PDMDeprecatedWarning: `version` in [project] no longer supports dynamic filling. Move it to [tool.pdm] or change it to static string.
It will raise an error in the next minor release.
meta_version = self.meta.version
tests/test_api.py::test_build_single_module
/tmp/pdm-pep517/pdm/pep517/base.py:347: PDMDeprecatedWarning: `version` in [project] no longer supports dynamic filling. Move it to [tool.pdm] or change it to static string.
It will raise an error in the next minor release.
version=meta.version or "UNKNOWN",
tests/test_api.py::test_build_single_module
/tmp/pdm-pep517/pdm/pep517/sdist.py:94: PDMDeprecatedWarning: `version` in [project] no longer supports dynamic filling. Move it to [tool.pdm] or change it to static string.
It will raise an error in the next minor release.
self.meta._metadata["version"] = self.meta.version
tests/test_metadata.py::test_parse_module
/tmp/pdm-pep517/tests/test_metadata.py:13: PDMDeprecatedWarning: `version` in [project] no longer supports dynamic filling. Move it to [tool.pdm] or change it to static string.
It will raise an error in the next minor release.
assert metadata.version == "0.1.0"
-- Docs: https://docs.pytest.org/en/stable/warnings.html
======================================================= short test summary info =======================================================
FAILED tests/test_wheel.py::test_override_tags_in_wheel_filename - AssertionError: assert 'pp38-pypy38_pp73-win_amd64' == 'cp36-abi3...
========================================= 1 failed, 48 passed, 1 xfailed, 6 warnings in 6.86s =========================================
Some of the tests fail on systems that haven't configured git. For example, some downstreams run tests out of a git tree and may not have configured git
.
==================================== ERRORS ====================================
___________ ERROR at setup of test_build_wheel_write_version_to_file ___________
tmp_path = PosixPath('/usr/src/tmp/pytest-of-builder/pytest-3/test_build_wheel_write_version0')
@pytest.fixture()
def project_with_scm(tmp_path):
project = FIXTURES / "projects/demo-using-scm"
shutil.copytree(project, tmp_path / project.name)
with utils.cd(tmp_path / project.name):
subprocess.check_call(["git", "init"])
subprocess.check_call(["git", "add", "."])
> subprocess.check_call(["git", "commit", "-m", "initial commit"])
tests/conftest.py:17:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
popenargs = (['git', 'commit', '-m', 'initial commit'],), kwargs = {}
retcode = 128, cmd = ['git', 'commit', '-m', 'initial commit']
def check_call(*popenargs, **kwargs):
"""Run command with arguments. Wait for command to complete. If
the exit code was zero then return, otherwise raise
CalledProcessError. The CalledProcessError object will have the
return code in the returncode attribute.
The arguments are the same as for the call function. Example:
check_call(["ls", "-l"])
"""
retcode = call(*popenargs, **kwargs)
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
> raise CalledProcessError(retcode, cmd)
E subprocess.CalledProcessError: Command '['git', 'commit', '-m', 'initial commit']' returned non-zero exit status 128.
/usr/lib64/python3.10/subprocess.py:369: CalledProcessError
---------------------------- Captured stdout setup -----------------------------
Initialized empty Git repository in /usr/src/tmp/pytest-of-builder/pytest-3/test_build_wheel_write_version0/demo-using-scm/.git/
---------------------------- Captured stderr setup -----------------------------
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Author identity unknown
*** Please tell me who you are.
Run
git config --global user.email "[email protected]"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
fatal: empty ident name (for <[email protected]>) not allowed
...
=========================== short test summary info ============================
XFAIL tests/test_metadata.py::test_license_classifiers_warning
This will fail until PEP 639 is finalized
XFAIL tests/test_metadata.py::test_deprecated_license_field_warning
Don't emit warning until PEP 639 is accepted
XFAIL tests/test_metadata.py::test_deprecated_license_file_warning
Don't emit warning until PEP 639 is accepted
ERROR tests/test_api.py::test_build_wheel_write_version_to_file - subprocess....
ERROR tests/test_api.py::test_build_wheel_write_version_to_file_template - su...
ERROR tests/test_metadata.py::test_project_version_use_scm - subprocess.Calle...
ERROR tests/test_metadata.py::test_project_version_use_scm_from_env - subproc...
=================== 70 passed, 3 xfailed, 4 errors in 3.84s ====================
Make sure you run commands with -v
flag before pasting the output.
run pdm sync
pdm.termui: Traceback (most recent call last):
pdm.termui: File "/home/bingxin.fan/sdk-python/setup.py", line 3, in <module>
pdm.termui: from setuptools import setup
pdm.termui: ModuleNotFoundError: No module named 'setuptools'
pdm.termui: /tmp/pdm-build-env-_b26g862-shared/lib/python3.8/site-packages/pdm/pep517/base.py:295: PDMWarning: 'license-expression' is missing
pdm.termui: "license": meta.license_expression,
pdm.termui: Traceback (most recent call last):
pdm.termui: File "/tmp/pdm-build-env-_b26g862-shared/lib/python3.8/site-packages/pdm/pep517/editable.py", line 88, in _build
pdm.termui: subprocess.check_call(build_args)
pdm.termui: File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
pdm.termui: raise CalledProcessError(retcode, cmd)
pdm.termui: subprocess.CalledProcessError: Command '['/usr/bin/python3.8', '/home/bingxin.fan/sdk-python/setup.py', 'build_ext', '--inplace']' returned non-zero exit status 1.
pdm.termui:
pdm.termui: During handling of the above exception, another exception occurred:
pdm.termui:
pdm.termui: Traceback (most recent call last):
pdm.termui: File "/home/bingxin.fan/.local/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
pdm.termui: main()
pdm.termui: File "/home/bingxin.fan/.local/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 345, in main
pdm.termui: json_out['return_val'] = hook(**hook_input['kwargs'])
pdm.termui: File "/home/bingxin.fan/.local/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 283, in build_editable
pdm.termui: return hook(wheel_directory, config_settings, metadata_directory)
pdm.termui: File "/tmp/pdm-build-env-_b26g862-shared/lib/python3.8/site-packages/pdm/pep517/api.py", line 101, in build_editable
pdm.termui: return Path(builder.build(wheel_directory)).name
pdm.termui: File "/tmp/pdm-build-env-_b26g862-shared/lib/python3.8/site-packages/pdm/pep517/wheel.py", line 77, in build
pdm.termui: self._build(zip_file)
pdm.termui: File "/tmp/pdm-build-env-_b26g862-shared/lib/python3.8/site-packages/pdm/pep517/editable.py", line 90, in _build
pdm.termui: raise BuildError(f"Error occurs when running {build_args}:\n{e}")
pdm.termui: pdm.pep517.exceptions.BuildError: Error occurs when running ['/usr/bin/python3.8', '/home/bingxin.fan/sdk-python/setup.py', 'build_ext', '--inplace']:
pdm.termui: Command '['/usr/bin/python3.8', '/home/bingxin.fan/sdk-python/setup.py', 'build_ext', '--inplace']' returned non-zero exit status 1.
The pdm sync
runs successfully.
# Paste the output of `pdm info && pdm info --env` below:
$ pdm info
PDM version:
2.0.0
Python Interpreter:
/usr/bin/python3.8 (3.8)
Project Root:
/home/bingxin.fan/sdk-python
Project Packages:
/home/bingxin.fan/sdk-python/__pypackages__/3.8
$ pdm info --env
{
"implementation_name": "cpython",
"implementation_version": "3.8.10",
"os_name": "posix",
"platform_machine": "x86_64",
"platform_release": "5.4.0-77-shopee-generic",
"platform_system": "Linux",
"platform_version": "#86+3 SMP Fri Oct 22 11:37:43 +08 2021",
"python_full_version": "3.8.10",
"platform_python_implementation": "CPython",
"python_version": "3.8",
"sys_platform": "linux"
}
The build
section of pyproject.toml
:
[tool.pdm.build]
setup-script = "cmake.py"
run-setuptools = false
is-purelib = true
I'm wondering why the setuptools
be called.
With a missing name in pyproject.toml (and a setup.cfg that enables pip-install to accept -e
) pdm.pep517
crashes:
Obtaining file://…/Vcs/pdm/test-editable
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... error
ERROR: Command errored out with exit status 1:
command: …/Vcs/pdm/test-editable/.venv/bin/python …/Vcs/pdm/test-editable/.venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpy1vo_95a
cwd: …/Vcs/pdm/test-editable
Complete output (18 lines):
Traceback (most recent call last):
File "…/Vcs/pdm/test-editable/.venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 349, in <module>
main()
File "…/Vcs/pdm/test-editable/.venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 331, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "…/Vcs/pdm/test-editable/.venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 151, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/tmp/pip-build-env-jjh39ad9/overlay/lib/python3.9/site-packages/pdm/pep517/api.py", line 42, in prepare_metadata_for_build_wheel
dist_info = Path(metadata_directory, builder.dist_info_name)
File "/tmp/pip-build-env-jjh39ad9/overlay/lib/python3.9/site-packages/pdm/pep517/wheel.py", line 128, in dist_info_name
name = to_filename(self.meta.project_name)
File "/tmp/pip-build-env-jjh39ad9/overlay/lib/python3.9/site-packages/pdm/pep517/metadata.py", line 318, in project_name
return safe_name(self.name)
File "/tmp/pip-build-env-jjh39ad9/overlay/lib/python3.9/site-packages/pdm/pep517/utils.py", line 24, in safe_name
return re.sub("[^A-Za-z0-9.]+", "-", name)
File "/usr/lib/python3.9/re.py", line 210, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or bytes-like object
----------------------------------------
Not sure what it should really do, which might be related to metadata.name
being defined in setup.cfg
or not - maybe it could fall back to using the directories name?
The idea here is to allow for the typical pip install -e .
(e.g. into a venv), and using pdm only to manage deps (in __pypackages__
).
Also not having a name allows for leaving out --no-self
with pdm-install.
btw: any tips on how to debug this? E.g. by keeping the temporary file to add pdb.set_trace()
, or injecting the local pdm/pdm.pep517 to be used.
Hello,
I just install numpy and pandas failed in 3.10 env. Then tried on 3.8 env, it succeeds.
This error message said something related to pep-517, so I open the issue in this repository.
The errors in 3.10:
add numpy failed:
Traceback (most recent call last):
File "/home/tv60xm3/.pyenv/versions/3.10.6/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/data/work/envs/3.10/lib/python3.10/site-packages/pdm/installers/synchronizers.py", line 225, in install_candidate
self.manager.install(can)
File "/data/work/envs/3.10/lib/python3.10/site-packages/pdm/installers/manager.py", line 39, in install
installer(str(prepared.build()), self.environment, prepared.direct_url())
File "/data/work/envs/3.10/lib/python3.10/site-packages/pdm/models/candidates.py", line 382, in build
builder.build(build_dir, metadata_directory=self._metadata_dir)
File "/data/work/envs/3.10/lib/python3.10/site-packages/pdm/builders/wheel.py", line 28, in build
filename = self._hook.build_wheel(out_dir, config_settings, metadata_directory)
File "/data/work/envs/3.10/lib/python3.10/site-packages/pep517/wrappers.py", line 209, in build_wheel
return self._call_hook('build_wheel', {
File "/data/work/envs/3.10/lib/python3.10/site-packages/pep517/wrappers.py", line 309, in _call_hook
self._subprocess_runner(
File "/data/work/envs/3.10/lib/python3.10/site-packages/pdm/builders/base.py", line 245, in subprocess_runner
return log_subprocessor(cmd, cwd, extra_environ=env)
File "/data/work/envs/3.10/lib/python3.10/site-packages/pdm/builders/base.py", line 84, in log_subprocessor
raise BuildError(
pdm.exceptions.BuildError: Call command ['/data/work/envs/3.10/bin/python3', '/data/work/envs/3.10/lib/python3.10/site-packages/pep517/in_process/_in_process.py', 'build_wheel',
'/tmp/tmp68cv_r9z'] return non-zero status(1). Make sure the package is PEP 517-compliant, or you can add `--no-isolation` to the command.
Is there a bug or something happens?
Please have a check, thanks!
[tool.pdm]
version = {use_scm = true}
Sorry to bother you, but I'm getting crazy trying to understand modern packaging rules, and here I'm asking an opinion/confirmation.
I'm trying to modernize this aspect of several projects of mine, replacing setup.py
with pyproject.toml
, and I had different issues with every build system I tried, maybe due to the peculiar name scheme I followed for them, or instead because the specifications are imprecise and everyone interprets them in different ways.
I found your project the less problematic as of now (read, I did not have any issue), and I'm taking the opportunity to say "thank you!".
Today I converted another project, whose distribution name is metapensiero.sphinx.autodoc_sa
: everything went smooth, except that I noticed a difference in the name of the generated file for sdist and wheel:
$ ls -rt1 dist/
...
metapensiero.sphinx.autodoc_sa-1.7.tar.gz
metapensiero.sphinx.autodoc_sa-2.0.tar.gz
metapensiero.sphinx.autodoc_sa-2.1.tar.gz
metapensiero.sphinx.autodoc-sa-2.2.dev0.tar.gz
metapensiero.sphinx.autodoc_sa-2.2.dev0-py3-none-any.whl
The latest version, 2.2.dev0, is the one converted to use pdm-pep517
: as you can see, the sdist contains autodoc-sa
(with an hypen) while the wheel has autodoc_sa
(with underscore). Previous versions were generated by setuptools
(that is, by python setup.py sdist
).
I tried uploading those new archives to test.pypi.org and they were accepted: at first I expected a rejection due to different name, as it happened for one other project when I used flit
as build system, but I was wrong :-)
To satisfy my curiosity, I tried to find references to the file name conventions, and I found this section on PyPA:
The file name of a sdist is not currently standardised, although the de facto form is {name}-{version}.tar.gz, where {name} is the canonicalized form of the project name (see PEP 503 for the canonicalization rules) with - characters replaced with _
That makes me wonder: it seems rather contradictory that it first suggests to replace runs of invalid characters with a single hyphen, but then says to replace that hyphen with an underscore.
So the question is: should I relax and accept the apparent confusion, or is there an issue in the way pdm-pep517
generates the sdist (using metadata.project_name) vs the wheel (that instead uses metadata.project_filename)?
Thanks again!
See flit's usage guide for prior-art on this. It's a small thing, but it saves duplicating the summary line between the module docs and the pyproject file.
Also currently converting a project from flit to pdm (as I did) results in the project description being empty, with this feature it could instead be converted to a dynamic reference to the project's __init__.py
file.
I'm thinking that using the tokenize
module would be the best way to implement this (re
won't really cut it). I can whip a PoC of this if you are interested. Cheers!
Avoid specifying dependencies=[]
.
* Building wheel...
- Adding example_pkg/__init__.py
- Adding example_pkg/example.py
- Adding example_package_pdm-0.0.7.dist-info/WHEEL
/private/var/folders/bw/ttgk1hcs0k989q2yzmxkzn8c0000gp/T/build-env-qbpgfp13/lib/python3.9/site-packages/pdm/pep517/base.py:367: PDMWarning: 'license-expression' is missing
license=meta.license_expression or "UNKNOWN",
/private/var/folders/bw/ttgk1hcs0k989q2yzmxkzn8c0000gp/T/build-env-qbpgfp13/lib/python3.9/site-packages/pdm/pep517/base.py:394: PDMWarning: License classifiers are deprecated in favor of PEP 639 'license-expression' field.
for classifier in meta.classifiers or []:
Traceback (most recent call last):
File "/Users/bhrutledge/Dev/example-pkg-bhrutledge/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/Users/bhrutledge/Dev/example-pkg-bhrutledge/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/Users/bhrutledge/Dev/example-pkg-bhrutledge/venv/lib/python3.9/site-packages/pep517/in_process/_in_process.py", line 261, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/private/var/folders/bw/ttgk1hcs0k989q2yzmxkzn8c0000gp/T/build-env-qbpgfp13/lib/python3.9/site-packages/pdm/pep517/api.py", line 74, in build_wheel
return Path(builder.build(wheel_directory)).name
File "/private/var/folders/bw/ttgk1hcs0k989q2yzmxkzn8c0000gp/T/build-env-qbpgfp13/lib/python3.9/site-packages/pdm/pep517/wheel.py", line 77, in build
self._write_metadata(zip_file)
File "/private/var/folders/bw/ttgk1hcs0k989q2yzmxkzn8c0000gp/T/build-env-qbpgfp13/lib/python3.9/site-packages/pdm/pep517/wheel.py", line 150, in _write_metadata
self._write_metadata_file(f)
File "/private/var/folders/bw/ttgk1hcs0k989q2yzmxkzn8c0000gp/T/build-env-qbpgfp13/lib/python3.9/site-packages/pdm/pep517/wheel.py", line 264, in _write_metadata_file
fp.write(self.format_pkginfo())
File "/private/var/folders/bw/ttgk1hcs0k989q2yzmxkzn8c0000gp/T/build-env-qbpgfp13/lib/python3.9/site-packages/pdm/pep517/base.py", line 398, in format_pkginfo
for dep in sorted(meta.dependencies):
TypeError: 'NoneType' object is not iterable
The field is optional. Suggestion: something like this would work:
for dep in sorted(meta.dependencies or []):
See https://gist.github.com/bhrutledge/01509dc194a6d0608407cc84b97ac58e
This breaks pypa/packaging.python.org#1031.
I may be tired, but i can't find a way to use a relative path in dependencies urls.
pdm.pep517._vendor.packaging.requirements throws me InvalidRequirement
Below you'll find a quick (and dirty) test i made reproducing the code in question:
import urllib.parse
tests = [
"foo/bar",
"./foo/bar",
"file:foo/bar",
"file:./foo/bar"
]
for t in tests:
parsed_url = urllib.parse.urlparse(t)
if parsed_url.scheme == "file":
ut = urllib.parse.urlunparse(parsed_url)
if ut != t:
print(f"Invalid URL given: {t} != {ut}")
print(parsed_url)
continue
elif not (parsed_url.scheme and parsed_url.netloc) or (
not parsed_url.scheme and not parsed_url.netloc
):
print(f"Invalid URL: {t}")
print(parsed_url)
continue
print(f"Valid: {t} == {ut}")
print(parsed_url)
Which gives:
Invalid URL: foo/bar
ParseResult(scheme='', netloc='', path='foo/bar', params='', query='', fragment='')
Invalid URL: ./foo/bar
ParseResult(scheme='', netloc='', path='./foo/bar', params='', query='', fragment='')
Invalid URL given: file:foo/bar != file:///foo/bar
ParseResult(scheme='file', netloc='', path='foo/bar', params='', query='', fragment='')
Invalid URL given: file:./foo/bar != file:///./foo/bar
ParseResult(scheme='file', netloc='', path='./foo/bar', params='', query='', fragment='')
file:///foo/bar
is not an option, as it would give an absolute path
According to pep 508, any of the tested url should be valid.
Ideas:
. First, call urllib as urllib.parse.urlparse(req.url, scheme="file")
, since, still according to pep 508, a url with no scheme is a path.
. Do not test if urllib.parse.urlunparse
is the same as the original, as you can see in the tests above, given a relative path, this method will produce... an absolute path (bug in urllib?)
. IMHO, testing if scheme is file
should be enough. Maybe even no testing at all, and let the frontend decide what to do with it.
note: i'm not using pdm, but experimenting with pdm-pep517 as a build-backend for pip
The src
layout has only one purpose: prevent running tests against the sources.
But it seems PDM, just like Poetry, adds my src
directory to PYTHONPATH:
$ python -c 'import sys; print("\n".join(sys.path))'
/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python38.zip
/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python3.8
/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python3.8/lib-dynload
/home/user/data/dev/project/__pypackages__/3.8/lib
/home/user/data/dev/project/src
/home/user/.local/lib/python3.8/site-packages
/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python3.8/site-packages
# disable PEP582
$ PYTHONPATH= python -c 'import sys; print("\n".join(sys.path))'
/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python38.zip
/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python3.8
/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python3.8/lib-dynload
/home/user/.local/lib/python3.8/site-packages
/home/user/.basher-packages/pyenv/pyenv/versions/3.8.11/lib/python3.8/site-packages
I would like PDM to avoid adding my src
folder to PYTHONPATH. I want the installed version of my package to be used when I run my tests, or simply run an interpreter, not the source version. Adding the sources to PYTHONPATH hides potential packaging errors.
I can't seem to find where the src
folder is added to PYTHONPATH, it does not seem to be in sitecustomize.py
at least, so not sure if it's really PDM that adds it.
A new PEP has been accepted:
pdm.termui: Traceback (most recent call last):
pdm.termui: File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 363, in <module>
pdm.termui: main()
pdm.termui: File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 345, in main
pdm.termui: json_out['return_val'] = hook(**hook_input['kwargs'])
pdm.termui: File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/pep517/in_process/_in_process.py", line 283, in build_editable
pdm.termui: return hook(wheel_directory, config_settings, metadata_directory)
pdm.termui: File "/tmp/pdm-build-env-85zkiyg_-shared/lib/python3.6/site-packages/pdm/pep517/api.py", line 98, in build_editable
pdm.termui: return Path(builder.build(wheel_directory)).name
pdm.termui: File "/tmp/pdm-build-env-85zkiyg_-shared/lib/python3.6/site-packages/pdm/pep517/wheel.py", line 75, in build
pdm.termui: self._build(zip_file)
pdm.termui: File "/tmp/pdm-build-env-85zkiyg_-shared/lib/python3.6/site-packages/pdm/pep517/editable.py", line 79, in _build
pdm.termui: setup_py = self.ensure_setup_py()
pdm.termui: File "/tmp/pdm-build-env-85zkiyg_-shared/lib/python3.6/site-packages/pdm/pep517/base.py", line 410, in ensure_setup_py
pdm.termui: setup_py_path.write_text(self.format_setup_py(), encoding="utf-8")
pdm.termui: File "/tmp/pdm-build-env-85zkiyg_-shared/lib/python3.6/site-packages/pdm/pep517/base.py", line 277, in format_setup_py
pdm.termui: "from {} import build\n".format(meta.build.split(".")[0]),
pdm.termui: AttributeError: 'dict' object has no attribute 'split'
✖ Install baize 0.17.1 failed
https://github.com/abersheeran/baize/runs/7168547020?check_suite_focus=true
pdm 2.0.0b1
Sorry that I'm not sure if this is a bug or an intentional behavior. I'm trying to build an architecture independent wheel consisting of only .py files and data files, and data file needs to be generated during build process.
Currently I'm trying to evaluate which build backend to use. PDM seems to indicate user can insert custom build step (though it's about creating C extension), therefore I wish to check out if it satisfy the need to generate extra file.
As a starter, I tried inserting a dummy build()
call:
def build(*args):
pass
Along with modification in pyproject.toml
:
[tool.pdm]
build = "extra_build.py"
Everything is mostly fine, but wheel is not architecture independent anymore.
.dist-info/WHEEL file content | |
---|---|
Before | Wheel-Version: 1.0 Generator: pdm-pep517 0.9.4 Root-Is-Purelib: True Tag: py3-none-any |
After | Wheel-Version: 1.0 Generator: pdm-pep517 0.9.4 Root-Is-Purelib: False Tag: cp38-cp38-win_amd64 |
So is it true that build = ...
is only intended for building architecture specific stuff?
> pdm info
PDM version: 1.11.3
Python Interpreter: [redacted virtualenv folder]/Scripts/python.EXE (3.8)
Project Root: [project root folder]
Project Packages: None
Make sure you run commands with -v
flag before pasting the output.
I'm using pdm build
to build a wheel (https://github.com/pglet/pglet-python). The package includes "bin" folder with a number of executables in it. Executables are downloaded during CI build and their execute permissions are set with chmod
: https://github.com/pglet/pglet-python/blob/main/appveyor.yml#L27-L42
Upon testing Python package built with PDM it looks like PDM drops "execute" permissions on files in bin
directory while making a package with pdm build
. Here is the recent version built with PDM: https://test.pypi.org/project/pglet/0.1.280/
On the other hand, the version originally built with setup.py
preserves "execute" permissions: https://test.pypi.org/project/pglet/0.1.251/
"Execute" permissions on executables in bin
folder are dropped while building a wheel with pdm build
.
"Execute" permissions are preserved.
# Paste the output of `pdm info && pdm info --env` below:
PDM version: 1.12.6
Python Interpreter: /home/appveyor/venv3.9.8/bin/python (3.9)
Project Root: /home/appveyor/projects/pglet-python
Project Packages: /home/appveyor/projects/pglet-python/__pypackages__/3.9
{
"implementation_name": "cpython",
"implementation_version": "3.9.8",
"os_name": "posix",
"platform_machine": "x86_64",
"platform_release": "5.4.0-1063-azure",
"platform_system": "Linux",
"platform_version": "#66~18.04.1-Ubuntu SMP Thu Oct 21 09:59:28 UTC 2021",
"python_full_version": "3.9.8",
"platform_python_implementation": "CPython",
"python_version": "3.9",
"sys_platform": "linux"
}
Custom setup scripts as described https://pdm.fming.dev/latest/pyproject/build/#custom-file-generation are only invoked when building wheels, not when building sdists.
I am not sure if that is intentional, but I am looking for a way to invoke a custom script for all builds, and there does not seem to be a way to invoke a custom script when building sdists.
Given a pyproject.toml like this:
[project]
name = "myproject"
version = ""
description = ""
authors = [
{name = "Foo bar", email = "[email protected]"},
]
dependencies = []
requires-python = ">=3.9"
dynamic = ["classifiers"]
license = {text = "MIT"}
[project.urls]
homepage = ""
[build-system]
requires = ["pdm-pep517"]
build-backend = "pdm.pep517.api"
And an empty setup.cfg
file (or one containing only setup for (generic) tools).
Running pip install -e .
fails:
Obtaining file://…/Vcs/pdm/test-editable
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Installing collected packages: myproject
Running setup.py develop for myproject
ERROR: Command errored out with exit status 1:
command: …/Vcs/pdm/test-editable/test-venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'…/Vcs/pdm/test-editable/setup.py'"'"'; __file__='"'"'…/Vcs/pdm/test-editable/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
cwd: …/Vcs/pdm/test-editable/
Complete output (3 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'setuptools'
----------------------------------------
ERROR: Command errored out with exit status 1: …/Vcs/pdm/test-editable/test-venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'…/Vcs/pdm/test-editable/setup.py'"'"'; __file__='"'"'…/Vcs/pdm/test-editable/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
(the code appears to be from pdm-pep517 (https://github.com/pdm-project/pdm-pep517/blob/master/pdm/pep517/base.py), and it does not fail when not using a build-system section in pyproject.toml (installing it as "UNKNOWN" then))
Note that pip install .
also fails:
Processing /home/user/Vcs/pdm/test-editable
DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory. We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.
pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issues/7555.
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Building wheels for collected packages: myproject
Building wheel for myproject (PEP 517) ... done
Created wheel for myproject: filename=myproject--py3-none-any.whl size=854 sha256=66ef6a33ad94bda6b4957a6c418d876b4368beae598138d293bc0760350ab17d
Stored in directory: /home/user/.cache/pip/wheels/70/04/a4/930f2de3c1b0ee34262a6e0bd34a225d359bc7601504de17c6
WARNING: Built wheel for myproject is invalid: Wheel has unexpected file name: expected 'UNKNOWN', got ''
Failed to build myproject
ERROR: Could not build wheels for myproject which use PEP 517 and cannot be installed directly
Note that the non-editable install also fails without a setup.cfg
.
pip 21.2.4
PEP 639 allows license files to be located in subdirectories. The _prepare_metadata
function does not account for that causing a FileNotFoundError
when using pdm-pep517
as build backend without pdm
(e.g. running pip install
on sdist).
Example:
project/
pyproject.toml
package/__init__.py
LICENSES/
MIT.txt
CC-BY-SA-4.0.txt
# pyproject.toml
[build-system]
requires = ["pdm-pep517==1.0.4"]
build-backend = "pdm.pep517.api"
[project]
name = "project"
license-expression = "MIT AND CC-BY-SA-4.0"
license-files.globs = ["LICENSES/*.txt"]
>> pdm build
Building sdist...
Built sdist at /tmp/test/project/dist/project-0.0.0.tar.gz
Building wheel...
Built wheel at /tmp/test/project/dist/project-0.0.0-py3-none-any.whl
>> pip install dist/project-0.0.0.tar.gz
Defaulting to user installation because normal site-packages is not writeable
Processing ./dist/project-0.0.0.tar.gz
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [16 lines of output]
Traceback (most recent call last):
File "/home/crai/.local/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
main()
File "/home/crai/.local/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/crai/.local/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
return hook(metadata_directory, config_settings)
File "/tmp/pip-build-env-wj5pgntw/overlay/lib/python3.10/site-packages/pdm/pep517/api.py", line 64, in prepare_metadata_for_build_wheel
return _prepare_metadata(builder, metadata_directory)
File "/tmp/pip-build-env-wj5pgntw/overlay/lib/python3.10/site-packages/pdm/pep517/api.py", line 52, in _prepare_metadata
(dist_info / "license_files" / license_file).write_bytes(
File "/usr/lib/python3.10/pathlib.py", line 1143, in write_bytes
with self.open(mode='wb') as f:
File "/usr/lib/python3.10/pathlib.py", line 1119, in open
return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-modern-metadata-wm8_1g5c/project-0.0.0.dist-info/license_files/LICENSES/MIT.txt'
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.