Giter Site home page Giter Site logo

Comments (16)

frostming avatar frostming commented on June 10, 2024 1
  • pdm download doesn't pull the build dependencies(pdm-backend) which are not in the lockfile
  • pdm export doesn't pin build dependencies either for the same reason
  • pip wheel doesn't know the current project you want to install as editable

To solve this, you can add pdm-backend as a dependency, repeat the same steps, and install offline with pdm install --no-isolation

BTW, Error: Client.get() got an unexpected keyword argument 'stream' this should be fixed by [email protected]

from pdm.

frostming avatar frostming commented on June 10, 2024 1

you need to completely ban pypi offline by setting the name of the local source to "pypi" instead of "local"

from pdm.

SealedServant avatar SealedServant commented on June 10, 2024

@frostming, thank you for the quick response. Lightning fast bug fix on pdm-download mate⚡

Is it intended behavior for pdm-download to download all the package binaries for every platform? pip seems to download just the wheels for the current platform.

Notes

Adding pdm-backend to the project dependencies works to get it downloaded as a wheel (in .packages\pdm_backend-2.2.1-py3-none-any.whl), but pip cannot seem to find pdm-backend in the venv when I try to install the project as editable.

$ pdm venv create --with-pip
Virtualenv C:\Users\xxx\xxx\.venv is created successfully
INFO: Virtualenv C:\Users\xxx\xxx\.venv is reused.

$  pdm run pip install -r .packages/requirements.txt --no-index -f .packages
Looking in links: .packages
Processing ...
Installing collected packages: ...
Successfully installed ... 

$  pdm run pip install -e .
Looking in indexes: https://xxx:****@xxx.xxx.xxx.xxx/repository/pypi-proxy/simple/
Obtaining file:///C:/Users/xxx/xxx
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Looking in indexes: https://xxx:****@xxx.xxx.xxx.xxx/repository/pypi-proxy/simple/
      WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x0000016E815A1EE0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /repository/pypi-proxy/simple/pdm-backend/
      WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x0000016E81608920>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /repository/pypi-proxy/simple/pdm-backend/
      WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x0000016E81608B60>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /repository/pypi-proxy/simple/pdm-backend/
      WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x0000016E81608D70>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /repository/pypi-proxy/simple/pdm-backend/
      WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x0000016E81608F80>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed')': /repository/pypi-proxy/simple/pdm-backend/
      ERROR: Could not find a version that satisfies the requirement pdm-backend (from versions: none)
      ERROR: No matching distribution found for pdm-backend
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

I tried using pdm install --no-isolation instead, but it still appears that pdm tries to connect to the internet and fails.

C:\Users\xxx\xxx(dev -> origin)
λ pdm install -v --no-isolation
STATUS: Resolving packages from lockfile...
unearth.preparer: The file . is a local directory, use it directly
pdm.termui: Running PEP 517 backend to get metadata for <Link file:///C:/Users/xxx/xxx (from None)>
pdm.termui: Preparing environment(Non-isolated mode) for PEP 517 build...
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   editables
pdm.termui:   python==3.12.2
pdm.termui:   Adding requirement editables
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\pdm\Bin\pdm.exe\__main__.py", line 7, in <module>
  File "C:\pdm\Lib\site-packages\pdm\core.py", line 358, in main
    return core.main(args or sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\core.py", line 276, in main
    raise cast(Exception, err).with_traceback(traceback) from None
  File "C:\pdm\Lib\site-packages\pdm\core.py", line 271, in main
    self.handle(project, options)
  File "C:\pdm\Lib\site-packages\pdm\core.py", line 207, in handle
    command.handle(project, options)
  File "C:\pdm\Lib\site-packages\pdm\cli\commands\install.py", line 100, in handle
    actions.do_sync(
  File "C:\pdm\Lib\site-packages\pdm\cli\actions.py", line 238, in do_sync
    synchronizer.synchronize()
  File "C:\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 395, in synchronize
    to_add, to_update, to_remove = self.compare_with_working_set()
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 215, in compare_with_working_set
    candidates = self.candidates.copy()
                 ^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\functools.py", line 995, in __get__
    val = self.func(instance)
          ^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 151, in candidates
    if self.should_install_editables():
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 169, in should_install_editables
    metadata = self.self_candidate.prepare(self.environment).metadata
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python312\Lib\functools.py", line 995, in __get__
    val = self.func(instance)
          ^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 139, in self_candidate
    return self.environment.project.make_self_candidate(not self.no_editable)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\project\core.py", line 521, in make_self_candidate
    can.prepare(self.environment).metadata
  File "C:\pdm\Lib\site-packages\pdm\models\candidates.py", line 610, in metadata
    result = self.prepare_metadata()
             ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\models\candidates.py", line 514, in prepare_metadata
    return self._get_metadata_from_build(self._unpacked_dir, metadata_parent)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\models\candidates.py", line 592, in _get_metadata_from_build
    self._metadata_dir = builder(source_dir, self.environment).prepare_metadata(metadata_parent)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\builders\editable.py", line 24, in prepare_metadata
    self.install(requires)
  File "C:\pdm\Lib\site-packages\pdm\builders\base.py", line 296, in install
    install_requirements(missing, env)
  File "C:\pdm\Lib\site-packages\pdm\installers\core.py", line 28, in install_requirements
    resolved, _ = resolve(resolver, reqs, environment.python_requires, max_rounds=resolve_max_rounds, keep_self=True)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\resolver\core.py", line 39, in resolve
    result = resolver.resolve(requirements, max_rounds)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\resolvelib\resolvers.py", line 546, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\resolvelib\resolvers.py", line 397, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "C:\pdm\Lib\site-packages\resolvelib\resolvers.py", line 173, in _add_to_criteria
    if not criterion.candidates:
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\resolvelib\structs.py", line 127, in __bool__
    next(iter(self))
         ^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\resolvelib\structs.py", line 136, in __iter__
    self._factory() if self._iterable is None else self._iterable
    ^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\resolver\providers.py", line 226, in matches_gen
    candidates = self._find_candidates(reqs[0])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\resolver\providers.py", line 196, in _find_candidates
    return self.repository.find_candidates(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\models\repositories.py", line 174, in find_candidates
    cans = LazySequence(self._find_candidates(requirement, minimal_version=minimal_version))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\models\repositories.py", line 436, in _find_candidates
    for c in finder.find_all_packages(requirement.project_name, allow_yanked=requirement.is_pinned)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\finder.py", line 310, in find_all_packages
    self._find_packages(package_name, allow_yanked), hashes=hashes or {}
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\finder.py", line 290, in _find_packages
    return sorted(all_packages, key=self._sort_key, reverse=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\collector.py", line 178, in collect_links_from_location
    yield from _collect_links_from_index(session, location)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\collector.py", line 198, in _collect_links_from_index
    page = fetch_page(session, location)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\collector.py", line 185, in fetch_page
    resp = _get_html_response(session, location)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\collector.py", line 220, in _get_html_response
    resp = session.get(
           ^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 1054, in get
    return self.request(
           ^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 827, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 914, in send
    response = self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 1015, in _send_single_request
    response = transport.handle_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\hishel\_sync\_transports.py", line 197, in handle_request
    regular_response = self._transport.handle_request(request)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_transports\default.py", line 232, in handle_request
    with map_httpcore_exceptions():
  File "C:\Program Files\Python312\Lib\contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "C:\pdm\Lib\site-packages\httpx\_transports\default.py", line 86, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 11001] getaddrinfo failed

from pdm.

frostming avatar frostming commented on June 10, 2024

@SealedServant did you set the index to the local path, which is similar to pip's -f option?

https://arc.net/l/quote/zjcoxfvv

from pdm.

SealedServant avatar SealedServant commented on June 10, 2024

Is there a way to call pdm install -f like with pip? Or perhaps to set the local location in pyproject.toml?

[[tool.pdm.source]]
name = "local"
url = "./packages"
type = "find_links"

from pdm.

frostming avatar frostming commented on June 10, 2024

Or perhaps to set the local location in pyproject.toml?

You can use the file url:

[[tool.pdm.source]]
name = "local"
url = "file:///path/to/packages"
type = "find_links"

from pdm.

SealedServant avatar SealedServant commented on June 10, 2024

Thanks @frostming, I will try this out on Monday and report back the results here. Will file URL work with relative path like so?

[[tool.pdm.source]]
name = "local"
url = "file:///.packages"
type = "find_links"

I have a private pypi repo set up in pyproject.toml that I am hoping doesn't interfere with adding this secondary source. This is to ensure all packages come from our pypi mirror rather than externally.

[[tool.pdm.source]]
name = "pypi"
url = "https://our.pypi.mirror"
verify_ssl = true
include_package = ["*"]

Some comments:

I think the .packages directory should be a temporary/transient place in the project dir to store the wheels for offline installation, so I am not sure if permanently being in the pyproject.toml is the best approach.

I made a pdm download command with this behavior:

[tool.pdm.scripts]
download = { shell = """
  mkdir .packages \
  pdm export -o .packages/requirements.txt --no-hashes --no-markers \
  pdm run pip wheel -r .packages/requirements.txt -w .packages
""" }

I envision a pdm install --no-index -f .packages so that pdm will do everything that we expect like generating/checking/syncing with the lock file, installing the current project as editable, et cetera, but it won't fail if it can't reach the configured sources.

Ultimately, I think the best approach would be if we could just move the entire project directory, which is the intent of the user. However, too many tools such as virtualenv, venv, editables, are creating hardcoded absolute file paths (!!!) and which totally break when you move the project location. This is not the fault of pdm by any means, but users suffer from these other tools not enabling portable virtual environments. If pdm fix could somehow detect broken paths in places like .venv, .pdm-build, .pdm-python and fix them, that would make it so that we don't have to inefficiently download and install everything to site-packages again just for the purpose of recreating the .venv so that it works. But this might be out of scope of pdm and understandable that we just need to work with such limitations.

from pdm.

SealedServant avatar SealedServant commented on June 10, 2024
[[tool.pdm.source]]
name = "local"
url = "file:///${PROJECT_ROOT}/.packages"
type = "find_links"

Running pdm install --no-isolation offline is giving the same error as before, unfortunately. I also tried with url set to file:///./.packages

Also seeing this when downloading the packages:

WARNING: Location 'file:///${PROJECT_ROOT}/packages/' is ignored: it is neither a file nor a directory.

from pdm.

frostming avatar frostming commented on June 10, 2024

Stick to absolute paths for now

Also seeing this when downloading the packages:

Where does this come from, neither pdm nor unearth emits that

from pdm.

frostming avatar frostming commented on June 10, 2024

Closed by dfa6a8d

from pdm.

SealedServant avatar SealedServant commented on June 10, 2024

Stick to absolute paths for now

Also seeing this when downloading the packages:

Where does this come from, neither pdm nor unearth emits that

It looks like it is being emitted by pip when I have the local source url in the pyproject.toml after running this command in my project root to get the wheels for offline installation:

$ pdm run pip wheel -r .packages/requirements.txt -w .packages
Looking in indexes: https://xxx:[email protected]/repository/pypi-proxy/simple/
Looking in links: file:///${PROJECT_ROOT}/.packages
WARNING: Location 'file:///${PROJECT_ROOT}/.packages' is ignored: it is neither a file nor a directory.

pyproject.toml look like this:

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"

# ...

[tool.pdm]
distribution = true

[[tool.pdm.source]]
name = "pypi"
url = "https://${USERNAME}:${PASSWORD}@xxx.xxx.xxx/repository/pypi-proxy/simple/"
verify_ssl = true
include_packages = ["*"]

[[tool.pdm.source]]
name = "local"
url = "file:///${PROJECT_ROOT}/.packages" # do I need to set this env var manually or does PDM know this is my project?
type = "find_links"

Installation offline fail with following message:

$ pdm install --no-isolation
WARNING: Project requires a python version of <3.13,>=3.12, The virtualenv is being created for you as it cannot be matched to the right version.
INFO: python.use_venv is on, creating a virtualenv for this project...
Virtualenv is created successfully at C:\Users\xxx\xxx\.venv
See C:\Users\xxx\AppData\Local\pdm\pdm\Logs\pdm-install-nyi26u0v.log for detailed debug log.
[ConnectError]: [Errno 11001] getaddrinfo failed
WARNING: Add '-v' to see the detailed traceback

from pdm.

frostming avatar frostming commented on June 10, 2024

It looks like it is being emitted by pip

it makes no sense to me either. by no means pip reads the config under tool.pdm

from pdm.

SealedServant avatar SealedServant commented on June 10, 2024

It looks like it is being emitted by pip

it makes no sense to me either. by no means pip reads the config under tool.pdm

Found this: https://stackoverflow.com/questions/43046544/unusual-error-while-installing-any-pip-based-package

I think I know why, the requirements.txt exported by pdm is including the indices at the bottom. Also note that it is including the sys_platform for select packages:

# This file is @generated by PDM.
# Please do not edit it manually.

-----(truncated)-----
ghp-import==2.1.0
griffe==0.44.0
httpx==0.27.0
identify==2.5.36
mypy-extensions==1.0.0
nodeenv==1.8.0
-----(truncated)-----
pywin32==306; sys_platform == "win32"
-----(truncated)-----
text-unidecode==1.3
tomlkit==0.12.4
types-python-dateutil==2.9.0.20240316
watchfiles==0.21.0
wcwidth==0.2.13
websockets==12.0
zipp==3.18.1
--index-url https://${USERNAME}:${PASSWORD}@xxx.xxx.xxx/repository/pypi-proxy/simple/
--find-links file:///${PROJECT_ROOT}/.packages

from pdm.

frostming avatar frostming commented on June 10, 2024

fine. it makes sense. now no need to complain anymore and try the main branch with pdm-download

from pdm.

SealedServant avatar SealedServant commented on June 10, 2024

fine. it makes sense. now no need to complain anymore and try the main branch with pdm-download

Sorry @frostming. I really like pdm and I am trying to avoid using poetry if I can help it 😄 Thanks for all the time you have put into pdm and helping people get issues resolved.

I tried the main branch and I tried coding the absolute path to the .packages folder with url = "file:///C:/Users/.../.../.packages" just to make sure it would work.

I am not sure why, but pdm install --no-isolation is still failing offline. It appears that termui / unearth are still trying to connect to pypi to obtain setuptools (also editables and pdm-backend) despite the fact that I have added these packages to the project dependencies in pyproject.toml, the local .packages folder, and pdm itself with command pdm self add pdm-backend editables setuptools.

I appears that pdm is not attempting to resolve the additional sources in the pyproject.toml and stops after the first source resolution failure.

unearth.collector: Collecting links from https://***@***.***.***.***/repository/pypi-proxy/simple/setuptools/
unearth.auth: Found credentials in url for ***.***.***.***
pdm.termui: Error occurs
Traceback (most recent call last):
  File "C:\pdm\Lib\site-packages\httpx\_transports\default.py", line 69, in map_httpcore_exceptions
    yield
  File "C:\pdm\Lib\site-packages\httpx\_transports\default.py", line 233, in handle_request
    resp = self._pool.handle_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpcore\_sync\connection_pool.py", line 216, in handle_request
    raise exc from None
  File "C:\pdm\Lib\site-packages\httpcore\_sync\connection_pool.py", line 196, in handle_request
    response = connection.handle_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpcore\_sync\connection.py", line 99, in handle_request
    raise exc
  File "C:\pdm\Lib\site-packages\httpcore\_sync\connection.py", line 76, in handle_request
    stream = self._connect(request)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpcore\_sync\connection.py", line 122, in _connect
    stream = self._network_backend.connect_tcp(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpcore\_backends\sync.py", line 205, in connect_tcp
    with map_exceptions(exc_map):
  File "C:\Program Files\Python312\Lib\contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "C:\pdm\Lib\site-packages\httpcore\_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: [Errno 11001] getaddrinfo failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\pdm\Lib\site-packages\pdm\termui.py", line 257, in logging
    yield logger
  File "C:\pdm\Lib\site-packages\pdm\cli\actions.py", line 238, in do_sync
    synchronizer.synchronize()
  File "C:\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 445, in synchronize
    handlers[kind](key, progress)
  File "C:\pdm\Lib\site-packages\pdm\installers\synchronizers.py", line 282, in install_candidate
    self.manager.install(can)
  File "C:\pdm\Lib\site-packages\pdm\installers\manager.py", line 33, in install
    prepared.build(),
    ^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\models\candidates.py", line 402, in build
    self._obtain(allow_all=False)
  File "C:\pdm\Lib\site-packages\pdm\models\candidates.py", line 438, in _obtain
    self.link = _find_best_match_link(
                ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\models\candidates.py", line 105, in _find_best_match_link
    found = attempt_to_find()
            ^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\pdm\models\candidates.py", line 96, in attempt_to_find
    best = finder.find_best_match(req.as_line(), hashes=hashes).best
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\finder.py", line 391, in find_best_match
    best_match = next(iter(applicable_candidates), None)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\finder.py", line 324, in _find_packages_from_requirement
    yield from self._find_packages(requirement.name, allow_yanked)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\finder.py", line 290, in _find_packages
    return sorted(all_packages, key=self._sort_key, reverse=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\collector.py", line 178, in collect_links_from_location
    yield from _collect_links_from_index(session, location)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\collector.py", line 198, in _collect_links_from_index
    page = fetch_page(session, location)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\collector.py", line 185, in fetch_page
    resp = _get_html_response(session, location)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\unearth\collector.py", line 220, in _get_html_response
    resp = session.get(
           ^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 1054, in get
    return self.request(
           ^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 827, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 914, in send
    response = self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_client.py", line 1015, in _send_single_request
    response = transport.handle_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\hishel\_sync\_transports.py", line 197, in handle_request
    regular_response = self._transport.handle_request(request)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\pdm\Lib\site-packages\httpx\_transports\default.py", line 232, in handle_request
    with map_httpcore_exceptions():
  File "C:\Program Files\Python312\Lib\contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "C:\pdm\Lib\site-packages\httpx\_transports\default.py", line 86, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 11001] getaddrinfo failed

from pdm.

SealedServant avatar SealedServant commented on June 10, 2024

@frostming, thank you for all the support. I was finally able to get everything to work seamlessly. I want to take some notes here for other users who might find this in the future. It would be nice if the documentation could have notes on using PDM offline, maybe in the advanced usage section.

Full permission granted to copy/paste/modify this for PDM documentation.

Use PDM in Offline Environments

Prerequisites

You must ensure that your build dependencies are in your pyproject.toml.

For example, if you are using pdm-backend like so

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"

then the following dependencies are required to use pdm-backend offline with the --no-isolation flag

[tool.pdm.dev-dependencies]
build = ["editables>=0.5", "pdm-backend>=2.3.0"]

Download the Dependencies

On an online system, ensure the PDM project is initialized normally:

pdm install

Make a directory .packages:

mkdir .packages

Export the dependencies to .packages/.requirements.txt:

pdm export --format requirements --output .packages/.requirements.txt --no-hashes --no-markers

Download the dependency wheels into the directory:

pdm run pip wheel --requirement .packages/.requirements.txt --wheel-dir .packages 

Download PDM

On an online system, create a virtual environment to install pdm into:

python -m venv .pdm

Activate the virtual environment with the platform-specific script located in .pdm/Scripts and install pdm to the virtual environment:

pip install pdm

Make a directory .pdm-packages:

mkdir .pdm-packages

Export the dependencies to .pdm-packages/.requirements.txt:

pip freeze > .pdm-packages/.requirements.txt

Download the pdm dependencies as wheels into the directory:

pip wheel --requirement .pdm-packages/.requirements.txt --wheel-dir .pdm-packages 

Optionally, perform a git clean to prepare the project for the offline system. Make sure to exclude the .packages and .pdm-packages directory.

WARNING: git clean -fdx will remove any files not tracked by git and any files included in .gitignore, such as a .venv inside the project. Since virtual environments are not portable, it is recommended to recreate it on the offline system.

git clean -fdx --exclude .packages --exclude .pdm-packages

Install PDM

On the offline system, install pdm to a directory of your choice. It is recommended to install pdm into a virtual environment.

With your virtual environment activated, install the packages from .pdm-packages:

pip install -r .pdm-packages/.requirements.txt --no-index --find-links .pdm-packages

Once pdm and it's dependencies are installed to the virtual environment, it is recommended to make a symlink to the pdm executable and add it to your system path.

Install the Dependencies

On the offline system, ensure that pdm is installed and available on the system PATH.

In the project directory you have transferred, make sure that pip is installed:

  pdm run python -m ensurepip --default-pip

Install the dependencies:

pdm run pip install -r .packages/.requirements.txt --no-index --find-links .packages

Check the project file for changes, update the lock file if needed, sync with the lock file, and install the project as editable:

pdm install --no-isolation

from pdm.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.