Comments (8)
Can you give a reproducible example? Very likely there's been a change in the dependency tree that's hitting a pathological use case. I might be able to provide an additional constraint on your requirements as a workaround to this error.
If you would like, I have a branch where I have tried to significantly reduce the chance of this happening, here is an example how how to install #12305 (comment). Feedback on if this branch helped or not would be helpful to me.
from pip.
Thanks @notatallshaw , let me give that a try, so you are suggesting to install pip from your branch. Is there any restriction on the version of setuptools to be used?
from pip.
@notatallshaw that worked like a charm! But how do we consume this, is it going to be merged to pip:main and available as a version of pip that we can directly install?
from pip.
My hope is eventually it will be merged into pip main, but I have not had chance to work on it recently, and getting it into pip requires multiple steps.
If you can give me your requirements that cause this issue for you, I can likely identify how to fix them so it works with regular pip.
from pip.
@notatallshaw our project has the following dependencies in setup.cfg for our project is internal to our org.
auditree-arboretum>=0.5.0
deprecated>=1.2.10
ibmcloud_tools[cos]>=2.36.0
python-dateutil<3.0.0
SoftLayer>=5.7.2
sdcclient>=0.10.0
pygithub
kubernetes>=21.7.0
pytenable
strenum
ibm-platform-services
ibm_cloud_sdk_core
Since yesterday we started to see our build failing with pip trying to resolve dependency by getting a faulty old version of pytenable (1.4.2), this halted our installation.
Then we tried to pin the version of pytenable to >=1.4.13, but this helped get rid of the failure with pytenable 1.4.2, but then we started to consistently see the pip._vendor.resolvelib.resolvers.ResolutionTooDeep: 200000 errors and looking at the logs it is not always throwing this error after getting a certain package or while trying to check dependencies for a specific package.
We are using pip version 24.0 and setuptools 70.0.0, we tried to downgrade the versions of these tools as well, but to no avail. Really appreciate your help with this.
This is the dependency tree got by running pipdeptree from. my project
deptree.txt
from pip.
I just tried this, but there's no project called ibmcloud_tools
on PyPI. Could you provide a full, reproducible example?
from pip.
I was able to reproduce by looking at the deptree and substituting in the non-public packages with their dependencies, I was testing on Python 3.11 Linux and I found I had to add a few extra lower bounds so pip didn't try to build a package too old that building just failed:
auditree-arboretum>=0.5.0
deprecated>=1.2.10
configparser
python-dateutil
PyYAML
requests
ilcli>=0.3.2
PyJWT>=2.6.0
python-dateutil
requests>=2.28.2
configparser
python-dateutil
PyYAML
requests
python-dateutil<3.0.0
SoftLayer>=5.7.2
sdcclient>=0.10.0
pygithub>=1.58.0
kubernetes>=21.7.0
pytenable>=1.4.12
strenum>=0.4.10
ibm-platform-services>=0.30.3
ibm_cloud_sdk_core>=3.16.1
GitPython
ibm-cos-sdk >=2.12.1
ibm-platform-services >=0.22.5
There's probably a simpler set of requirements to reproduce this, but it takes a long time to test reproducing.
First, let's check if this resolves quickly from two days ago, testing using pypi-timemachine
I set the date to "2024-06-05", and indeed it resolved very quickly. So the only thing that changed was the ecosystem, not pip or setuptools.
Looking at what packages downloaded instead of using cached versions when I ran the version against "2024-06-05" the big thing that stood out to me was sdcclient which had a release on the 6th June.
If I put an upper bound on sdcclient sdcclient>=0.10.0,<0.17.2
then indeed it quickly resolves again! So this is our culprit but it is not satisfying to put an upper bound on it and leave it at that.
Comparing the metadata from the two releases:
- https://files.pythonhosted.org/packages/68/45/837a47a66ee48f38996aafc5c12f7c00c5aa9f62223e3e9e7aecbf0ae71a/sdcclient-0.17.2-py3-none-any.whl.metadata
- https://files.pythonhosted.org/packages/31/c8/645fae65dba11dbeb7d49f4b134bbaa7ac6f2bf294a8e27d26e23c19af7d/sdcclient-0.17.1-py3-none-any.whl.metadata
There are two significant changes requests-toolbelt (>=0.9.1,<0.10.0)
to requests-toolbelt (>=1.0.0,<2.0.0)
and urllib3 (>=1.26.0,<2.0.0)
to urllib3 (>=2.2.1,<3.0.0)
, we can find the likely culprit by swapping out sdcclient>=0.10.0
with either of these two. And indeed, the culprit reveals itself to be urllib3 (>=2.2.1,<3.0.0)
, swapping that in produces the error again.
A little further investigation revealed that ibm-cos-sdk-core always depends on some urllib3 <1.27 and therefore is not comatible with sdcclient 0.17.2.
So your solution for now is to:
- Put an upper bound sdcclient<0.17.2
- And, reach out to
ibm-cos-sdk-core
and ask them to removeurllib3 <1.27
unless they know for sure newer versions break their framework, and in general remind library authors not to eagerly put upper bounds on dependencies unless they have good reason (https://iscinumpy.dev/post/bound-version-constraints/) as it generally causes long term damage to the ecosystem - Politely ask sdcclient to loosen their requirements also, urllib3 1.x is still supported, are they actually already depending on 2.x features?
Hope that helps.
Pip is particularly bad at handeling these situations where it encounters a bad upper bound deep into the dependency tree. But this is a duplicate of #12305 and hopefully we can eventually get pip much better.
from pip.
@notatallshaw Thanks a ton for helping debug the problem for us. It immensely helped us to understand the whole problem we faced wrt the sudden change in our project's behaviour in dependency resolution without any changes from our end.
Agree that this is a duplicate of #12305 and I hope your fix for the same can make it to a future pip version soon.
from pip.
Related Issues (20)
- [24.2] installs fail on macOS 10.12 and older due to bundled `truststore` issue HOT 9
- `pip install torch` reinstalls it even if no `--upgrade` flag is specified HOT 2
- Multiple config paths treated as single path HOT 6
- Using a private package index with pip 24.2 results in `ValueError: check_hostname requires server_hostname` HOT 6
- Pip CI should be tested against MacOS ARM (M1) HOT 5
- `python -m pip` behaves differently from `pip` (regarding stubs installation) with `pyenv` Python HOT 3
- index-url extra-index-url install priority order HOT 1
- pip incorrectly interprets version symlinked whl file name HOT 3
- pip accepts non-PEP440 version `0.1_1` in wheel file name HOT 1
- Switch default pip install compile option to `False` HOT 6
- metadata resolve workstream HOT 9
- pip index versions is inconsistent in its output when package has not been installed or only one version is available HOT 1
- Disable defaulting to user install when site-packages not writeable HOT 5
- pip install prints the wrong packages installed HOT 6
- stupid upgrade HOT 1
- Wrong info message shown for installation from URL HOT 3
- pip inspect shows wrong installer HOT 1
- Reduce license verbosity in `pip show` output HOT 1
- EOFError: "pip subprocess to install build dependencies" asks for auth for `extra-index-url` inside subprocess, where user can't enter it HOT 1
- pip doesn't install sdist formats other than tar.gz and zip HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pip.