Comments (5)
Ok, thanks for confirming! If you are curious, I found these by iterating through every single record in the channel and trying to resolve them individually.
from rattler.
I've found a performance improvement that fixes these cases (they resolve instantly) but makes all other solves about 50% slower... Need more investigation. 😅
from rattler.
Thanks for these! I can confirm that these are horribly slow and instant with libsolv. We will investigate!
from rattler.
We found a solution for this case by changing our selection heuristic. Currently we are deciding packages with few options first, in order to minimize backtracking. But it seems to not play well with Python constraints since apparently we are doing excessive backtracking.
If we reverse the heuristic, things are super fast (ie. decide packages with many options first).
Did you write some kind of benchmark script @JeanChristopheMorinPerso? I think we'll adjust the heuristic, but it would be great to have more extensive benchmarks to check that we don't introduce big regressions.
For the "fix", I just changed the <
to >
in this line: https://github.com/mamba-org/resolvo/blob/56fa93de027f483223c6525671e243d89cd805fe/src/solver/mod.rs#L761
from rattler.
Great news!
Did you write some kind of benchmark script @JeanChristopheMorinPerso? I think we'll adjust the heuristic, but it would be great to have more extensive benchmarks to check that we don't introduce big regressions.
I don't think what I have can be called a benchmark... I basically take repodata and try to resolve every single "latest" record one by one (including all its variants).
Something like
import collections
import rattler
timedelta = collections.namedtuple("timedelta", ["microseconds"])
channels = [rattler.Channel["main"]]
platforms = [rattler.Platform("osx-arm64"), rattler.Platform("noarch")]
virtual_packages = [p.into_generic() for p in rattler.VirtualPackage.current()]
repo_datas = await rattler.fetch_repo_data(
channels=channels,
platforms=platforms,
cache_path="/tmp/py-rattler-cache/repodata",
callback=None,
)
for subdir in repo_datas:
for package_name in subdir.package_names():
all_records = sorted(
subdir.load_records(rattler.PackageName(package_name)),
key=lambda x: (x.version, x.build_number),
)
latest = all_records[-1]
records = [
r
for r in all_records
if r.version == latest.version and r.build_number == latest.build_number
]
futures = []
for record in records:
futures.append(
packages = await rattler.solve(
channels,
[rattler.MatchSpec(str(record))],
platforms=platforms,
virtual_packages=virtual_packages,
timeout=timedelta(microseconds=30 * 1000000),
)
)
await asyncio.gather(*futures)
(I did not test this specific code)
I'm doing this to see if a channel is solvable and also to do some channel analysis and thought it would be simpler and faster to use rattler instead of either using subprocs to conda conda create or use libmamba.
from rattler.
Related Issues (20)
- Fix panic in matchspec parser HOT 1
- Race condition in `link_file` leads to spurious errors (and linking fails) HOT 3
- [Feature Request] Add functionality to install package files HOT 6
- `repodata not found` error on valid channel containing only `noarch` subdir HOT 4
- [py-rattler] AttributeError: 'NoneType' object has no attribute 'text' when a PathsEntry has no filemode HOT 2
- Bug: `PrefixRecord` Parses Incorrect Version for some cal-ver Versioned Packages HOT 2
- CTRL-C doesn't work with py-rattler HOT 6
- [Bug] PyPrefixPathType Has no attribute `windows_python_entry_point_script` HOT 1
- about.json - not all fields are parsed HOT 2
- pyrattler: authenticated client should also use retry and oci middlewares
- RepodataGateway Reporter - more information
- Cannot solve cmdstan=2.33.1=hc24a2e5_0 (linux) from osx-arm64 HOT 12
- Add `VersionSpec` python bindings
- The python bindings quick start example is out of date
- Timestamp in RepoDataRecord is in seconds instead of milliseconds HOT 10
- Dependency graph from a resolve HOT 7
- Use `rustc-hash` instead of `fxhash` HOT 2
- how do you feed entries from `run_constrains` to the `solver` function in py-rattler? HOT 2
- `InvalidBracket` for matchspec HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rattler.