Comments (9)
Among RFC 1, 2, and 3, IMO this is the best option for the long-term. I mean, in the future, we hope people can upload the Emscripten wheels to PyPI or to other package indices, and then people can create a lock file from it.
Then people who are okay with resolving dependencies dynamically can use Simple API (supported by micropip) to download packages from package indices, while people who make an application with fixed dependencies and don't want dependency resolution can create a lockfile.
from pyodide-lock.
@ryanking13 could you rename PyPA (Python Packaging Authority) to PyPI (Python Packages Index) in that script? It's kinda confusing to see.
Sure. Thanks for pointing that out.
from pyodide-lock.
I'd prefer the second option, to create a separate lockfile from the original pyodide-lock file, [...], and I suspect that this would cause version conflicts between duplicate packages.
@ryanking13 Not necessarily, I think, if you have some priority between channels (--extra-index-url in pip pip option for instance) it should pick the version in the highest priority channel .
from pyodide-lock.
Expose a Simple JSON API for the Pyodide distribution (so that Python package managers could understand it). This should already be done, but I can't find where (cc @ryanking13 )
Right now this is done in pyodide-build/create_pypa_index.py, so it can be used in pip
after calling pyodide venv
. By the way, it uses Simple HTML API but I think we better change it to Simple JSON API.
from pyodide-lock.
By the way, it uses Simple HTML API but I think we better change it to Simple JSON API.
Thanks! Absolutely. We cannot deploy HTML with the current CDN.
from pyodide-lock.
A Simple JSON API for this experimentation can be found in pyodide/pyodide#3979
from pyodide-lock.
@juntyr This could also be a useful direction to explore.
The main part is, take an example list of requirements, some standard Python packaging tool (TBD), specify that the supported wheels are only of the form *cp311-emscripten_3_1_32_wasm32.whl
or *py3-none-any.whl
, point it to the Pyodide's Simple JSON API (above) + PyPI and ask it to generate a lock file (see the issue description above for more details). If you have any questions let me know.
For instance, jazzband/pip-tools#585 was essentially this feature request, but they decided it's out of scope for pip-compile.
Maybe @pradyunsg would also have some general advice or comments on this topic.
from pyodide-lock.
This is what pyodide build does - using resolvelib.
Incidentally, I do wonder how much this could be integrated into existing pyodide build functionality by making pyodide_build use this library to output a pyodide-lock.json. Or maybe just nick the logic from pypi.py in pyodide_build
e.g. thinking about use cases
-
You're building your own wheel from source (plus dependencies which are a combination of pypi and things built into pyodide) -
pyodide build --build-dependencies
does the correct resolution here, all it needs to do is generate a pyodide-lock.json instead of the current output which is a lockfile. -
You want to include a wheel from pypi and all non-pyodide dependencies.
pyodide build <pypi-version-spec> --build-dependencies
does the resolution here, again needs to generate a lockfile. -
You want to get multiple pypi wheels and dependencies
pyodide build -r requirements.txt --build-dependencies
-
You have a bunch of wheels, and you want to create a lockfile that includes all of them and their dependencies
pyodide build -r requirements.txt --build-dependencies
with local paths to the wheels in the requirements.txt (right now I think local paths don't work, but is a trivial addition to the code.
from pyodide-lock.
Right now this is done in pyodide-build/create_pypa_index.py, so it can be used in
pip
after callingpyodide venv
. By the way, it uses Simple HTML API but I think we better change it to Simple JSON API.
(with my PyPA hat on)
@ryanking13 could you rename PyPA (Python Packaging Authority) to PyPI (Python Packages Index) in that script? It's kinda confusing to see.
For instance, jazzband/pip-tools#585 was essentially this feature request, but they decided it's out of scope for pip-compile.
That's not entirely accurate, but is a rather complicated topic. That issue was closed automatically because of a linked documentation PR, while there are other issues centered around the same discussion, including mine. Here's a few comments I made there that might be of interest when we're talking about "lockfiles": jazzband/pip-tools#826 (comment) / jazzband/pip-tools#1326 (comment).
Though, is it just a single platform in case of pyodide? FWIW it might be a good idea to participate in reviving PEP 665.
from pyodide-lock.
Related Issues (11)
- 0.1.0a2 not deployed to PyPI HOT 2
- Make cli run under pyodide-cli? HOT 1
- Pin to pydantic <2
- pyodide lockfile add-wheels does not append files
- Generate JSON Schema HOT 3
- Move test file unvendoring functionality from pyodide-build to pyodide-lock
- Design CLI API HOT 6
- Move parse_top_level_import_name from Pyodide build HOT 2
- RFC 1: micropip.freeze() → `pyodide-lock.json`
- RFC 2: partial lockfiles HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyodide-lock.