sbidoul/pip-deepfreeze

A simple pip freeze workflow for Python application developers

pip-deepfreeze's Issues

Support overrides

uv supports overrides, and pip may support them at some point.

Design

• detect overrides.txt
• when installing, remove entries that are in requirements*.txt from overrides.txt
• pass the resulting overrides to the install command with -o (uv only for now)

Add option to update transivitive dependencies

When using --update DEP it could be useful to also update transitive dependencies of DEP.

Pin pip, setuptools, wheel and distribute when they are dependencies

When pip, setuptools, wheel and distribute , they are never present in locked requirements.
This is because pips omits them by default in pip freeze output.

We change to call pip freeze --all but never propose to uninstall these as they may be necessary to build legacy projects without build isolation.

Support extras

Add `lock` command when the target pip supports `pip install --dry-run --report`

Don't warn about `wheel` being absent when using pip >= 23.1

pip now builds in an isolated environment when wheel is absent, so no need to warn about it.

Pass `--no-input` to pip commands

This is important to avoid blocking silently.

Environment sanity checks

• verify pip, setuptools and wheel are installed
• check pip version, and recommend >=20.1

Support environments where pkg_resources is not available

The script that inspects the content of the environment currently relies on pkg_resources, hence setuptools. In modern packaging, setuptools is not necessarily installed in the virtual environment. So we should have a mode that works with importlib[._]metadata.

Support uv

Support PEP 660 (editables)

Do not use pip install --constraints

Pip constraints have limitations, especially with the new resolver.

Error when adding extras to an already installed project

To reproduce, create a project with a setup.py and pip-df sync it.
Then add and extras_require with an extra named, say, dev.
Then run pip-df sync -e dev. It complains that dev is not an extra of the project.

To make it work, you need to pip uninstall the project then pip-df sync again.

Better UX when there is no setup.py nor pyproject.toml

Use pip inspect when available

Instead of our pip_list_json.py script. This will also resolve #54.

Option to uninstall unneeded dependencies

Virtualenv detection may be too strict

From the FAQ: pip-deepfreeze erroneously complains python is not running in a virtualenv.

The most probable cause is that you used an older version of virtualenv which does not generate PEP 405 compliant virtual environments. virtualenv version 20 and later are supported, as well as the Python 3 native venv module. Should this problem be prevalent in practice, we may add support for older virtualenv versions, or add an option to ignore the virtualenv sanity check (which is only there to prevent pip-deepfreeze to corrupt the system Python packages by accident).

Add --verbose option

To control debug level logging

Switch toml parser

Looks like the packaging community is moving to tomli, which is smaller and fully toml 1.0 compliant.

Do not print a stack trace on pip errors

pkg_resources.VersionConflict error when downgrading an already installed dependency

How to reproduce:

1. create project depending on packaging
2. install it
3. update project to depend on packaging<20.1

=> stack trace pkg_resources.VersionConflict

Update not working for dependecy installed from GIT hidden ref if ref comes from a fork

Github allows you to reference a pull request as a hidden ref you can fetch. This feature is useful to keep a log of you open PR required by your project.

In order to declare a dependency to a new patched version of one of my project's dependencies, I've added into my requirements.txt.in the following line:

odoo10-addon-account-banking-mandate @ git+https://github.com/oca/bank-payment.git@refs/pull/727/head#subdirectory=setup/account_banking_mandate

and used the update command to install and freeze this new version of my dependency
pip-df sync --update odoo10-addon-account-banking-mandate

(PR from acsone/bank-payment to oca/bank-payment)

At the end of the process, the dependency is correctly installed however the updated entry into my requirements.txt file is not right. Indeed, even if the sha is the right one, the ORG into the github url is the one on which the pr has been made not the one from which the changes are submitted....

odoo10-addon-account-banking-mandate @ git+https://github.com/oca/bank-payment.git@84d5fd2cc8c9bbca6b036ad5f8d87776d1b3fdad#subdirectory=setup/account_banking_mandate

Is-it a know limitation or a bug?

Warn if not in virtualenv?

What criteria to detect a safe situation?
What if in a virtualenv with system site packages?

Search for `py` before `python` when looking for the default python interpreter

Don't show a stack trace when post sync commands fail

Rename requirements.txt.in to constraints.txt

requirements.txt.in is somewhat misnamed, as it actually contains pip constraints and options.

constraints.txt would more accurately reflect what the file is for, i.e. constraining dependencies when it is not practical or desired to put such constraints in pyproject.toml project.dependencies.

Detect egg= requirement names

Better UX when the project does not support editable installs

--update/--update-all not working in some edge cases

When a new dependency is added to the project, and it is already installed while not being part of requirements.txt, using --update or --update-all may not update it to the latest available version.

This is a very edge case that is probably not an issue in practice.

assertion error when running pip-df sync -e with unknown extras

When running pip-df sync -e with unknown extras, an assertion error is raised.

The desired behaviour is to warn the user but otherwise ignore the unknown extras.

Allow non-editable install

It would appear that pip-deepfreeze tries to install the local package in editable mode:

pip install -e .

Unfortunately, I have a case where this does not work reliably. The reason being that I have multiple local packages which pip does not resolve well. The reason being that pip believes packages to be conflicting if one package is installed in editable mode, and the other is installed in non-editable mode from a path. There are also issues where pip believes there is a conflict if the paths are not identical (e.g. /src/pkg-b/../pkg-a and /src/pkg-a are the same, but pip will see a conflict).

I could not see an option with pip-deepfreeze to allow a non-editable installation. Could this be added behind a flag?

Generate hashes in requirements.txt

This will need to be implemented in pip freeze first.

Platform awareness

Currently, the naming scheme of generated requirement files is hard coded (requirements.txt, requirement-{extra}.txt) and is generated for the platform/interpreter on which pip-deepfreeze runs.

It might be useful to have an optional naming scheme that includes platform/interpreter information.

Support environments where pip is not installed

This could be done by providing pip-deepfreeze with a target pip command in addition to a target python interpreter.
That pip command could then be target-pip --python {target-python} ....

Default extras

Developers typically need to remember to use pip-df sync -x test,dev. This is not optimal.

Two possibilities:

• sync install all extras by default
• configure default extras in pyproject.toml, in a tool.pip-deepfreeze section.

Declare minimum pip-deepfreeze version in pyproject.toml

tool.pip-deepfreeze.min_version