paketo-buildpacks/poetry-install

Poetry Install Cloud Native Buildpack

gcr.io/paketo-buildpacks/poetry-install

The Paketo Buildpack for Poetry Install is a Cloud Native Buildpack that installs packages using Poetry and makes the installed packages available to the application.

The buildpack is published for consumption at gcr.io/paketo-buildpacks/poetry-install and paketobuildpacks/poetry-install.

Behavior

This buildpack participates if pyproject.toml exists at the root the app.

The buildpack will do the following:

• At build time:
  • Creates a virtual environment, installs the application packages to it, and makes this virtual environment available to the app via a layer called poetry-venv.
  • Configures poetry to locate this virtual environment via the environment variable POETRY_VIRTUAL_ENVS_PATH.
  • Prepends the layer poetry-venv onto PYTHONPATH.
  • Prepends the bin directory of the poetry-venv layer to the PATH environment variable.
• At run time:
  • Does nothing

Integration

The Poetry Install CNB provides poetry-venv as a dependency. Downstream buildpacks can require the poetry-venv dependency by generating a Build Plan TOML file that looks like the following:

[[requires]]

  # The name of the dependency provided by the Poetry Install Buildpack is
  # "poetry-venv". This value is considered part of the public API for the
  # buildpack and will not change without a plan for deprecation.
  name = "poetry-venv"

  # The Poetry Install buildpack supports some non-required metadata options.
  [requires.metadata]

    # Setting the build flag to true will ensure that the poetry-venv
    # dependency is available on the $PYTHONPATH for subsequent
    # buildpacks during their build phase. If you are writing a buildpack that
    # needs poetry-venv during its build process, this flag should be
    # set to true.
    build = true

    # Setting the launch flag to true will ensure that the poetry-venv
    # dependency is available on the $PYTHONPATH for the running
    # application. If you are writing an application that needs poetry-venv
    # at runtime, this flag should be set to true.
    launch = true

Usage

To package this buildpack for consumption:

$ ./scripts/package.sh --version x.x.x

This will create a buildpackage.cnb file under the build directory which you can use to build your app as follows: pack build <app-name> -p <path-to-app> -b <cpython buildpack> -b <poetry buildpack> -b build/buildpackage.cnb -b <other-buildpacks..>.

To run the unit and integration tests for this buildpack:

$ ./scripts/unit.sh && ./scripts/integration.sh

Known issues and limitations

• This buildpack will not work in an offline/air-gapped environment: vendoring of dependencies is not supported. This is a limitation of poetry - which itself does not support vendoring dependencies.

poetry-install's Issues

Failure: Publish draft releases

Publish All Draft Releases workflow failed.

[Question ❓] How to install only "root/run" dependencies and ignore other groups dependencies

General summary of the issue

I am suprised that running "pack build" with paketo-buildpacks/poetry-install installs all the dependencies described in my pyproject.toml file.

For production, I only want to install run dependencies (in [tool.poetry.dependencies] section), dependencies in other groups (such as [tool.poetry.group.dev.dependencies]) are not necessary and make the built image heavier than necessary

Expected Behavior

When running pack build [my_image] --builder paketobuildpacks/builder:base (using paketo-buildpacks/poetry-install in Detection phase) I expect paketo-buildpacks/poetry-install to run the following command :

POETRY_CACHE_DIR=/layers/paketo-buildpacks_poetry-install/cache POETRY_VIRTUALENVS_PATH=/layers/paketo-buildpacks_poetry-install/poetry-venv poetry install --only main

Having the following dependencies in pyproject.toml, I expect only fastapi and uvicorn to be installed, without pytest ✅ :

[tool.poetry.dependencies]
python = "^3.11"
fastapi = "^0.100.0"
uvicorn = {extras = ["standard"], version = "^0.23.1"}


[tool.poetry.group.dev.dependencies]
pytest = "^7.4.0"

Current Behavior

When running pack build [my_image] --builder paketobuildpacks/builder:base (using paketo-buildpacks/poetry-install in Detection phase), I see in the log that the following command is run :

POETRY_CACHE_DIR=/layers/paketo-buildpacks_poetry-install/cache POETRY_VIRTUALENVS_PATH=/layers/paketo-buildpacks_poetry-install/poetry-venv poetry install

Having the following dependencies in pyproject.toml, pytest is installed 😢 :

[tool.poetry.dependencies]
python = "^3.11"
fastapi = "^0.100.0"
uvicorn = {extras = ["standard"], version = "^0.23.1"}


[tool.poetry.group.dev.dependencies]
pytest = "^7.4.0"

I see the following logs in the console :

Paketo Buildpack for Poetry Install 0.3.16
  Executing build process
    Running 'POETRY_CACHE_DIR=/layers/paketo-buildpacks_poetry-install/cache POETRY_VIRTUALENVS_PATH=/layers/paketo-buildpacks_poetry-install/poetry-venv poetry install'
      Installing dependencies from lock file
      
      Package operations: 4 installs, 0 updates, 0 removals
      
        • Installing iniconfig (2.0.0)
        • Installing packaging (23.1)
        • Installing pluggy (1.2.0)
        • Installing pytest (7.4.0)

Question

I don't see any configuration or environment variable to add arguments to "poetry install" command in the README.md file, is there something that I missed ?

Steps to Reproduce

Tell me if this is necessary and I can setup a repository with reproducible steps, as it can take me some time to do it. I'll do it happily :)

Motivations

How has this issue affected you? What are you trying to accomplish? What is the impact? Providing context helps us come up with a solution that is most useful in the real world.

In the current situation, having poetry installing all dependencies, even the ones I do not need is a no-go for any of my projects in production.

Failure: Go get update workflow

Go get update workflow failed.

Explore poetry install --sync

Describe the Enhancement

Poetry 1.2.0 adds support for a --sync flag to the poetry install command. Official docs. We should evaluate whether this is something we should add to the Poetry Install buildpack.

Failure: Push Buildpackage workflow

Push Buildpackage workflow failed.

Failure: Create Draft Release workflow

Create Draft Release workflow failed.

It is possible and desirable to skip lock file creation?

Running poetry install creates a poetry.lock file. Typically this file is checked in and already exists when this buildpack runs. If it doesn't exist, however, this buildpack will create it.

It isn't actually required for poetry run (or running python some-file.py via a Procfile), as we already populate the specific versions of the application dependencies in the virtual environment, so this could potentially be an unnecessary step.

Can we skip the generation? Is this a desirable step?

Conform to Paketo RFC0038: Generate SBOM for application dependencies

To implement Paketo RFC0038, this buildpack will need store SBOM information for application dependencies in files that the CNB lifecycle can manipulate during the build. The RFC outlines what these files are and what they should contain.

See also paketo-buildpacks/python#399.

See the nodejs buildpack integration tests for an example on how to test this.

Leverage `scribe.Emitter` functionality

This buildpack uses scribe.NewFormattedMapFromEnvironment to log environment variables, when we could instead use logger.EnvironmentVariables. We should make this change as it is simpler and provides an improved log output.

See also paketo-buildpacks/cpython#337

Implement poetry-install

Background

RFC 0006 outlines the proposed addition of support for Poetry in the Python buildpack. In order to conform to the RFC, a new buildpack named poetry-install should be created.

Proposal

Create a buildpack using packit that installs application dependencies using poetry.

The buildpack API should be as follows:
provides: poetry-venv
requires: cpython, pip, and poetry during build

Failure: Approve bot PR workflow

Approve bot PR workflow failed.

Upgrade to packit v2

Please upgrade to the latest packit v2 release to enable new features and extended support.

Confirm to RFC RFC0043: Reproducible builds

To confirm to RFC0043 this buildpack should ensure that builds are reproducible. Specifically it should not include a built_at metadata field. In the tests that leverage this field to assert layer reuse, we should instead compare layer SHA values across rebuilds.

See also the tracking issue: paketo-buildpacks/rfcs#165.