frostming/pdm-packer

A PDM plugin that packs your packages into a zipapp

pdm-packer's Issues

Can I hide console window and set an icon for exe?

Like PyInstaller’s -w and -i.

Support excluding py files in the zipapp

The problem with git-lfs....

Unless git-lfs is installed (and configured possibly requiring extra paid services from GitHub) a plain git clone of this repo fails.
This is creating problems like @kloczek reported in pdm-project/pdm#1084

And I reckon that your position is pdm-project/pdm#1084 (comment)

Checking in binary files in a git repository is not a good practice. Because Git will store all histories in the .git directory, if the file is a binary the storage is not efficient: it can't store the delta changes like a plain text file.

I understand your point, but here this is eventually creating difficulties for any adopter and aspiring contributor. IMHO this may not be the effect you are looking for? The whole set of wheels and archives is about 3.1MB total and this are binaries but not what I would call large files.

I would suggest to consider using something else:

• just copy the test wheels inside the repo... these are not super big and not evolving quickly
• or use a separate repo just for the test wheels and use git modules to get it
• or put these in a tarball under a release of this repo and fetch these from there

pdm packer cant run compiler from zipapp distributed pdm

file only works if the python module is actually stored as a file... this is not necessarily the case. When running packer from a zip app.:

Local Python environment used as shebang in pyz

Describe the bug

The shebang in my pyz file points to the build-time project venv. This won't exist on target environments.

To reproduce

1. Create local environment with pdm sync --dev
2. Build pyz file with pdm pack
3. Open pyz file in a text editor and inspect first line

Expected behavior

To the best of my knowledge, the universal shebang for Python 3 is:

#!/usr/bin/env python3

Instead I see:

#!/home/user/Documents/myproject/.venv/bin/python

System

$ pipx list --include-injected
...
   package pdm 2.5.2, installed using Python 3.10.4
    - pdm
    Injected Packages:
      - keyring 23.9.3
      - pdm-packer 0.6.0

$ lsb_release --all
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.2 LTS
Release:	22.04
Codename:	jammy

Cannot find psycopg2-binary

Traceback (most recent call last):
  File "/usr/lib64/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "./nina_database_manager.pyz/__main__.py", line 2, in <module>
  File "/usr/local/lib/python3.8/site-packages/manager.py", line 9, in <module>
    import psycopg2
  File "<frozen zipimport>", line 259, in load_module
  File "./nina_database_manager.pyz/psycopg2/__init__.py", line 51, in <module>
ModuleNotFoundError: No module named 'psycopg2._psycopg'

Describe the bug
psycopg2-binary is a dependency of the project, but it is not found when I try to execute the .pyz zip file.

To Reproduce
Steps to reproduce the behavior:

1. Add psycopg2-binary
2. Import psycopg2 in your app
3. Pack your app
4. Run it
5. See error

Expected behavior
psycopg2 should be imported as usual.

System (please complete the following information):

• pdm-packer version: 0.2.1
• Python version: 3.8
• OS: Linux

Additional context
Other dependencies, such as ldap, seems to work as usual.

The project files were not packaged in when packaging

This will cause python app.pyz to not run correctly when I only give the pyz package.

I checked the source code and this should happen here: https://github.com/frostming/pdm-packer/blob/main/src/pdm_packer/command.py#L100

Add option to include/exclude pyc files

Write bytes into the file stream instead of memory

How do you pack all files within project and access them?

Probably just some examples I'm missing but how do you pack and access resource files? For example my project consists of:

lib/python/.... (python modules)
run.py (run function as entrypoint)
.env
README.md
etc.

How do I access .env file for example? because now my app is failing due to dotenv.load_dotenv function

Traceback (most recent call last):
  File "/home/mariusg/.pyenv/versions/3.9.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/mariusg/.pyenv/versions/3.9.9/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/mariusg/Documents/Projects/infra-terraform/./app.pyz/__main__.py", line 3, in <module>
  File "/tmp/pdm-pack-rpnrz65t/lib/run.py", line 10, in run
    load_dotenv()
  File "/tmp/pdm-pack-rpnrz65t/lib/dotenv/main.py", line 331, in load_dotenv
  File "/tmp/pdm-pack-rpnrz65t/lib/dotenv/main.py", line 298, in find_dotenv
  File "/tmp/pdm-pack-rpnrz65t/lib/dotenv/main.py", line 255, in _walk_to_root
OSError: Starting path not found

run.py:

#!/usr/bin/env python3

from dotenv import load_dotenv

def run():
    # Load up .env files
    load_dotenv()

Package command:

pdm pack -m run:run -c --pyc --no-py --exe -o app.pyz

packing pdm itself apears only to work with python 3.11, but neither 3.9 nor 3.10 on win10

Hi,

I was succesfully able to pack pdm into a zipapp. This works nicely when I use python 3.11 to execute the zipapp.

I encountered 2 problems with the exe target.
1.) I needed to use the -i command to specify the interpreter "py" which is to my understanding the recommended way to find python on windows.
2.) on the target machine python 3.11 needs to be the default selected by "py".

The issue appears to only affect sync, not lock or search.

I would like to know if there is a possibility to specify the python version which should've used, like "py -3.11" when building an exe

This is the way I create the zipapp.:

git clone https://github.com/pdm-project/pdm.git cd pdm git checkout 2.3.4 pdm add pdm-packer pdm pack pdm pack --exe -i py.exe

Any hints?

Thanks in advance,
Franz

crosspack zipapps

It would be awesome if it would be possible to run

pdm pack --target=cp312-win_amd64 -o win.pyz
pdm pack --target=cp312-manylinux_x86_64 -o linux.pyz

to pack for multiple targets on the same machine...

what do you think?

pdm_packer:plugin: cannot import name 'cached_property' from 'pdm.utils' .

After installing packer, i get the following error message.:

Failed to load plugin pdm-packer=pdm_packer:plugin: cannot import name 'cached_property' from 'pdm.utils' .

pdm version is 2.3.2
pdm packer version is 0.4.0

any thoughts?

Latest release is incompatible with latest pdm release

Describe the bug
The latest released version of pdm-packer is not compatible with the latest version of pdm (currently 1.14.1).
This is indicated by an error message when calling pdm as well as the pdm pack not working.
Thankfully, the underlying problem is already fixed on main :-)

To Reproduce

1. Run pdm plugin add pdm-packer and see the installation succeed
2. Run pdm plugin list and see this error:

   Failed to load plugin pdm-packer=pdm_packer:plugin: cannot import name 'get_architecture' from 'pdm.models.in_process' (/some/venv/pdm/lib/python3.10/site-packages/pdm/models/in_process/__init__.py)
   

3. Install pdm-packer from the latest main branch in Git instead, e.g. with pipx: pipx inject pdm "git+https://github.com/frostming/pdm-packer.git@main"
4. Run pdm plugin list: No error shows up

Expected behavior
The latest version of pdm-packer in PyPI should work with the latest version of pdm.

System

• pdm-packer version: 0.3
• pdm version: 1.14.1
• Python version: 3.8, 3.10
• OS: Ubuntu 20.04, macOS 12.3

when packing pdm the version given at the stdout is not the same as the one given when doing --version

I can see the version of pdm used listed on stdout when creating the zipapp, but when running the zip app with --version, I get a different version.

To Reproduce
git checkout 2.3.3
pdm add pdm-packer
pdm pack
---------8<---------8<---------8<---------8<---------8<---------
...
[2022-12-15T20:38:56.771Z] v Install pdm 2.3.3+d20221215 successful
[2022-12-15T20:38:56.771Z]
[2022-12-15T20:38:56.771Z] All complete!
[2022-12-15T20:38:56.771Z]
[2022-12-15T20:38:56.771Z] Packages are prepared at xx
[2022-12-15T20:38:56.771Z] Creating zipapp...
[2022-12-15T20:38:57.026Z] Zipapp is generated at pdm.pyz
[2022-12-15T20:38:57.586Z] pdm.pyz --version
[2022-12-15T20:38:58.947Z] PDM, version 0.0.0+local
---------8<---------8<---------8<---------8<---------8<---------

I would have expected the version to be 2.3.3+d20221215, or 2.3.3 but not 0.0.0+local

Is this expected? What would I need to do to get the version to be 2.3.3+packedbymyorgnamehere?

Thanks in advance,
Franz

[QUESTION]. how to use it?

I created a test project

pdm init 

add run.py

def run():
    print('hello')

pack

pdm pack -m run:run -o app.zip -v
Packing packages...
All packages are synced to date, nothing to do.

  All complete!
Packages are prepared at U:\Temp\pdm-pack-a64tsc6l\lib
Creating zipapp...
Traceback (most recent call last):
  File "C:\Python\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Python\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\users\user\.local\bin\pdm.exe\__main__.py", line 7, in <module>
  File "C:\Users\user\.local\pipx\venvs\pdm\lib\site-packages\pdm\core.py", line 254, in main
    return Core().main(args)
  File "C:\Users\user\.local\pipx\venvs\pdm\lib\site-packages\pdm\core.py", line 187, in main
    raise cast(Exception, err).with_traceback(traceback)
  File "C:\Users\user\.local\pipx\venvs\pdm\lib\site-packages\pdm\core.py", line 182, in main
    f(options.project, options)
  File "C:\Users\user\.local\pipx\venvs\pdm\lib\site-packages\pdm_packer\command.py", line 120, in handle
    zipapp.create_archive(
  File "C:\Python\lib\zipapp.py", line 111, in create_archive
    raise ZipAppError("Source does not exist")
zipapp.ZipAppError: Source does not exist

Cannot pack if there are no packages? - pdm add colorama
(why not use without packages)
and try again

pdm pack -m run:run -o app.zip -v
Packing packages...
Synchronizing working set with lock file: 1 to add, 0 to update, 0 to remove

unearth: Downloading <Link https://files.pythonhosted.org/packages/77/8b/7550e87b2d308a1b711725dfaddc19c695f8c5fa413c640b2be01662f4e6/colorama-0.4.5-py2.py3-none-any.whl (from None)> to U:\Temp\pdm-build-42ueqwgg\colorama-0.4.5-py2.py3-none-any.whl
  v Install colorama 0.4.5 successful

  All complete!
Packages are prepared at U:\Temp\pdm-pack-1trs0fvn\lib
Creating zipapp...
Zipapp is generated at app.zip

app.zip structure

        colorama
        __main__.py

no run.py file

pdm pack --help
Usage: pdm pack [-h] [-v] [-g] [-p PROJECT_PATH] [-m MAIN] [-o OUTPUT] [-c] [--pyc] [--no-py] [-i INTERPRETER] [--exe]

Pack the packages into a zipapp

only packages without project files? then why need -m run:run?

pdm info && pdm info --env

pdm info && pdm info --env
PDM version:
  2.0.0
Python Interpreter:
  C:\Python\python.exe (3.10)
Project Root:
  z:/testapp
Project Packages:
  z:\testapp\__pypackages__\3.10
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.5",
  "os_name": "nt",
  "platform_machine": "AMD64",
  "platform_release": "10",
  "platform_system": "Windows",
  "platform_version": "10.0.19044",
  "python_full_version": "3.10.5",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "win32"
}

Executable throws exception on numpy dependency

Describe the bug

I'm packaging up a python application that has numpy as a dependency. I package the application with pdm pack --exe -g -p ./ - when I run ./myexecutable myFunction "arg1" I get the following error

ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.9 from "/Users/tpeterson/Code/ai_software/luna/.venv/bin/python"
  * The NumPy version is: "1.24.2"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: No module named 'numpy.core._multiarray_umath'

I've tried python 3.8-3.11 and have reinstalled numpy several times. numpy is imported inside of "myFunction" - other python functions I call that don't have numpy as an import succeed.

To Reproduce
Steps to reproduce the behavior:

1. Go to '...'
2. Run command '...'
3. Scroll down to '...'
4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

System (please complete the following information):

• pdm-packer version: latest
• Python version: 3.8
• OS: MacOS with M2 Max

Additional context
Add any other context about the problem here.