mwilliamson/stickytape

Convert Python packages into a single script

stickytape: Convert Python packages into a single script

Stickytape can be used to convert a Python script and any Python modules it depends on into a single-file Python script. There are likely better alternatives depending on what you're trying to do. For instance:

• If you want to create a single file that can be executed by a Python interpreter, use zipapp.
• If you need to create a standalone executable from your Python script, I recommend using an alternative such as PyInstaller.

Since Stickytape relies on correctly analysing both your script and any dependent modules, it may not work correctly in all circumstances. I bodged together the code a long time ago for a specific use case I had, so many normal uses of Python imports are not properly supported.

Installation

pip install stickytape

Usage

You can tell stickytape which directories to search using the --add-python-path argument. For instance:

stickytape scripts/blah --add-python-path . > /tmp/blah-standalone

Or to output directly to a file:

stickytape scripts/blah --add-python-path . --output-file /tmp/blah-standalone

You can also point stickytape towards a Python binary that it should use sys.path from, for instance the Python binary inside a virtualenv:

stickytape scripts/blah --python-binary _virtualenv/bin/python --output-file /tmp/blah-standalone

Stickytape cannot automatically detect dynamic imports, but you can use --add-python-module to explicitly include modules:

stickytape scripts/blah --add-python-module blah.util

By default, stickytape will ignore the shebang in the script and use "#!/usr/bin/env python" in the output file. To copy the shebang from the original script, use --copy-shebang:

stickytape scripts/blah --copy-shebang --output-file /tmp/blah-standalone

As you might expect with a program that munges source files, there are a few caveats:

• Due to the way that stickytape generates the output file, your script source file should be encoded using UTF-8. If your script doesn't declare its encoding in its first two lines, then it will be UTF-8 by default as of Python 3.
• Your script shouldn't have any from __future__ imports.
• Anything that relies on the specific location of files will probably no longer work. In other words, __file__ probably isn't all that useful.
• Any files that aren't imported won't be included. Static data that might be part of your project, such as other text files or images, won't be included.

stickytape's Issues

how to coverage

Holding of the __init__.py during 'make_package' cause failure on exit (Windows + IronPython)

The creation of the 'init.py' not close the file in time. This cause failure on deleting the temp folder on exit:
prelude.py#L27

Traceback (most recent call last):
  File "D:/project/example.py", line 65, in <module>
  File "C:\Program Files\IronPython 2.7\Lib\contextlib.py", line 35, in __exit__
  File "D:/project/example.py", line 14, in __stickytape_temporary_dir
  File "C:\Program Files\IronPython 2.7\Lib\shutil.py", line 261, in rmtree
  File "C:\Program Files\IronPython 2.7\Lib\shutil.py", line 266, in rmtree
  File "C:\Program Files\IronPython 2.7\Lib\shutil.py", line 240, in onerror
  File "C:\Program Files\IronPython 2.7\Lib\shutil.py", line 264, in rmtree
WindowsError: [Errno 13] The process cannot access the file because it is being used by another process

The following changes (prelude.py) work for me:

                ...
                if not os.path.exists(partial_path):
                    os.mkdir(partial_path)
                    # open(os.path.join(partial_path, "__init__.py"), "w").write("\n")
                    with open(os.path.join(partial_path, "__init__.py"), "w") as f:
                        f.write("\n")

Relative imports matching stdlib modules are incorrectly ignored

Relative imports (e.g. from .xml import a) are incorrectly considered to be stdlib modules, and ignored. We should check whether something is in the stdlib using the absolute module path.

Relative import still doesn't work?

Reproduction

mkdir -p foo/src
cd foo
touch src/__init__.py

Create the following files:
src/server.py:

from .foo import getab

src/foo.py:

def getab():
    return ''

Run:

stickytape src/server.py --add-python-path . --output-file tmp.py
Traceback (most recent call last):
  File "/Users/nelson.yeung/Documents/python-playground/.venv/bin/stickytape", line 8, in <module>
    sys.exit(main())
  File "/Users/nelson.yeung/Documents/python-playground/.venv/lib/python3.9/site-packages/stickytape/main.py", line 9, in main
    output = stickytape.script(
  File "/Users/nelson.yeung/Documents/python-playground/.venv/lib/python3.9/site-packages/stickytape/__init__.py", line 27, in script
    output.append(_generate_module_writers(
  File "/Users/nelson.yeung/Documents/python-playground/.venv/lib/python3.9/site-packages/stickytape/__init__.py", line 69, in _generate_module_writers
    generator.generate_for_file(path, add_python_modules=add_python_modules)
  File "/Users/nelson.yeung/Documents/python-playground/.venv/lib/python3.9/site-packages/stickytape/__init__.py", line 87, in generate_for_file
    self._generate_for_module(ImportTarget(python_file_path, relative_path=None, is_package=False, module_name=None))
  File "/Users/nelson.yeung/Documents/python-playground/.venv/lib/python3.9/site-packages/stickytape/__init__.py", line 95, in _generate_for_module
    for import_line in import_lines:
  File "/Users/nelson.yeung/Documents/python-playground/.venv/lib/python3.9/site-packages/stickytape/__init__.py", line 173, in _find_imports_in_module
    package_name = ".".join(python_module.module_name.split(".")[:-level])
AttributeError: 'NoneType' object has no attribute 'split'

Info

python --version
Python 3.9.5

OS: macOS 11.4
stickytape: 0.2.1

Related: #22 #24

Questions Regarding Useage

Had some questions regarding stickytape; was directed by developer to upload as issues here.

There does not seem to be documentation regarding which platforms/operating systems stickytape works for. Slashes in examples appear to indicate UNIX-based systems, but I'm unsure about use on Windows, or which UNIX-based system(s). Also unsure about how to "stickytape" my python code based on its version (currently working with variations of both versions 2 and 3, and need resulting code for both).

Documentation also is unfamiliar to me regarding command line examples, e.g., the bit " . > " in the command line example:
stickytape scripts/blah --add-python-path . > /tmp/blah-standalone

The most successful attempt so far has produced a Python 3 script that contains mentions of all modules needed, but does not appear to include modules in the resultant script in any way.
I am happy to load my own code and attempt to upload exact errors as needed.

Thanks :)

How to "make"

There is no "stickytape" binary in the GitHub package. I see the "makefile" but don't know how to "make" it. Simply typing "make" generated errors.

when i exec a cmpiled file in a cmpiled file ,raise NameError: global name '__stickytape_working_dir' is not defined

such as
i use stickytape A project to a sigle file named A, and stickytaped B project into a sigle file named B,

when i exec(B) in A raise NameError: global name '__stickytape_working_dir' is not defined

could you tell me hao to deal with it?

Renaming of modules

I'd like to be able to rename the included modules. Since they are found via imports, seems like it ought to be possible to rewrite the imports and the paths.

Motivation would be to avoid name conflicts.

Is this reasonable?

Move future statements at the top

Description:
Trying to stickytape a file containing a future statement results in an error:

SyntaxError: from __future__ imports must occur at the beginning of the file

Steps to reproduce:

1. Write file a.py

from __future__ import annotations

def f(x: str | int) -> str | int:
    return 2 * x

2. Run stickytape a.py > b.py
3. Run python b.py

Expected result:
Nothing

Actual result:

  File "b.py", line 37
    from __future__ import annotations
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: from __future__ imports must occur at the beginning of the file

Possible fix:
Detect such statements, for instance with a regex pattern, and move them at the beginning of the file.

re.match('from __future__ import (.*)$', line)

Dynamic imports don't work if the python path is a level up from their location

Given a directory structure like this:

project_folder/
└── my_module/
    ├── some_submodules/
    │   ├── module_a.py
    │   └── module_b.py
    └── main.py

If I use absolute imports in my code (e.g. import my_module.some_submodules.module_a), then my $PYTHONPATH must be project_folder, so I execute stickytape in this folder and --add-python-path .

However, if instead of importing the submodules directly in main.py, and instead import using importlib.import_module...

from importlib import import_module
import_module('my_module.some_submodules.module_a.py')

...when I run the stickytape script, I get a traceback ending in ModuleNotFoundError: No module named 'my_module'.

Obviously this is a simplified example but it demonstrates the behaviour I'm getting in some software where modules are dynamically imported with importlib in this way. Perhaps it could be possible to add these submodules to stickytape somehow, like how you can specify hidden modules in PyInstaller?

How to include Python standard library

Let say I'w like to flatify F:/a.py

import requests
import re
come codes here

and the Python+it libs are in C:/anaconda3

Should I use: F:/> stickytape a.py --add-python-path C:/anaconda3 --output-file F:/a-flatified.py or how?

I've tried different arguments but a.py still contains 'import requests' and 'import re'.

Output file still want to import files

Please notice that I do not normally use python! So I do not know the import system at all.

I tried to make a single python file for a youtube-dl case:

`stickytape %mypar%/youtube_dl/mytest.py --add-python-path . --output-file st-temp.py`

The output file works in the same directory where it is written. But it still imports other files (which I to my surprise noticed when I tried to move it).

The file mytest.py looks like this (this file is mostly just copied from main.py):

from future import unicode_literals

import sys

import os.path
path = os.path.realpath(os.path.abspath(file))
sys.path.insert(0, os.path.dirname(os.path.dirname(path)))

import youtube_dl

if name == 'main':
sys.argv = [sys.argv[0], "-g", "https://www.youtube.com/watch?v=N5xyr1_DhpA" ]
youtube_dl.main()

It is import youtube-dl that is the still there. (And that is of course just a start of the imports I want to avoid.)

(There is another problem too. The from __future__ import unicode_literals does not work. Moving it to the top of the output file fixes this problem.)

Using sticky tape to pass code to external engine

I want to be able to compile a python module down to a string, that can be passed along with arguments to an external engine which then loads the module and runs the run function with the arguments.

So my basic test is if the following is posible:

import stickytape
def test(*args, **kwargs):raise ImportError("No test function loaded")
source_code = stickytape.script("stickytape_test/__init__.py")
exec(source_code)
assert test("test") == "test"

# stickytape_test/__init__.py
from .core import *

# stickytape_test/core.py
def test(out):
     return out

I get a
NameError: name '__stickytape_working_dir' is not defined

Any suggestions on how to do something like this?

Explicitly add modules inline

Is there an adjustment that can be made to remove local imports from the file.
I want the imported local python codes embedded in the code without import statements.
I'm trying to run my code in firebase functions and it won't accept local imports rn. That's why I'm trying to merge my codebase into one main.py file with no local imports.

Wrong directory separator on windows

__stickytape_write_module('''path\\to\\Module.py''',....

That's the code being generated on Windows. It fails both on windows and linux. If i manually change them to / it works everywhere.

Could you fix that please?

from __future__ import unicode_literals

I get this error as a part of the problems in #20. I think it is better to have this as a separate issue.

from future import unicode_literals
^
SyntaxError: from future imports must occur at the beginning of the file

PermissionError: [WinError 5] Access is denied Error in Windows environment

Hi, getting stuck with this problem with the following details:
Kindly guide.

Thanks in advance

Command:
stickytape C:\Users\xyz\Documents\xyz\PyProjects\myproj\mypscript.py --add-python-path C:\Users\xyz\anaconda3\ --python-binary C:\Users\xyz\anaconda3\Lib\site-packages --output-file C:\Users\xyz\Documents\xyz\PyProjects\myproj\mypscriptOne.py

Additional information.

• command invoked from directory .. C:\Users\xyz\Documents\xyz\PyProjects\myproj\
• windows command shell in administrator mode
• Environment - Windows 10, anaconda 3, python 3.7

Output:
Traceback (most recent call last):
File "c:\users\panicker\anaconda3\lib\runpy.py", line 193, in run_module_as_main
"main", mod_spec)
File "c:\users\panicker\anaconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\panicker\anaconda3\Scripts\stickytape.exe_main.py", line 7, in
File "c:\users\panicker\anaconda3\lib\site-packages\stickytape\main.py", line 10, in main
args.script, args.add_python_path, args.python_binary)
File "c:\users\panicker\anaconda3\lib\site-packages\stickytape_init.py", line 9, in script
python_paths = [os.path.dirname(path)] + add_python_paths + read_sys_path_from_python_bin(python_binary)
File "c:\users\panicker\anaconda3\lib\site-packages\stickytape_init.py", line 24, in _read_sys_path_from_python_bin
env={}
File "c:\users\panicker\anaconda3\lib\subprocess.py", line 411, in check_output
**kwargs).stdout
File "c:\users\panicker\anaconda3\lib\subprocess.py", line 488, in run
with Popen(*popenargs, **kwargs) as process:
File "c:\users\panicker\anaconda3\lib\subprocess.py", line 800, in init
restore_signals, start_new_session)
File "c:\users\panicker\anaconda3\lib\subprocess.py", line 1207, in _execute_child
startupinfo)
PermissionError: [WinError 5] Access is denied

Binary execution issues due to shebang

Hey there, it's me again !

Due to 6476a77, there could be an issue when executing directly a Stickytape-d resulting output.
A Python 3 compatible-only script may be executed against a Python 2 interpreter (on distributions having Python 2 as the default interpreter).

Maybe the code should contain one more condition to correctly figure out whether it should used #!/usr/bin/env python or #!/usr/bin/env python3 ?

I think reverting the commit referenced above should work, but I guess you had a reason to implement it.

Bye 👋

Library to script conversion possible?

I just tried to use stickytape upon better-exceptions library, but I couldn't figure out what I should use as the "scripts/blah" parameter. Everything I tried resulted in errors.

Can I get some help? We could even enhance the readme after that.

Thanks!

how to convert many py files to a py file without make tempfile

Improve list of stdlib modules

The list of modules in the stdlib is currently hard-coded. This varies across Python versions: it'd be nice to find an automatic way of detecting stdlib modules.

Handle relative imports from parent packages

We ignore the level of import froms, meaning that any relative import from a parent package isn't correctly detected.

SyntaxError: Non-UTF-8 code starting with '\xff'

I am following the doc to get started.
Code:

pip install stickytape
stickytape .\myscript.py --add-python-path . > singlefilescript.py
python \.singlefilescript.py

Error:

  File ".\singlefilescript.py", line 1
SyntaxError: Non-UTF-8 code starting with '\xff' in file .\single.py on line 1, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details

my python version is 3.7.2

UnicodeEncodeError: 'charmap' codec can't encode characters in position 83853-83854

Using stickytape on my example in issue #20 now works excellent in the Ubuntu subsystem (20-21) on Windows 10.

However if I try to use the python file produced there directly in Window 10 (cmd shell) I get the error in the subject line.