pyo3/setuptools-rust

Setuptools plugin for Rust support

Setuptools plugin for Rust extensions

setuptools-rust is a plugin for setuptools to build Rust Python extensions implemented with PyO3 or rust-cpython.

Compile and distribute Python extensions written in Rust as easily as if they were written in C.

Quickstart

The following is a very basic tutorial that shows how to use setuptools-rust in pyproject.toml. It assumes that you already have a bunch of Python and Rust files that you want to distribute. You can see examples for these files in the examples/hello-world directory in the github repository. The PyO3 docs have detailed information on how to write Python modules in Rust.

hello-world
├── python
│   └── hello_world
│       └── __init__.py
└── rust
    └── lib.rs

Once the implementation files are in place, we need to add a pyproject.toml file that tells anyone that wants to use your project how to build it. In this file, we use an array of tables (TOML jargon equivalent to Python's list of dicts) for [[tool.setuptools-rust.ext-modules]], to specify different extension modules written in Rust:

# pyproject.toml
[build-system]
requires = ["setuptools", "setuptools-rust"]
build-backend = "setuptools.build_meta"

[project]
name = "hello-world"
version = "1.0"

[tool.setuptools.packages]
# Pure Python packages/modules
find = { where = ["python"] }

[[tool.setuptools-rust.ext-modules]]
# Private Rust extension module to be nested into the Python package
target = "hello_world._lib"  # The last part of the name (e.g. "_lib") has to match lib.name in Cargo.toml,
                             # but you can add a prefix to nest it inside of a Python package.
path = "Cargo.toml"      # Default value, can be omitted
binding = "PyO3"         # Default value, can be omitted

Each extension module should map directly into the corresponding [lib] table on the Cargo manifest file:

# Cargo.toml
[package]
name = "hello-world"
version = "0.1.0"
edition = "2021"

[dependencies]
pyo3 = "0.20.3"

[lib]
name = "_lib"  # private module to be nested into Python package,
               # needs to match the name of the function with the `[#pymodule]` attribute
path = "rust/lib.rs"
crate-type = ["cdylib"]  # required for shared library for Python to import from.

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
# See also PyO3 docs on writing Cargo.toml files at https://pyo3.rs

You will also need to tell Setuptools that the Rust files are required to build your project from the source distribution. That can be done either via MANIFEST.in (see example below) or via a plugin like setuptools-scm.

# MANIFEST.in
include Cargo.toml
recursive-include rust *.rs

With these files in place, you can install the project in a virtual environment for testing and making sure everything is working correctly:

# cd hello-world
python3 -m venv .venv
source .venv/bin/activate  # on Linux or macOS
.venv\Scripts\activate     # on Windows
python -m pip install -e .
python
>>> import hello_world
# ... try running something from your new extension module ...
# ... better write some tests with pytest ...

Next steps and final remarks

• When you are ready to distribute your project, have a look on the notes in the documentation about building wheels.

• Cross-compiling is also supported, using one of crossenv, cross or cargo-zigbuild. For examples see the test-crossenv and test-cross and test-zigbuild Github actions jobs in ci.yml.

• You can also use [[tool.setuptools-rust.bins]] (instead of [[tool.setuptools-rust.ext-modules]]), if you want to distribute a binary executable written in Rust (instead of a library that can be imported by the Python runtime). Note however that distributing both library and executable (or multiple executables), may significantly increase the size of the wheel file distributed by the package index and therefore increase build, download and installation times. Another approach is to use a Python entry-point that calls the Rust implementation (exposed via PyO3 bindings). See the hello-world example for more insights.

• For a complete reference of the configuration options, see the API reference. You can use any parameter defined by the RustExtension class with [[tool.setuptools-rust.ext-modules]] and any parameter defined by the RustBin class with [[tool.setuptools-rust.bins]]; just remember to replace underscore characters _ with dashes - in your pyproject.toml file.

• Cargo.toml allow only one [lib] table per file. If you require multiple extension modules you will need to write multiple Cargo.toml files. Alternatively you can create a single private Rust top-level module that exposes multiple submodules (using PyO3's submodules), which may also reduce the size of the build artifacts. You can always keep your extension modules private and wrap them in pure Python to have fine control over the public API.

• If want to include both [[tool.setuptools-rust.bins]] and [[tool.setuptools-rust.ext-modules]] in the same macOS wheel, you might have to manually add an extra build.rs file, see PyO3/setuptools-rust#351 for more information about the workaround.

• For more examples, see:

  • hello-world: a more complete version of the code used in this tutorial that mixes both [[tool.setuptools-rust.ext-modules]] and [[tool.setuptools-rust.bins]] in a single distribution.
  • html-py-ever: a more advanced example that uses Rust crates as dependencies.
  • rust_with_cffi: uses both Rust and CFFI.
  • namespace_package: integrates Rust-written modules into PEP 420 namespace packages.
  • hello-world-script: uses Rust only for creating binary executables, not library modules.

setuptools-rust's Issues

add "test" command

"python setup.py test" should run "cargo test" as well

Unable to find generated shared library in target/debug

On my machine, cargo creates two folders under target/: debug and x86_64-unknown-linux-gnu/debug. The final result of the compilation (the dynamic library in this case) is only generated in the latter (target-specific) directory. setuptools-rustonly looks for the artifact in the target/debug directory and complains:

Finished dev [unoptimized + debuginfo] target(s) in 10.56s
cargo rustc --lib --manifest-path Cargo.toml --features pyo3/python3 pyo3/extension-module -- --crate-type cdylib
error: rust build failed; unable to find any *fails.so in /home/victor/Coding/python/ground/fails/target/debug

➜ ls target/**/*.so
target/debug/deps/libpyo3cls-dc3865a11bebf80d.so 
target/x86_64-unknown-linux-gnu/debug/deps/libfails.so
target/x86_64-unknown-linux-gnu/debug/libfails.so

AFAIK, rustc is inherently a cross-compiler so it can generate code for several targets, such as x86_64-unknown-linux-gnu, x86_64-unknown-linux-musl, x86_64-unknown-unknown-wasm. It makes sense that libfails.so compiled with the GNU C standard library to live in a different directory from the one compiled with musl, but how should setuptools-rust know to choose the right one. Which library should be the right one in this case?

Thanks!

`setup.py develop` blocks later usage of optimized builds

I'm not sure about the parallels between setuptools and setuptools-rust in this case (pretty hazy on a lot of the details tbh), but here's a problem we came across. It may just need documenting as it's a bit of a gotcha.

When you setup.py install, the egg gets built with a release/optimized binary and then copied to your site-packages. When you setup.py develop, the egg gets built with a debug binary, which is copied into your python source tree, and then that is symlinked ?somewhere?. The important thing is that even if you setup.py install after having setup.py developed, the old debug binary is found via the symlink to the source tree before the new optimized binary ins site-packages. We were wondering why our benchmarks were around 50x slower than expected (credit to @aschampion for figuring it out).

This possibly impacts the use of usedevelop in the tox config (#3) - while it might be nice to run unit tests against a debug build, if you're using something like pytest-benchmark you want pytest to be running against the optimized build.

Ignore strip option when platform is win32

Does this support Windows?

Does this support the latest rust on Windows as well? Also will this check for rust updates before compile to be sure the current build tool chain is installed/up to date?

Windows builds broken with 0.11.x

Hi!

Something about the recent updates to setuptools-rust has broken the windows builds for CPython 3.6, 3.7, and 3.8 for my json-logic-rs project. You can see a failing build here. That branch contains commits using 0.10.6, 0.11.0, 0.11.1, and 0.11.2. You can check out all the builds here. The failure only is reproducible with a fresh cache.

The failure occurs when running bdist_wheel. It appears to compile most of the dependencies with no problem, but when going to compile my crate, I get the following error (here is a direct link to the line in the CI output):

Compiling jsonlogic-rs v0.2.0 (D:\a\json-logic-rs\json-logic-rs)
     Running `rustc --crate-name jsonlogic_rs --edition=2018 'src\lib.rs' --error-format=json --json=diagnostic-rendered-ansi --crate-type cdylib --crate-type lib --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --crate-type cdylib --cfg 'feature="cpython"' --cfg 'feature="default"' --cfg 'feature="python"' -C metadata=355d8e99c9c8f6f3 --out-dir 'D:\a\json-logic-rs\json-logic-rs\target\release\deps' -L 'dependency=D:\a\json-logic-rs\json-logic-rs\target\release\deps' --extern 'cpython=D:\a\json-logic-rs\json-logic-rs\target\release\deps\libcpython-52b22c086a3919cb.rlib' --extern 'phf=D:\a\json-logic-rs\json-logic-rs\target\release\deps\libphf-bd8c006b98c564f5.rlib' --extern 'serde_json=D:\a\json-logic-rs\json-logic-rs\target\release\deps\libserde_json-a0af68d18c919709.rlib' --extern 'thiserror=D:\a\json-logic-rs\json-logic-rs\target\release\deps\libthiserror-ff9a854b2d25ae87.rlib' -L 'native=D:\a\json-logic-rs\json-logic-rs\venv\libs'`
error: linking with `link.exe` failed: exit code: 1181
  |
  = note: "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Enterprise\\VC\\Tools\\MSVC\\14.26.28801\\bin\\HostX64\\x64\\link.exe" "/NOLOGO" "/NXCOMPAT" "/LIBPATH:C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.0.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.1.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.10.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.11.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.12.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.13.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.14.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.15.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.2.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.3.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.4.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.5.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.6.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.7.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.8.rcgu.o" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.jsonlogic_rs.289a1ozj-cgu.9.rcgu.o" "/OUT:D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.dll" "/DEF:C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\rustcyGSbro\\lib.def" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.yd4qbihkqgjzzdf.rcgu.o" "/OPT:REF,ICF" "/DLL" "/IMPLIB:D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\jsonlogic_rs.dll.lib" "/DEBUG" "/NATVIS:C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\intrinsic.natvis" "/NATVIS:C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\liballoc.natvis" "/NATVIS:C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libcore.natvis" "/NATVIS:C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\etc\\libstd.natvis" "/LIBPATH:D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps" "/LIBPATH:D:\\a\\json-logic-rs\\json-logic-rs\\venv\\libs" "/LIBPATH:C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libcpython-52b22c086a3919cb.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libpaste-9d204b7814376191.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libnum_traits-cb0dce9c11a39169.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libpython3_sys-a40b245c5fda99ee.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\liblibc-a83758c8effee76a.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libphf-bd8c006b98c564f5.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libphf_shared-3b3c3d07abc41e18.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libsiphasher-815fdd8da4492d35.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libthiserror-ff9a854b2d25ae87.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libserde_json-a0af68d18c919709.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libryu-daa005afbd1aea40.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libitoa-b7d6d7b01ab1c8ad.rlib" "D:\\a\\json-logic-rs\\json-logic-rs\\target\\release\\deps\\libserde-2465fb80283a4ccf.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libstd-c98dfc5362dcc6b7.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libpanic_unwind-aec539acbc1e14cf.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libhashbrown-cc228c1040b043cc.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_alloc-372b0d9c1f16645d.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libbacktrace-06d3255123b024fd.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_demangle-2a7699cacc651ebb.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libunwind-dba2f7e740fad11f.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcfg_if-4a7eb8f8ce110bcb.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liblibc-26e0b7bec9208373.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\liballoc-1a4f31a2385aa353.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\librustc_std_workspace_core-bf1779ce10dfac6f.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcore-b622bd5c9beb555f.rlib" "C:\\Rust\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib\\libcompiler_builtins-ad9f0d3895dbdfda.rlib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "python38.lib" "advapi32.lib" "ws2_32.lib" "userenv.lib" "msvcrt.lib"
   = note: LINK : fatal error LNK1181: cannot open input file 'python38.lib'
          

error: aborting due to previous error

error: could not compile `jsonlogic-rs`.

Caused by:
  process didn't exit successfully: `rustc --crate-name jsonlogic_rs --edition=2018 'src\lib.rs' --error-format=json --json=diagnostic-rendered-ansi --crate-type cdylib --crate-type lib --emit=dep-info,link -C opt-level=3 -Cembed-bitcode=no --crate-type cdylib --cfg 'feature="cpython"' --cfg 'feature="default"' --cfg 'feature="python"' -C metadata=355d8e99c9c8f6f3 --out-dir 'D:\a\json-logic-rs\json-logic-rs\target\release\deps' -L 'dependency=D:\a\json-logic-rs\json-logic-rs\target\release\deps' --extern 'cpython=D:\a\json-logic-rs\json-logic-rs\target\release\deps\libcpython-52b22c086a3919cb.rlib' --extern 'phf=D:\a\json-logic-rs\json-logic-rs\target\release\deps\libphf-bd8c006b98c564f5.rlib' --extern 'serde_json=D:\a\json-logic-rs\json-logic-rs\target\release\deps\libserde_json-a0af68d18c919709.rlib' --extern 'thiserror=D:\a\json-logic-rs\json-logic-rs\target\release\deps\libthiserror-ff9a854b2d25ae87.rlib' -L 'native=D:\a\json-logic-rs\json-logic-rs\venv\libs'` (exit code: 1)
cargo rustc --lib --manifest-path Cargo.toml --features python cpython/python3-sys cpython/extension-module --release --verbose -- --crate-type cdylib
error: cargo failed with code: 101

Here is my setup.py for one of the failing commits.

The setup.py command is run via a virtualenv, which is created via the Makefile.

I'm using rust-cpython. Here is the cargo file.

The same build is working without issues on Ubuntu and MacOS on the newest version of setuptools-rust. I am very much a windows novice, so any help at all would be appreciated. I'm happy to provide any other information that might be useful.

In the meantime, pinning to 0.10.6 is no problem!

Thanks for your work on this project. It's been a great help!

Conflict when running bdist_wheel

With the new release 0.11.0 there is an issue when trying to create a package by running:
python setup.py bdist_wheel
we got this error:

  File "setup.py", line 33, in <module>
    'hdbscan', 'oni-analysis~=3.1'],
  File "/home/********/repo/venv/lib/python3.6/site-packages/setuptools/__init__.py", line 140, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/********/repo/venv/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 259, in run
    self.run_command('install')
  File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.6/distutils/command/install.py", line 557, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/home/********/repo/venv/lib/python3.6/site-packages/setuptools/command/install_lib.py", line 12, in run
    outfiles = self.install()
  File "/usr/local/lib/python3.6/distutils/command/install_lib.py", line 111, in install
    outfiles = self.copy_tree(self.build_dir, self.install_dir)
  File "/home/********/repo/venv/lib/python3.6/site-packages/setuptools/command/install_lib.py", line 91, in copy_tree
    exclude = self.get_exclusions()
  File "/home/********/repo/venv/lib/python3.6/site-packages/setuptools/command/install_lib.py", line 24, in get_exclusions
    for ns_pkg in self._get_SVEM_NSPs()
  File "/home/********/repo/venv/lib/python3.6/site-packages/setuptools/command/install_lib.py", line 61, in _get_SVEM_NSPs
    svem = install_cmd.single_version_externally_managed
  File "/usr/local/lib/python3.6/distutils/cmd.py", line 103, in __getattr__
    raise AttributeError(attr)
AttributeError: single_version_externally_managed

Rolling back to 0.10.6 solved the issue.

Investigate options for automatic installation of setuptools_rust

Currently, anything built using this requires a manual pip install setuptools-rust before its setup.py can be run.

I've never encountered any other setup.py-based solution where pip install whatever may fail for lack of a dependency that can be pip installed, but requires a separate command. An attempt should be made to find a workaround for that.

Tox, PyTest, sdist...requires usedevelop for idiomatic tox tests to work

I discovered this by trial and error, so perhaps I'm off the mark here. People more familiar with the PyTest/Tox/Rust trifecta (??) may know a better way.

Basically, I've been trying to port a Cython-accelerated python module to Rust, as a learning exercise and as a way of bringing the module into 2017 (previously it was Py2 only). The module uses PyTest and Tox. I was at the point where, using setuptools-rust, I could build and manually test the code, and it worked great (thanks!).

However, it would consistently fail with tox, complaining that the rust extension module wasn't present.

In the end, I learned that tox uses sdist normally, and apparently this won't trigger Rust module build? Because, it was fixed when I added usedevelop=True to the [testenv] block of my tox.ini; because setuptools-rust supports build when pythonX setup.py develop is invoked, this magically solved my problem.

All of which is to say... does it make sense to add a build-hook somewhere to support tox when it's operating in the default sdist-centric mode? :)

Thanks again for this module, it's amazing to be able to ship Rust so easily, at last. Next thing is learning about how to do binary wheels...

Docker build images

Hey folks,

I've just gone through building a docker image just to build binary wheels for a version of python I didn't have installed.

For the whole Rust:Python interact, it'd be cool to have a set of docker images with various Python/Rust versions and setuptools-rust preinstalled, to facilitate the ecosystem better.

Is anyone working on this already? I'm not a very good candidate for "dockerfile maintainer" as I have very limited experience so far, but I may still start a crappy repo if nobody else is doing yet yet.

setuptools is not listed as installation requirement

setuptools_rust has a runtime dependency on setuptools. However setup.cfg does not include setuptools as a requirement. While pip-based installations and virtual environments always have setuptools, other package managers do not necessarily install setuptools.

For example Fedora's RPM macros automatically create dependencies from package metadata. The auto-generate RPM package for setuptools-rust 0.11.6 is missing a runtime dependency on setuptools.

Patched commands ignore `setup.cfg` defined options

Hi there,
the setup.cfg file allows to define default values for arguments of setuptools / distutils commands. With the following file:

[build_rust]
inplace = true
release = true

simply running python setup.py build_rust will build all the extensions inplace and in release mode as expected.

Unfortunately, running python setup.py build or python setup.py build_ext does not. I suspect there's a step in the monkey-patching where the build_rust command is ran without having finalize_options called or something like that.

Add travis tests

we should compile example as a test.

Please include version in setup.cfg

I'm currently in the process to package setuptools-rust for Fedora Linux. There is a small, easy to fix problem in packaging of setuptools-rust. setup.cfg does not include version number of the package. The RPM packaging build chain re-creates the egg-info directory based on information from setup.py and setup.cfg. Without a version entry in setup.cfg, setuptools assumes version 0.0.0.

$ rpmbuild -bb python-setuptools-rust.spec
...
RPM build errors:
    File not found: /BUILDROOT/python-setuptools-rust-0.11.5-1.fc32.x86_64/usr/lib/python3.8/site-packages/setuptools_rust-0.11.5-py3.8.egg-info
$ ls /BUILDROOT/python-setuptools-rust-0.11.5-1.fc32.x86_64/usr/lib/python3.8/site-packages/
setuptools_rust  setuptools_rust-0.0.0-py3.8.egg-info

• https://packaging.python.org/guides/distributing-packages-using-setuptools/#version
• https://packaging.python.org/guides/distributing-packages-using-setuptools/#choosing-a-versioning-scheme

Can not build up example code

I got a problem to build up the example code.

If someone has any idea, please kindly tell me.

Thank you.

Environment:
python 3.6.5
rustup 1.11
rustc 1.27.0-nightly
cargo 1.27.0-nightly

$ python setup.py build

error: expected one of ( or <, found "_helloworld"
--> src/lib.rs:9:1
|
9 | #[modinit("_helloworld")]
| ^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

error: Could not compile hello-world.

Working with rust-cpython

I'm working on an implementation that binds to Rust via rust-cpython, which evidently is a supported binding. The declaration of Python module using rust-cpython involves using a macro (py_module_initializer!), rather than the pyo3 method of declaring module names using Rust attributes.

As a result, I'm having some trouble writing a functioning setup.py. Any guidance for determining what I should pass to RustExtension as the name of the module?

Generate console-script for Binding.Exec

check_rust shouldn't fail when running sdist

event if no rust extension is optional

Support optional option

Just like distutils:

https://docs.python.org/3/distutils/setupscript.html#other-options

The optional option is a boolean; if it is true, a build failure in the extension will not abort the build process, but instead simply not install the failing extension.

Show better error message when parsing Cargo.toml failed

I was confounded by the error message error: rust build failed; unable to find any *'...'.so in .... It turned out that I used single quote instead of double quote when quoting a string.
For example, changing the Cargo.toml in example/ to

[lib]
crate-type = ["cdylib"]
name = 'helloworld'

cause this confusing error.

Autopy installation error

Hi, not sure if this is the right place to open this issue.

But I don't know what else to do. I'm trying to install autopy and I get the following errors.
Thanks for all the help.

pi@raspberrypi:/ $ pip install autopy
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting autopy
  Using cached https://files.pythonhosted.org/packages/c6/36/bb1af0948e12cedef71da68b80960b3905805e30990d934102dcab012ddf/autopy-1.1.1.tar.gz
Building wheels for collected packages: autopy
  Building wheel for autopy (setup.py) ... error
  ERROR: Complete output from command /usr/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-lqf7zrrc/autopy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-c85mze7e --python-tag cp37:
  ERROR: running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib
  creating build/lib/autopy
  copying autopy/__init__.py -> build/lib/autopy
  running build_ext
  running build_rust
      Updating crates.io index
  cargo rustc --lib --manifest-path Cargo.toml --features pyo3/extension-module --release --verbose -- --crate-type cdylib
     Compiling arrayvec v0.4.11
     Compiling proc-macro2 v0.4.30
     Compiling lazy_static v1.3.0
     Compiling nodrop v0.1.13
       Running `rustc --crate-name build_script_build /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/arrayvec-0.4.11/build.rs --color never --crate-type bin --emit=dep-info,link -C opt-level=3 -C metadata=c8a43e12762695d1 -C extra-filename=-c8a43e12762695d1 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/build/arrayvec-c8a43e12762695d1 -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow`
       Running `rustc --crate-name build_script_build /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-0.4.30/build.rs --color never --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg 'feature="default"' --cfg 'feature="proc-macro"' -C metadata=d21b74293a25bb41 -C extra-filename=-d21b74293a25bb41 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/build/proc-macro2-d21b74293a25bb41 -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow`
       Running `rustc --crate-name lazy_static /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/lazy_static-1.3.0/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C metadata=7517fd82bb3c82a3 -C extra-filename=-7517fd82bb3c82a3 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/deps -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow`
       Running `rustc --crate-name nodrop /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/nodrop-0.1.13/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C metadata=3f4e338c46e747d6 -C extra-filename=-3f4e338c46e747d6 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/deps -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow`
  error: Could not compile `lazy_static`.
  
  Caused by:
    process didn't exit successfully: `rustc --crate-name lazy_static /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/lazy_static-1.3.0/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C metadata=7517fd82bb3c82a3 -C extra-filename=-7517fd82bb3c82a3 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/deps -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
  warning: build failed, waiting for other jobs to finish...
  error: Could not compile `arrayvec`.
  
  Caused by:
    process didn't exit successfully: `rustc --crate-name build_script_build /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/arrayvec-0.4.11/build.rs --color never --crate-type bin --emit=dep-info,link -C opt-level=3 -C metadata=c8a43e12762695d1 -C extra-filename=-c8a43e12762695d1 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/build/arrayvec-c8a43e12762695d1 -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
  warning: build failed, waiting for other jobs to finish...
  error: Could not compile `proc-macro2`.
  
  Caused by:
    process didn't exit successfully: `rustc --crate-name build_script_build /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-0.4.30/build.rs --color never --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg 'feature="default"' --cfg 'feature="proc-macro"' -C metadata=d21b74293a25bb41 -C extra-filename=-d21b74293a25bb41 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/build/proc-macro2-d21b74293a25bb41 -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
  error: cargo failed with code: 101
  
  ----------------------------------------
  ERROR: Failed building wheel for autopy
  Running setup.py clean for autopy
Failed to build autopy
Installing collected packages: autopy
  Running setup.py install for autopy ... error
    ERROR: Complete output from command /usr/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-lqf7zrrc/autopy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_w1lyn8p/install-record.txt --single-version-externally-managed --compile:
    ERROR: running install
    running build
    running build_py
    creating build
    creating build/lib.linux-armv7l-3.7
    creating build/lib.linux-armv7l-3.7/autopy
    copying autopy/__init__.py -> build/lib.linux-armv7l-3.7/autopy
    running build_ext
    running build_rust
    cargo rustc --lib --manifest-path Cargo.toml --features pyo3/extension-module --release --verbose -- --crate-type cdylib
       Compiling arrayvec v0.4.11
       Compiling libc v0.2.60
       Compiling nodrop v0.1.13
       Compiling cfg-if v0.1.9
         Running `rustc --crate-name build_script_build /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/arrayvec-0.4.11/build.rs --color never --crate-type bin --emit=dep-info,link -C opt-level=3 -C metadata=c8a43e12762695d1 -C extra-filename=-c8a43e12762695d1 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/build/arrayvec-c8a43e12762695d1 -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow`
         Running `rustc --crate-name build_script_build /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.60/build.rs --color never --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=2e454f5878c8d297 -C extra-filename=-2e454f5878c8d297 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/build/libc-2e454f5878c8d297 -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow`
         Running `rustc --crate-name cfg_if /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/cfg-if-0.1.9/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C metadata=a882545820a0c8d5 -C extra-filename=-a882545820a0c8d5 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/deps -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow`
         Running `rustc --crate-name nodrop /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/nodrop-0.1.13/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C metadata=3f4e338c46e747d6 -C extra-filename=-3f4e338c46e747d6 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/deps -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow`
       Compiling lazy_static v1.3.0
         Running `rustc --crate-name lazy_static /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/lazy_static-1.3.0/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C metadata=7517fd82bb3c82a3 -C extra-filename=-7517fd82bb3c82a3 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/deps -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow`
       Compiling autocfg v0.1.5
         Running `rustc --crate-name autocfg /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/autocfg-0.1.5/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C metadata=375f307f6fed4821 -C extra-filename=-375f307f6fed4821 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/deps -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow`
    error: Could not compile `lazy_static`.
    
    Caused by:
      process didn't exit successfully: `rustc --crate-name lazy_static /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/lazy_static-1.3.0/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C metadata=7517fd82bb3c82a3 -C extra-filename=-7517fd82bb3c82a3 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/deps -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
    warning: build failed, waiting for other jobs to finish...
    error: Could not compile `libc`.
    
    Caused by:
      process didn't exit successfully: `rustc --crate-name build_script_build /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.60/build.rs --color never --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=2e454f5878c8d297 -C extra-filename=-2e454f5878c8d297 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/build/libc-2e454f5878c8d297 -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
    warning: build failed, waiting for other jobs to finish...
    free(): invalid next size (fast)
    error: Could not compile `arrayvec`.
    
    Caused by:
      process didn't exit successfully: `rustc --crate-name build_script_build /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/arrayvec-0.4.11/build.rs --color never --crate-type bin --emit=dep-info,link -C opt-level=3 -C metadata=c8a43e12762695d1 -C extra-filename=-c8a43e12762695d1 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/build/arrayvec-c8a43e12762695d1 -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow` (signal: 6, SIGABRT: process abort signal)
    warning: build failed, waiting for other jobs to finish...
    error: Could not compile `autocfg`.
    
    Caused by:
      process didn't exit successfully: `rustc --crate-name autocfg /home/pi/.cargo/registry/src/github.com-1ecc6299db9ec823/autocfg-0.1.5/src/lib.rs --color never --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C metadata=375f307f6fed4821 -C extra-filename=-375f307f6fed4821 --out-dir /tmp/pip-install-lqf7zrrc/autopy/target/release/deps -L dependency=/tmp/pip-install-lqf7zrrc/autopy/target/release/deps --cap-lints allow` (signal: 11, SIGSEGV: invalid memory reference)
    error: cargo failed with code: 101
    
    ----------------------------------------
ERROR: Command "/usr/bin/python -u -c 'import setuptools, tokenize;__file__='"'"'/tmp/pip-install-lqf7zrrc/autopy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-_w1lyn8p/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-install-lqf7zrrc/autopy/

Cannot compile the example

warning: crate-type "cdylib" was not one of lib|rlib|dylib|staticlib
error: cannot compile `hello-world v0.1.0 (file:///home/dmitry/Projects/setuptools-rust/example)` as the target `x86_64-unknown-linux-gnu` does not support any of the output crate types

I tried removing crate-type = ["cdylib"] but then I get many errors like:

/root/.cargo/registry/src/github.com-1ecc6299db9ec823/num-traits-0.1.40/src/lib.rs:104:7: 104:21 error: use of unstable library feature 'op_assign_traits': recently added (see issue #28235)

active toolchain

nightly-x86_64-unknown-linux-gnu (default)
rustc 1.21.0-nightly (dd53dd5f9 2017-08-01)

Improve error message for missing rust compiler

It looks like the folks at pyca/cryptography are having some fun with lots of confused users having to install a Rust compiler for the first time.

We can almost certainly help out here with some better error messages and documentation upstream.

I've been meaning to add some Rust installation notes to the PyO3 guide for a while. Also the setuptools-rust error "can not find Rust compiler" should probably at the very least link to rustup. (And maybe to the PyO3 guide?)

cc @alex @tiran I see you two have been fielding lots of questions, so would be happy to do anything I can to ease the pain! Any suggestions from you two on what you'd like to see if I were to add some stuff?

PEP 517 support

Just a question: is PEP517 currently supported?

I see there have been a discussion about it in PyO3/maturin#2 and this repo specify a pyproject.toml, however when I build my project (cf setup.py, pyproject.toml), I get an error,

$ python -m pip install .
[...]
  error: failed to load source for a dependency on `vtext`
  
  Caused by:
    Unable to update /tmp
  
  Caused by:
    failed to read `/tmp/Cargo.toml`
[...]
...
ERROR: Could not build wheels for vtext which use PEP 517 and cannot be installed directly

python -m pip install .
Processing /home/rth/src/vtext/python
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting setuptools-rust>=0.10.2 (from vtext==0.1.0a3)
  Using cached https://files.pythonhosted.org/packages/5b/55/c19472293d7e77cd9b436dae17e4ff755545c0442b5af9e7929f277a637b/setuptools_rust-0.10.6-py3-none-any.whl
Collecting numpy>=1.15.0 (from vtext==0.1.0a3)
  Using cached https://files.pythonhosted.org/packages/1f/df/7988fbbdc8c9b8efb575029498ad84b77e023a3e4623e85068823a102b1d/numpy-1.18.4-cp37-cp37m-manylinux1_x86_64.whl
Collecting scipy>=1.1.0 (from vtext==0.1.0a3)
  Using cached https://files.pythonhosted.org/packages/dd/82/c1fe128f3526b128cfd185580ba40d01371c5d299fcf7f77968e22dfcc2e/scipy-1.4.1-cp37-cp37m-manylinux1_x86_64.whl
Requirement already satisfied: setuptools in /home/rth/miniconda3/envs/rust-env/lib/python3.7/site-packages (from vtext==0.1.0a3) (41.2.0)
Collecting toml>=0.9.0 (from setuptools-rust>=0.10.2->vtext==0.1.0a3)
  Using cached https://files.pythonhosted.org/packages/a2/12/ced7105d2de62fa7c8fb5fce92cc4ce66b57c95fb875e9318dba7f8c5db0/toml-0.10.0-py2.py3-none-any.whl
Collecting semantic-version>=2.6.0 (from setuptools-rust>=0.10.2->vtext==0.1.0a3)
  Using cached https://files.pythonhosted.org/packages/a5/15/00ef3b7888a10363b7c402350eda3acf395ff05bebae312d1296e528516a/semantic_version-2.8.5-py2.py3-none-any.whl
Building wheels for collected packages: vtext
  Building wheel for vtext (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/rth/miniconda3/envs/rust-env/bin/python /home/rth/miniconda3/envs/rust-env/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmpnf1rtlow
       cwd: /tmp/pip-req-build-tv9xeim2
  Complete output (86 lines):
  running bdist_wheel
  running build
  running build_py
  creating build/lib
  creating build/lib/vtext
  copying vtext/__init__.py -> build/lib/vtext
  copying vtext/stem.py -> build/lib/vtext
  copying vtext/vectorize.py -> build/lib/vtext
  copying vtext/tokenize.py -> build/lib/vtext
  creating build/lib/vtext/metrics
  copying vtext/metrics/__init__.py -> build/lib/vtext/metrics
  copying vtext/metrics/string.py -> build/lib/vtext/metrics
  creating build/lib/vtext/tests
  copying vtext/tests/__init__.py -> build/lib/vtext/tests
  copying vtext/tests/test_metrics.py -> build/lib/vtext/tests
  copying vtext/tests/test_stem.py -> build/lib/vtext/tests
  copying vtext/tests/test_vectorize.py -> build/lib/vtext/tests
  copying vtext/tests/test_tokenize.py -> build/lib/vtext/tests
  running egg_info
  writing vtext.egg-info/PKG-INFO
  writing dependency_links to vtext.egg-info/dependency_links.txt
  writing requirements to vtext.egg-info/requires.txt
  writing top-level names to vtext.egg-info/top_level.txt
  reading manifest file 'vtext.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'vtext.egg-info/SOURCES.txt'
  running build_ext
  running build_rust
  error: failed to load source for a dependency on `vtext`
  
  Caused by:
    Unable to update /tmp
  
  Caused by:
    failed to read `/tmp/Cargo.toml`
  
  Caused by:
    No such file or directory (os error 2)
  Traceback (most recent call last):
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 207, in <module>
      main()
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 197, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 141, in build_wheel
      metadata_directory)
    File "/tmp/pip-build-env-i0ssb0jt/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 213, in build_wheel
      wheel_directory, config_settings)
    File "/tmp/pip-build-env-i0ssb0jt/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 198, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-i0ssb0jt/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 250, in run_setup
      self).run_setup(setup_script=setup_script)
    File "/tmp/pip-build-env-i0ssb0jt/overlay/lib/python3.7/site-packages/setuptools/build_meta.py", line 143, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 28, in <module>
      packages=find_packages(),
    File "/tmp/pip-build-env-i0ssb0jt/overlay/lib/python3.7/site-packages/setuptools/__init__.py", line 144, in setup
      return distutils.core.setup(**attrs)
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-i0ssb0jt/overlay/lib/python3.7/site-packages/wheel/bdist_wheel.py", line 223, in run
      self.run_command('build')
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-i0ssb0jt/overlay/lib/python3.7/site-packages/setuptools_rust/build_ext.py", line 26, in run
      build_rust.run()
    File "/tmp/pip-build-env-i0ssb0jt/overlay/lib/python3.7/site-packages/setuptools_rust/build.py", line 313, in run
      self.build_extension(ext)
    File "/tmp/pip-build-env-i0ssb0jt/overlay/lib/python3.7/site-packages/setuptools_rust/build.py", line 95, in build_extension
      metadata = json.loads(check_output(metadata_command).decode("utf-8"))
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/subprocess.py", line 395, in check_output
      **kwargs).stdout
    File "/home/rth/miniconda3/envs/rust-env/lib/python3.7/subprocess.py", line 487, in run
      output=stdout, stderr=stderr)
  subprocess.CalledProcessError: Command '['cargo', 'metadata', '--manifest-path', 'Cargo.toml', '--format-version', '1']' returned non-zero exit status 101.
  ----------------------------------------
  ERROR: Failed building wheel for vtext
  Running setup.py clean for vtext
Failed to build vtext
ERROR: Could not build wheels for vtext which use PEP 517 and cannot be installed directly

while at the same time for me python setup.py install and bdist_wheel works fine.

Am I doing something wrong or is it simply not yet supported?

GitHub and Conda package sources for v0.11.6 don't match

I'm keen to use the change in 060d8b4 as I'd like to build using GNU instead of MSVC on Windows. That change was committed on 2020-11-14, and https://github.com/PyO3/setuptools-rust/tree/v0.11.6 was produced on 2020-12-13, and I can see the commit I care about in that tag's history. However, https://anaconda.org/conda-forge/setuptools-rust/0.11.6/download/noarch/setuptools-rust-0.11.6-pyhd8ed1ab_0.tar.bz2 doesn't contain this change.

This seems to indicate a problem of the release process here, if the GitHub and Conda sources don't match. Is this something that can be looked into, please?

Support multiple executables

"ModuleNotFoundError" for setuptools_rust module

When i try to install a cryptography library in ubuntu pc, i'm facing an below error.
Below package is getting installed as part of step in DOCKER IMAGE CREATION.

Collecting cryptography>=2.5 (from paramiko==2.7.2)
18:48:28 Downloading https://files.pythonhosted.org/packages/27/5a/007acee0243186123a55423d49cbb5c15cb02d76dd1b6a27659a894b13a2/cryptography-3.4.4.tar.gz

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-build-qcmsvhgh/cryptography/setup.py", line 14, in <module>
    from setuptools_rust import RustExtension
   ModuleNotFoundError: No module named 'setuptools_rust'

can someone please let me know, how to overcome this issue?

Setup :
OS - UBUNTU 18. 04. 05 LTS
Python3 : 3.6.9

Steps tried out :
-> I have tried to initiate the same step after upgrade pip install --> however, situation did not improve..
pip - 21.0.1
semantic-version 2.8.5
setuptools 40.6.2
setuptools-rust 0.11.6

Occasionally breaks pip installs of other software

I just got this error:

cathal@europa:~/Projects/python/PyGram/src$ sudo pip2 install ipython
[sudo] password for cathal: 
The directory '/home/cathal/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/cathal/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting ipython
  Downloading ipython-5.3.0-py2-none-any.whl (750kB)
    100% |████████████████████████████████| 757kB 420kB/s 
Requirement already satisfied (use --upgrade to upgrade): setuptools>=18.5 in /usr/lib/python2.7/dist-packages (from ipython)
Collecting backports.shutil-get-terminal-size; python_version == "2.7" (from ipython)
  Downloading backports.shutil_get_terminal_size-1.0.0-py2.py3-none-any.whl
Collecting pickleshare (from ipython)
  Downloading pickleshare-0.7.4-py2.py3-none-any.whl
Collecting simplegeneric>0.8 (from ipython)
  Downloading simplegeneric-0.8.1.zip
Collecting decorator (from ipython)
  Downloading decorator-4.0.11-py2.py3-none-any.whl
Collecting prompt-toolkit<2.0.0,>=1.0.4 (from ipython)
  Downloading prompt_toolkit-1.0.14-py2-none-any.whl (248kB)
    100% |████████████████████████████████| 256kB 668kB/s 
Collecting pexpect; sys_platform != "win32" (from ipython)
  Downloading pexpect-4.2.1-py2.py3-none-any.whl (55kB)
    100% |████████████████████████████████| 61kB 871kB/s 
Collecting pathlib2; python_version == "2.7" or python_version == "3.3" (from ipython)
  Downloading pathlib2-2.2.1-py2.py3-none-any.whl
Collecting pygments (from ipython)
  Downloading Pygments-2.2.0-py2.py3-none-any.whl (841kB)
    100% |████████████████████████████████| 849kB 534kB/s 
Collecting traitlets>=4.2 (from ipython)
  Downloading traitlets-4.3.2-py2.py3-none-any.whl (74kB)
    100% |████████████████████████████████| 81kB 882kB/s 
Requirement already satisfied (use --upgrade to upgrade): six>=1.9.0 in /usr/lib/python2.7/dist-packages (from prompt-toolkit<2.0.0,>=1.0.4->ipython)
Collecting wcwidth (from prompt-toolkit<2.0.0,>=1.0.4->ipython)
  Downloading wcwidth-0.1.7-py2.py3-none-any.whl
Collecting ptyprocess>=0.5 (from pexpect; sys_platform != "win32"->ipython)
  Downloading ptyprocess-0.5.1-py2.py3-none-any.whl
Collecting scandir; python_version < "3.5" (from pathlib2; python_version == "2.7" or python_version == "3.3"->ipython)
  Downloading scandir-1.5.tar.gz
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/scandir.egg-info
    writing pip-egg-info/scandir.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/scandir.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/scandir.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/scandir.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-ex6U0I/scandir/setup.py", line 54, in <module>
        'Programming Language :: Python :: Implementation :: CPython',
      File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
        dist.run_commands()
      File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 195, in run
        self.find_sources()
      File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 222, in find_sources
        mm.run()
      File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 306, in run
        self.add_defaults()
      File "/usr/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 335, in add_defaults
        sdist.add_defaults(self)
      File "/usr/lib/python2.7/dist-packages/setuptools/command/sdist.py", line 171, in add_defaults
        build_ext = self.get_finalized_command('build_ext')
      File "/usr/lib/python2.7/distutils/cmd.py", line 312, in get_finalized_command
        cmd_obj.ensure_finalized()
      File "/usr/lib/python2.7/distutils/cmd.py", line 109, in ensure_finalized
        self.finalize_options()
      File "/usr/lib/python2.7/dist-packages/setuptools/command/build_ext.py", line 134, in finalize_options
        self.check_extensions_list(self.extensions)
      File "/usr/local/lib/python2.7/dist-packages/setuptools_rust/build_ext.py", line 16, in check_extensions_list
        _build_ext.check_extensions_list(extensions)
    TypeError: unbound method check_extensions_list() must be called with build_ext instance as first argument (got list instance instead)

Add custom class to setuptools-rust example

New release breaks on python 3.5

The new release of setuptools-rust isn't compatible with python 3.5 since it uses fstrings. This is fine, especially since 3.5 is EoL now, but there is no minimum python version set in the package metadata. This means that people trying to use python 3.5 with setuptools-rust it will have pip install the new package version and fail. For example:

https://github.com/Qiskit/retworkx/pull/189/checks?check_run_id=1347529773#step:6:30

It would be good if the next release marked that the minimum supported version was 3.6.

release version is not working on osx

Release version broken:

rust|user@host ~ $ python --version
Python 3.5.3

rust|user@host ~ $ git clone https://github.com/PyO3/setuptools-rust.git
Cloning into 'setuptools-rust'...
[...skipped...]

rust|user@host ~ $ cd setuptools-rust/
rust|user@host ~/setuptools-rust $ pip install .
Processing /Users/user/setuptools-rust
[..skipped..]

rust|user@host ~/setuptools-rust $ cd example/
rust|user@host ~/setuptools-rust/example $ pip install .
Processing /Users/user/setuptools-rust/example
Installing collected packages: hello-rust
  Running setup.py install for hello-rust ... done
Successfully installed hello-rust-1.0

rust|user@host ~/setuptools-rust/example $ pip freeze
appdirs==1.4.3
hello-rust==1.0
packaging==16.8
pyparsing==2.2.0
semantic-version==2.6.0
setuptools-rust==0.5.1
six==1.10.0

rust|user@host ~/setuptools-rust/example $ python -c 'import hello_rust; hello_rust.hello()'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/user/setuptools-rust/example/hello_rust/__init__.py", line 1, in <module>
    from . import _helloworld
ImportError: cannot import name '_helloworld'

Debug version is working fine:

[...same steps from scratch as above...]

rust|user@host ~/setuptools-rust $ cd example/
rust|user@host ~/setuptools-rust/example $ pip install -e .
Obtaining file:///Users/user/setuptools-rust/example
Installing collected packages: hello-rust
  Running setup.py develop for hello-rust
Successfully installed hello-rust

rust|user@host ~/setuptools-rust/example $ pip freeze
appdirs==1.4.3
-e git+https://github.com/PyO3/setuptools-rust.git@4272c1c8d2dcd383e935bfe87913098a9fe4cc88#egg=hello_rust&subdirectory=example
packaging==16.8
pyparsing==2.2.0
semantic-version==2.6.0
setuptools-rust==0.5.1
six==1.10.0

rust|user@host ~/setuptools-rust/example $ python -c 'import hello_rust; hello_rust.hello()'
Rust says: Hello Python!
Rust got 42

RustExtension(optional=True) does not catch all build errors

The optional argument of RustExtension works well with missing or outdated Rust compiler. However it does not catch Cargo related build issues like transient network errors and offline builds. Is this a deliberate choice or a bug?

(I'm using a modified version of cryptography to demonstrate the issue.)

Without rustc installed

$ python3 setup.py build
...
running build_rust
Can not find Rust compiler
...

✔️

With mismatching Rust version

$ python3 setup.py build
...
running build_rust
Build optional Rust extension _rust failed.
Rust 1.49.0 does not match extension requirement >=1.99.0
...

✔️

With failing cargo command (e.g. offline builds)

$ CARGO_NET_OFFLINE=true python3 setup.py build
...
running build_rust
error: no matching package named `pyo3` found
location searched: registry `https://github.com/rust-lang/crates.io-index`
required by package `cryptography-rust v0.1.0 (/home/heimes/dev/py/cryptography/src/rust)`
As a reminder, you're using offline mode (--offline) which can sometimes cause surprising resolution failures, if this error is too confusing you may wish to retry without the offline flag.
...
Traceback (most recent call last):
  File "/home/heimes/dev/py/cryptography/setup.py", line 72, in <module>
    setup(
  File "/usr/lib/python3.9/site-packages/setuptools/__init__.py", line 165, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib64/python3.9/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib64/python3.9/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/lib64/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib64/python3.9/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/lib64/python3.9/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib64/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/lib64/python3.9/site-packages/cffi/setuptools_ext.py", line 142, in run
    base_class.run(self)
  File "/usr/lib64/python3.9/site-packages/cffi/setuptools_ext.py", line 142, in run
    base_class.run(self)
  File "/home/heimes/.local/lib/python3.9/site-packages/setuptools_rust/setuptools_ext.py", line 23, in run
    build_rust.run()
  File "/home/heimes/.local/lib/python3.9/site-packages/setuptools_rust/build.py", line 357, in run
    self.build_extension(ext)
  File "/home/heimes/.local/lib/python3.9/site-packages/setuptools_rust/build.py", line 120, in build_extension
    metadata = json.loads(check_output(metadata_command).decode("utf-8"))
  File "/usr/lib64/python3.9/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/lib64/python3.9/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cargo', 'metadata', '--manifest-path', 'src/rust/Cargo.toml', '--format-version', '1']' returned non-zero exit status 101.

🔴

Constant name for the generated library when no binding is used

Hi,

I have a simple python program (the repo is located here https://github.com/bmatthieu3/rust-pypi) that loads a dynamic library generated from cargo and using the CFFI package.

I would like to generate wheels for different platforms and for that purpose I am using setuptools-rust. I declare a new RustExtension with binding=NoBinding. (See https://github.com/bmatthieu3/rust-pypi/blob/master/setup.py )

I have noticed that depending on the version of python I'm using, the name of the library generated (a .so on linux) changes when I'm doing the command:
python setup.py build_rust

Compiling using python2 works as it generates the lib with the expected name (rust_pypi.so). That file will be correctly found by my python script.
However when executing the previous command using python3.6 I get a rust_pypi.cpython-36m-x86_64-linux-gnu.so . In other words, for all the python3.6 wheels, my python package will not be able to found the dynamic library.

How can I set a constant library name and be sure that it will always remain for the wheels that will be generated (independently from the platform and the python version used) ?

Thank you for your answer,
Matthieu

Problems trying to build a hello world app using setuptools-rust

I've created a hello world app https://github.com/brupelo/pysublime_fuzzy and when I try to generate some wheels out of it I'm facing some errors:

[vagrant@localhost ~]$ git clone https://github.com/brupelo/pysublime_fuzzy
Cloning into 'pysublime_fuzzy'...
remote: Enumerating objects: 23, done.
remote: Counting objects: 100% (23/23), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 23 (delta 6), reused 20 (delta 3), pack-reused 0
Unpacking objects: 100% (23/23), done.
[vagrant@localhost ~]$ cd pysublime_fuzzy/
[vagrant@localhost pysublime_fuzzy]$ docker pull quay.io/pypa/manylinux1_x86_64
Using default tag: latest
latest: Pulling from pypa/manylinux1_x86_64
7d0d9526f38a: Already exists
54e27cdac081: Already exists
45d8f3bab714: Already exists
b2e1d01082bb: Already exists
Digest: sha256:c47410598bc47614ef19b4320cf8203f1597bf847c9e504c2b0ba41af7c0f824
Status: Image is up to date for quay.io/pypa/manylinux1_x86_64:latest
[vagrant@localhost pysublime_fuzzy]$ docker run --rm -v `pwd`:/io quay.io/pypa/manylinux1_x86_64 /io/build-wheels.sh
docker: Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "exec: \"/io/build-wheels.sh\": permission denied": unknown.
[vagrant@localhost pysublime_fuzzy]$

Any idea how to proceed here? I've been trying for hours trying to figure out how to build my rust stuff for linux with no success so far.

Thanks in advance.

Running: "python setup.py bdist_rpm --requires python"

When running the command "python setup.py bdist_rpm --requires python" it fails with the following error:

Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.eUdFWG

• umask 022
• cd /workspace/workspace/multi_build_rust/OPERATING_SYSTEM/Fedora-25/PRODUCT/setuptools-rust/RELEASE/production/build/build/bdist.linux-x86_64/rpm/BUILD
• cd setuptools-rust-0.10.5
• python setup.py build
  Traceback (most recent call last):
  File "setup.py", line 15, in
  long_description=open("README.md").read(),
  IOError: [Errno 2] No such file or directory: 'README.md'
  error: Bad exit status from /var/tmp/rpm-tmp.eUdFWG (%build)
  Bad exit status from /var/tmp/rpm-tmp.eUdFWG (%build)

"universal2" and alternate deployment target on macOS

Platform: macOS, in particular M1 systems.

These systems can run both arm64 (native) and x86_64 (in emulation) code, with per process selection using the "arch" command.

For C extensions it is possible to build both architectures on an M1 system (basically a form of cross compiling), including merging code for the two architectures in a fat binary ("Universal 2" in Apple-speak).

setuptools_rust will currently always build arm64 code only, even if python was started in x86_64 mode (e.g. arch -x86_64 python3.9 setup.py build_ext).

It would be nice if setuptools_rust were able to build "universal2" wheels.

An related to this: for C extensions Python by default builds extension that work on the deployment target used for the python binary. It might be necessary to enhance setuptools_rust to do something similar.

BTW. I hope to have some time to work on a PR in the near future.

ubuntu cannot find rust compiler

Currently I face the issue someone had here #54. Sadly i cannot preform the command without sudo, as I cant really change the requirements for every file it needs to access. Entering root mode is impossible on ubuntu, and running the command from bash file with sudo doesnt work either.

OS: lubuntu 18.04
CMD: python3 setup.py install
without sudo: Errno 13 permission denied
with sudo: Cannot find rust compiler

How do I run it?

html-py-ever link in README.md is broken

I just noticed and wanted to let you know that the link to html-py-ever in the Setup section in you README is broken.
Greetings

"Error: Can not find Rust compiler"

I am trying to run the "word-count" example and am getting a "Can not find Rust compiler" error, which I believe is being thrown from the get_rust_version method in setuptools_rust/utils.py. Running rustc -V gives me "rustc 1.36.0-nightly (d595b1135 2019-05-10)".

🌍 Environment

• Your operating system and version: Ubuntu 18.04.1
• Your python version: 3.6.7 [GCC 8.2.0] on Linux
• How did you install python (e.g. apt or pyenv)? Did you use a virtualenv?: apt. No.
• Your rust version (rustc --version): "rustc 1.36.0-nightly"
• Are you using the latest pyo3 version? Have you tried using latest master (replace version = "0.x.y" with git = "https://github.com/PyO3/pyo3")? Yes. Yes.

💥 Reproducing

CLI:
[/pyo3/examples/word-count/] sudo python3 setup.py install

Trace:

running install
running bdist_egg
running egg_info
writing word_count.egg-info/PKG-INFO
writing dependency_links to word_count.egg-info/dependency_links.txt
writing top-level names to word_count.egg-info/top_level.txt
reading manifest file 'word_count.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'word_count.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/word_count
copying word_count/init.py -> build/lib.linux-x86_64-3.6/word_count
running build_ext
running build_rust
error: Can not find Rust compiler

use CARGO_TARGET_DIR env variable

build working for linux, macos and windows with github actions

I've just setup rtoml to build and upload binaries for linux, macos and windows using github actions with setuptools-rust.

The build uses cibuildwheel rather than the manual build-wheels.sh approach. See here for details.

This is not an issue or request for any changes (though perhaps it would be helpful to add a link to rtoml as an example?) so feel free to close this issue, I just thought it might be a helpful pointer for anyone else using PyO3 and/or setuptools-rust.

Thanks so much for PyO3 and setuptools-rust.

Difference in use cases between setuptools-rust and pyo3-pack

The first line of the readme currently states,

You might want to check out pyo3-pack, the successor to this project, which allows to develop, build and upload without any configuration

However, if I have a python package with a few rust extensions, one cannot use pyo3-pack to build wheels for it (that would also include the python code and possibly some C Python extensions), right? So in that use case, setuptools-rust will always be needed unless I am missing something?

Maybe that could be more clearly stated both in the pyo3-pack and setuptools-rust readme? I started by using pyo3-pack as it was advertised as "better" before realizing that setuptools-rust is more appropriate for my use-case.

Tag for release 0.10.5

Could you please create a tag for version 0.10.5? That would be good! :)

Specify --target automatically based on which python is in use?

Currently setuptools-rust always invokes cargo in a way that uses the default target. This works great for most use cases. We're currently hitting an issue where this leads to pain when using a 32-bit python on a 64-bit windows, since your default rust target will also be 64-bit!

The easiest thing would be for setuptools-rust to automatically detect this, and specify --target.

Questions:
a) Does this make sense, or is it your expectation that the caller is responsible for discovering this and configuring appropriately?
b) My instinct is to make this handle the exactly one situation of "windows with 32-bit Python and 64-bit rust", is that ok, or do you want a much more general solution?

license?

Hi! I would like to package this for Arch Linux's AUR, but there is no clear indication of the license this software is released under. PyPi lists it as MIT but I just want to confirm.

bdist_wheel fails if build_ext isn't run first.

I have two questions; one is about a possible bug, the other is about project structure.
The "bug" is if you don't run build_ext before running bdist_wheel, the build happens after the package (or at least the egg) is made. As a result the shared library never gets packaged. I posted an example output at the bottom. You'll see there is no .so added to the wheel in the place where I expect it.

Would it be possible to run the Cargo build before the wheel is built? I'm not sure how regular setuptools handles this.

The second issue is more general; how do I locate the shared object and header? Currently I have them looking for the .so/.dll/.dylib/.pyo along a relative path.
However, I see that the object is copied to the root. Is that all I should package and look for? Will that work for development and packaging?
Is there an existing setuptools convention?

python setup.py --verbose bdist_wheel
running bdist_wheel
running build
running build_py
creating build
creating build/lib
creating build/lib/rust_pypi_example
copying rust_pypi_example/cli.py -> build/lib/rust_pypi_example
copying rust_pypi_example/rust_pypi_example.py -> build/lib/rust_pypi_example
copying rust_pypi_example/__init__.py -> build/lib/rust_pypi_example
running egg_info
creating rust_pypi_example.egg-info
writing rust_pypi_example.egg-info/PKG-INFO
writing dependency_links to rust_pypi_example.egg-info/dependency_links.txt
writing entry points to rust_pypi_example.egg-info/entry_points.txt
writing requirements to rust_pypi_example.egg-info/requires.txt
writing top-level names to rust_pypi_example.egg-info/top_level.txt
writing manifest file 'rust_pypi_example.egg-info/SOURCES.txt'
reading manifest file 'rust_pypi_example.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'rust_pypi_example.egg-info/SOURCES.txt'
creating build/lib/rust_pypi_example/rust
copying rust_pypi_example/rust/Cargo.toml -> build/lib/rust_pypi_example/rust
creating build/lib/rust_pypi_example/rust/src
copying rust_pypi_example/rust/src/lib.rs -> build/lib/rust_pypi_example/rust/src
copying rust_pypi_example/rust/src/rust_pypi_example.h -> build/lib/rust_pypi_example/rust/src
running build_ext
running build_rust
cargo rustc --lib --manifest-path rust_pypi_example/rust/Cargo.toml --release -- --crate-type cdylib
   Compiling rust_pypi_example v0.8.0 (file:///home/matt/workspace/rust_pypi_example/rust_pypi_example/rust)
    Finished release [optimized] target(s) in 0.72 secs
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/rust_pypi_example
creating build/bdist.linux-x86_64/wheel/rust_pypi_example/rust
creating build/bdist.linux-x86_64/wheel/rust_pypi_example/rust/src
copying build/lib/rust_pypi_example/rust/src/lib.rs -> build/bdist.linux-x86_64/wheel/rust_pypi_example/rust/src
copying build/lib/rust_pypi_example/rust/src/rust_pypi_example.h -> build/bdist.linux-x86_64/wheel/rust_pypi_example/rust/src
copying build/lib/rust_pypi_example/rust/Cargo.toml -> build/bdist.linux-x86_64/wheel/rust_pypi_example/rust
copying build/lib/rust_pypi_example/cli.py -> build/bdist.linux-x86_64/wheel/rust_pypi_example
copying build/lib/rust_pypi_example/rust_pypi_example.py -> build/bdist.linux-x86_64/wheel/rust_pypi_example
copying build/lib/rust_pypi_example/__init__.py -> build/bdist.linux-x86_64/wheel/rust_pypi_example
copying build/lib/rust_pypi_example.cpython-36m-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/wheel
running install_egg_info
Copying rust_pypi_example.egg-info to build/bdist.linux-x86_64/wheel/rust_pypi_example-0.8.0-py3.6.egg-info
Copying entry_points.txt to build/bdist.linux-x86_64/wheel/rust_pypi_example-0.8.0-py3.6.egg-info/entry_points.txt
Copying PKG-INFO to build/bdist.linux-x86_64/wheel/rust_pypi_example-0.8.0-py3.6.egg-info/PKG-INFO
Copying SOURCES.txt to build/bdist.linux-x86_64/wheel/rust_pypi_example-0.8.0-py3.6.egg-info/SOURCES.txt
Copying not-zip-safe to build/bdist.linux-x86_64/wheel/rust_pypi_example-0.8.0-py3.6.egg-info/not-zip-safe
Copying top_level.txt to build/bdist.linux-x86_64/wheel/rust_pypi_example-0.8.0-py3.6.egg-info/top_level.txt
Copying requires.txt to build/bdist.linux-x86_64/wheel/rust_pypi_example-0.8.0-py3.6.egg-info/requires.txt
running install_scripts
creating build/bdist.linux-x86_64/wheel/rust_pypi_example-0.8.0.dist-info/WHEEL

Note no .so

Support "cross"

cross is a great tool for cross compile Rust code to other platform, support it will be awesome ! (so build Python wheel for more platform will be easier)

modification we may need:

• let user can build wheel for different target platform (need to pass --target $TARGET to rustc)
• install cross, this will be required only if you want to use cross feature
• let build command pass cross instead of cargo (if user choose cross)

html-py-ever: travis -> pypi

It would be very useful to see an example of how to deploy python/rust wheels to PyPI from travis.

I imagine it's a fairly simple frankensteining of the current html-py-ever and PyPA's manylinux demo but a concrete example would be great.

Rust binding in Cryptography 3.4+ dynamically links with non-existent library on macOS

Issue cloned from the Cryptography issue tracker per request, as setuptools-rust handles this for them: pyca/cryptography#5807

The Cryptography wheel for macOS includes _rust.abi3.so. This references a library that isn't shipped and has a path clearly from the build machine:

(foo) dpage@hal:~/git/pgadmin4/foo/lib/python3.8/site-packages/cryptography/hazmat/bindings$ otool -L _rust.abi3.so 
_rust.abi3.so:
	/Users/runner/work/cryptography/cryptography/cryptography-3.4.4/src/rust/target/release/deps/libcryptography_rust.dylib (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
	/usr/lib/libresolv.9.dylib (compatibility version 1.0.0, current version 1.0.0)

Whilst this may not break the package, it:

A) Is not good practice
B) Breaks build systems that automatically fixup library paths when creating relocatable appbundles.

B is the issue I'd like to see fixed, as the current situation requires me to modify code that's worked for many years (15+) to ignore errors when processing this particular file.

An additional issue, again likely benign for the general use case, is that _rust.abi3.so does not have the executable bit set as is generally the norm for shared libraries on *nix platforms. This also required me to change build code, as searches for binaries that need to be codesigned filtered possible targets based on the executable bit, before doing further tests to see if the file was actually an executable or library (which gets very expensive without that check, in a codebase with thousands of Python/JS/HTML files etc).

Probably not relevant, but here's my system info:

(foo) dpage@hal:~/git/pgadmin4/foo/lib/python3.8/site-packages/cryptography/hazmat/bindings$ pip freeze
cffi==1.14.4
cryptography==3.4.4
pycparser==2.20
(foo) dpage@hal:~/git/pgadmin4/foo/lib/python3.8/site-packages/cryptography/hazmat/bindings$ python --version
Python 3.8.2
(foo) dpage@hal:~/git/pgadmin4/foo/lib/python3.8/site-packages/cryptography/hazmat/bindings$ pip --version
pip 21.0.1 from /Users/dpage/git/pgadmin4/foo/lib/python3.8/site-packages/pip (python 3.8)
(foo) dpage@hal:~/git/pgadmin4/foo/lib/python3.8/site-packages/cryptography/hazmat/bindings$ sw_vers -productVersion
11.1

Thanks!

Incorrectly pack multiple versions of .so in one wheel

When using build-wheels.sh, in the for-loop of PYBIN:

1. When PYBIN=py35, pip wheel generates build/lib/xxx.cpython-35m-x86_64-linux-gnu.so and pack it into py35.whl

2. When PYBIN=py36, pip wheel generates build/lib/xxx.cpython-36m-x86_64-linux-gnu.so and pack BOTH py35.so and py36.so into py36.whl

and so on.

Thus, the resulting py38.whl contains all .so files, from py35.so to py38.so.

I don't how setuptools works, maybe the build/lib should be cleaned in advance?