Python crashes on import when using pip editable install on M1

[drdavella]

Python crashes on import when using pip editable install on M1

[davidhewitt]

Sorry for the slow reply. Would you be willing to push your project with instructions as a minimal repro? I just tested on my M1 Mac and don't get crashes with pip install -e.

[drdavella]

@davidhewitt no problem at all. Here's a package that minimally reproduces for me: https://github.com/drdavella/rust-extension. I'm installing from within that directory using pip install -e ..

Oddly, changing the number of println statements in the print_hello function seems to change the behavior on import. If I delete one of the statements the import works fine. This is all very strange to me and it's possible that I've just done something silly.

[davidhewitt]

Definitely working fine for me. Here's the instructions I'm running:

conda create -n str-326 python=3.10
conda activate str-326
pip install -e .
python -c "import mypackage.bindings._corelib; mypackage.bindings._corelib.print_hello()"

I just get the expected prints, and can edit the statements and recompile with pip install -e again fine. What rust compiler are you using? Sounds more like a miscompilation than anything else.

[drdavella]

It looks like I was using an older version of nightly from October, but even when I updated to latest (rustc 1.70.0-nightly (9df3a39fb 2023-04-11)) I have the same problem. Just as a sanity check, is there a way to verify which version is getting picked up by setuptools-rust? I don't see it in the output with pip -vv install -e .

[drdavella]

Alright after a bit more investigation, it looks like this is a problem with System Integrity Protection on M1 and not a problem with setuptools-rust itself. I found the following crash report related to this process:

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGKILL (Code Signature Invalid))
Exception Codes:       UNKNOWN_0x32 at 0x00000001033e4000
Exception Codes:       0x0000000000000032, 0x00000001033e4000
Exception Note:        EXC_CORPSE_NOTIFY

Termination Reason:    Namespace CODESIGNING, Code 2 

VM Region Info: 0x1033e4000 is in 0x1033e4000-0x1034ec000;  bytes after start: 0  bytes before end: 1081343
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      VM_ALLOCATE                 1032e4000-1033e4000    [ 1024K] rw-/rwx SM=PRV  
--->  mapped file                 1033e4000-1034ec000    [ 1056K] r--/rwx SM=COW  ...t_id=ced2d11d
      GAP OF 0x4b14000 BYTES
      MALLOC_MEDIUM               108000000-108800000    [ 8192K] rw-/rwx SM=PRV  

I was able to resolve the issue with the following command:

$ codesign --sign - src/mypackage/bindings/_corelib.abi3.so

I found a similar issue here nodejs/node#40827.

Just adding documentation in case anyone else encounters this issue. I'm not sure whether there's a way to address this on your end, but please feel free to close. Thanks for your help.

[davidhewitt]

Ah, that makes sense. Thanks for sharing the solution here!