Will mdtraj run on Python 3.11?

[cclauss]

Will mdtraj run on Python 3.11?

[cclauss]

It's not obvious to me what versions of Cython support Python 3.11.

Cython v0.29.28 is the minimum for Python 3.11 and Cython v0.29.32 is current.

• https://cython.readthedocs.io/en/latest/src/changes.html#id44

[truatpasteurdotfr]

Thx for the fix conda-forge/mdtraj-feedstock@00cc3e8, building now works after the force cython pass.

[mattwthompson]

I'm not very familiar with Nix so I can't help a whole lot with the automation here, unfortunately. If 3.11rcN is built somewhere on Conda I could try it out that way. In general I expect it to work since the code has barely changed in a couple of years, but it might take a few minor tweaks.

[mattwthompson]

conda-forge/python-feedstock#577

[mattwthompson]

Related conda-forge/mdtraj-feedstock#45

  mdtraj/geometry/drid.cpp:230:12: fatal error: longintrepr.h: No such file or directory
    230 |   #include "longintrepr.h"
        |            ^~~~~~~~~~~~~~~
  compilation terminated.
  error: command '/home/conda/feedstock_root/build_artifacts/mdtraj_1666877920242/_build_env/bin/x86_64-conda-linux-gnu-cc' failed with exit code 1
  error: subprocess-exited-with-error
  
  × Building wheel for mdtraj (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

[chrisburr]

The import probably just needs to be changed to Python.h, from the Python 3.11 release notes:

The non-limited API files cellobject.h, classobject.h, code.h, context.h, funcobject.h, genobject.h and longintrepr.h have been moved to the Include/cpython directory. Moreover, the eval.h header file was removed. These files must not be included directly, as they are already included in Python.h: Include Files. If they have been included directly, consider including Python.h instead. (Contributed by Victor Stinner in bpo-35134.)

[mattwthompson]

Thanks! I think that file is generated by Cython, let's see what I can get rolling in CI builds ...

[chrisburr]

Ah yes, the sources have to be cythonized with a newer version of cython for Python 3.11 to work.

[truatpasteurdotfr]

Will that be included in the next release version? self compiled on CentOS-7 python 3.11 from source, with openssl 3.0.7 from source is failing pip3 install mdtraj (mdtraj-1.9.7.tar.gz) with:

...
      building 'mdtraj.geometry.drid' extension
      gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -I/c7/shared/openssl/3.0.7-gcc-9.3.1-devtoolset-9/include -I/opt/rh/devtoolset-9/root/usr/include -I/c7/shared/openssl/3.0.7-gcc-9.3.1-devtoolset-9/include -I/opt/rh/devtoolset-9/root/usr/include -I/c7/shared/Python/3.11.0/include -I/c7/shared/openssl/3.0.7-gcc-9.3.1-devtoolset-9/include -I/opt/rh/devtoolset-9/root/usr/include -I/c7/shared/Python/3.11.0/include -I/c7/shared/openssl/3.0.7-gcc-9.3.1-devtoolset-9/include -I/opt/rh/devtoolset-9/root/usr/include -fPIC -Imdtraj/geometry/include -I/tmp/pip-build-env-_yeski1m/overlay/lib/python3.11/site-packages/numpy/core/include -I/c7/shared/Python/3.11.0/include/python3.11 -c mdtraj/geometry/drid.cpp -o build/temp.linux-x86_64-cpython-311/mdtraj/geometry/drid.o -fopenmp -msse2 -mssse3 -O3 -funroll-loops --std=c++11 -Wno-unused-function -Wno-unreachable-code -Wno-sign-compare
      mdtraj/geometry/drid.cpp:230:12: fatal error: longintrepr.h: No such file or directory
        230 |   #include "longintrepr.h"
            |            ^~~~~~~~~~~~~~~
      compilation terminated.
      error: command '/opt/rh/devtoolset-9/root/usr/bin/gcc' failed with exit code 1
      [end of output]
 ...

[mattwthompson]

Probably at some point, either I need to get #1758 or working or somebody else can fix it. PRs welcome.

[mattwthompson]

It's not obvious to me what versions of Cython support Python 3.11. I have had a hard time figuring out what is compatible with what. I'm not eager to push this forward until a new Cython release materializes.

[peastman]

For what it's worth, OpenMM builds and runs correctly under 3.11, and it also uses Cython. I didn't need to make any changes for it.

[mattwthompson]

Could you (or maybe @chrisburr) guide me on how to cythonize the codebase? The farthest I got was cythonize -3 -f mdtraj generating a long list of errors, which led me to believe that's not what I was supposed to be doing.

[peastman]

You shouldn't need to do that. It's done automatically at install time in setup.py:

mdtraj/setup.py

Line 304 in 1a75727

metadata['ext_modules'] = cythonize(extensions, language_level=sys.version_info[0])

[mattwthompson]

That's what I thought ... I can get it built on my machine and tests pass (except for optional dependencies I don't want to bother installing). Maybe the real issue is in the conda build process?

[mattwthompson]

Thanks to @chrisburr I think this is resolved now.

We still need to get the source builds online again but that's blocked by other issues.