MDTraj 1.9.7 on M1 chip fails with image_molecules()

[alexholehouse]

MDTraj 1.9.7 on M1 chip fails with image_molecules()

[alexholehouse]

@rmcgibbo - absolutely; well aware, and your involvement here has been above and beyond anyone's wildest dreams :).

Actually, providing a greater level of administrative/computational support for mdtraj is something Greg (Bowman) and I have discussed on and off quite a few times over the last few years. I'll connect with you directly to follow up here.

[sroet]

that specific error during compilation seems to be with compiling numpy from source?

Building wheels for collected packages: numpy
      Building wheel for numpy (setup.py): started
      Building wheel for numpy (setup.py): finished with status 'error'

I don't have access to an M1 mac, but you could try to pull all the dependencies from conda and only install mdtraj like:

conda create -n test python=3.9
conda activate test
conda install mdtraj --only-deps
python -m pip install mdtraj --no-cache-dir

and see if that works? If that install works, but still gives you nans, do you mind repeating the above with a development install of mdtraj?
If so, you can run:

conda deactivate
conda create -n test python=3.9
conda activate test
conda install mdtraj --only-deps
python -m pip install git+https://github.com/mdtraj/mdtraj.git

and see if that issue has already been resolved since the last release?

[alexholehouse]

Thanks for the quick response!

So I'd actually done exactly this (albeit backwards - install mdtraj with dependencies, then uninstall ONLY mdtraj with --force flag, and reinstall again). This, and the exact receipt above:

conda create -n test python=3.9
conda activate test
conda install mdtraj --only-deps
python -m pip install mdtraj --no-cache-dir

Fail again with a LONG list of warnings/errors and then:

ERROR: Failed building wheel for numpy
      Failed to build numpy
      ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects

Finally got around this by making the pyproject.toml file invisible and manually addressing the Python version-specific numpy dependencies defined here.

## Content of pyproject.toml
[build-system]
requires = [
    "setuptools",
    "wheel",
    "numpy==1.11.3; python_version=='3.6'",
    "numpy==1.14.6; python_version=='3.7'",
    "numpy==1.17.5; python_version=='3.8'",
    "numpy==1.19.5; python_version=='3.9'",
    "numpy==1.21.3; python_version=='3.10'",
    "numpy; python_version>='3.11'",
    "Cython>0.28",
]

For completeness in case someone else runs into this issue for building from source on ARM architecture, here are the steps I took (assumes you already have conda-forge in your channels, ofc):

conda deactive # make sure all this is done from base
conda create -n test python=3.9
conda activate test
conda install mdtraj --only-deps

# remove numpy because we're going to manually force a numpy 
# install with the right version
conda uninstall numpy --force

# install 1.19.5 as per requirement above
conda install numpy=1.19.5

# now manually install Cython, which is needed to build from source
# but is not part of the strict mdtraj dependencies from conda-forge
conda install cython 

# grab current repo
git clone [email protected]:mdtraj/mdtraj.git
cd mdtraj

# move pyproject.toml so it's invisible and not used during construction.
# I don't understand why, but having this here seems to always trigger 
# a new numpy compilation from source? Not gonna try and figure that one
# now but just FYI - it didn't seem to do this on the x86 architecture but I
# haven't rigorously checked all python/library versions match, so, take
# with a grain of salt
mv pyproject.toml ignore_me.txt

pip install -e .

Having done all this:

1. Builds successfully and other mdtraj functionality works (good!)
2. image_molecules() remains broken with the same issue (bad!)

But at least now I have a local buildable version to debug from! I will (probably over the weekend) dive into this and see if I can figure out what's what; in the meantime, if anyone else wants to address this please do!

[rmcgibbo]

Hi Alex and Sander,

I just wanted to give you a heads up re: https://github.com/mdtraj/mdtraj/pull/1738/files.

This is one of those issues where I'm probably not able to help, and I don't want anyone to have the wrong impression about the maintenance status of this project.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[sukritsingh]

With the release of 1.9.8 – is this issue still existing for you, Alex? I still am using x86 only so I've never had a chance to try (and fail) on M1

[alexholehouse]

YES! At least the original issue I had now works! Thanks all; please feel free to close this!