Unable to install with `conda env create -f dicodile_env.yml`

[chmendoza]

Unable to install with `conda env create -f dicodile_env.yml`

[tomMoral]

Hello, thanks for the report.
This error seems strange as it seems that what is failing when trying to install dask-jobqueue.
However, dask-jobqueue is not used in the project, so we shoudl simply remove it from the env. (I think this is a leftover thing from when I tried to make a non-MPI backend).
Could you try removing dask-jobqueue from the dependencies and retry building the env?

[tomMoral]

It seems that there is something more profound than that as our CI is broken now...
https://github.com/tomMoral/dicodile/pull/44/checks?check_run_id=3828456175

I will try to check why the install is not working. Most probably, we need to remove the toml import which does not seem to be a deps of setuptools_scm anymore?

[chmendoza]

Hi Thomas,

Thanks for getting back to me. I removed the line

- git+https://github.com/dask/dask-jobqueue.git

from dicodile_env.yml, but it keeps throwing an error related to the import toml line in setup.py:

Installing pip dependencies: - Ran pip subprocess with arguments:
['/home/cmendoza/anaconda3/envs/dicodile/bin/python', '-m', 'pip', 'install', '-U', '-r', '/home/cmendoza/dicodile/condaenv.va6vdibt.requirements.txt']
Pip subprocess output:
Obtaining file:///home/cmendoza/dicodile (from -r /home/cmendoza/dicodile/condaenv.va6vdibt.requirements.txt (line 2))
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'

Pip subprocess error:
  ERROR: Command errored out with exit status 1:
   command: /home/cmendoza/anaconda3/envs/dicodile/bin/python /home/cmendoza/anaconda3/envs/dicodile/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpcjatgml0
       cwd: /home/cmendoza/dicodile
  Complete output (17 lines):
  Traceback (most recent call last):
    File "/home/cmendoza/anaconda3/envs/dicodile/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 349, in <module>
      main()
    File "/home/cmendoza/anaconda3/envs/dicodile/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 331, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/cmendoza/anaconda3/envs/dicodile/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 117, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-sqtpssok/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 154, in get_requires_for_build_wheel
      return self._get_build_requires(
    File "/tmp/pip-build-env-sqtpssok/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 135, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-sqtpssok/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 150, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 5, in <module>
      import toml  # noqa: F401
  ModuleNotFoundError: No module named 'toml'
  ----------------------------------------
WARNING: Discarding file:///home/cmendoza/dicodile. Command errored out with exit status 1: /home/cmendoza/anaconda3/envs/dicodile/bin/python /home/cmendoza/anaconda3/envs/dicodile/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpcjatgml0 Check the logs for full command output.
ERROR: Command errored out with exit status 1: /home/cmendoza/anaconda3/envs/dicodile/bin/python /home/cmendoza/anaconda3/envs/dicodile/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpcjatgml0 Check the logs for full command output.

failed

CondaEnvException: Pip failed

I added a line in setup.py to print the system path variable:

['/home/cmendoza/anaconda3/envs/dicodile/lib/python3.8/site-packages/pip/_vendor/pep517/in_process', '/tmp/pip-build-env-sqtpssok/site', '/home/cmendoza/anaconda3/envs/dicodile/lib/python38.zip', '/home/cmendoza/anaconda3/envs/dicodile/lib/python3.8', '/home/cmendoza/anaconda3/envs/dicodile/lib/python3.8/lib-dynload', '/tmp/pip-build-env-sqtpssok/overlay/lib/python3.8/site-packages', '/tmp/pip-build-env-sqtpssok/normal/lib/python3.8/site-packages']

I don't have experience packaging on Python, but you have this line in pyproject.toml:

requires = ["setuptools>=42", "wheel", "setuptools_scm[toml]>=3.4"]

so, maybe toml is being used by setuptools_scm...

[chmendoza]

After a "setuptools_scm toml import error" search, I found this: pypa/setuptools-scm#608

I replaced toml by tomli in the two instances where it appears, and voilá, I was able to build dicodile! No errors, no warnings.

More specifically, I changed this line pyproject.toml

requires = ["setuptools>=42", "wheel", "setuptools_scm[toml]>=3.4"]

by this line

requires = ["setuptools>=42", "wheel", "setuptools_scm[tomli]>=3.4"]

and the impor toml line in setup.py by import tomli.

My system:
Ubuntu 18.04
conda 4.9.2

After skimming the docs of setuptools_scm, I am guessing that you might not need to specify tomli as requirement to setuptools_scm, and that as you said import tomli is not even necessary. I could test that later. Do you want me to send a PR with those changes?

[chmendoza]

Ok, I just double checked, you can just get rid of those two occurrences of toml in your package. It will build without them.

[tomMoral]

Thanks a lot for getting back to us!
Apparently, setuptools_scm changed a bit and we did not detect it as the CI is not running dayly. :)
We removed the import on toml altogether as it is not needed anymore.

I just merged #45 that should resolve this on master.

[tomMoral]

I closed the issue, let us know if you have further problems!