Installation via miniconda & native python breaks at build step for cffi

[abiddanda54gene]

Installation via miniconda & native python breaks at build step for cffi

[abiddanda54gene]

The conda environment definition above helps with the issue regarding blist in #179 to some degree.

[pjvandehaar]

Thanks, this is a good error-report. I appreciate that, because it makes these miserable installation issues at least a little more pleasant to work on. I'm busy today, so I'm sorry that this response isn't great, but I'll try to look into this more later.

On macOS:

• Maybe adding wheel to your pheweb.yaml would help, so that it wouldn't have to compile blist?
• If that doesn't fix it, maybe replace your third line with pip install blist and if that breaks, talk to the blist devs at https://github.com/DanielStutzbach/blist ? I see the issue DanielStutzbach/blist#90 there, which looks like it might be the same problem. Maybe bump that issue, because the blist's author is still active.
• I've been gradually replacing C libraries (like blist) with python code, because of these frustrating compilation errors, but I'd like to keep blist around if this can be resolved. And unfortunately I can't reproduce your error on my mac because I'm still on clang v11 on Big Sur and that's causing other errors. Are you on clang v12? What version of macOS?

On Linux: that error is puzzling. Does the file build/temp.linux-x86_64-3.9/pheweb.load.cffi._x.cpp actually not exist? Sorry, I can't think of any ways to figure out this problem. Maybe there's some command line option for pip that will change how it compiles cffi cpp? Or maybe there's a way to use strace or some file system logging to check why that file isn't being created?

[abiddanda54gene]

Is there a stable release version tagged @pjvandehaar? Or potentially to have a version installed on bioconda? That would certainly help with installation issues like this if possible. I'll fiddle around a little bit more, but let me know if there is anything else I could potentially help with.

[pjvandehaar]

1. I uploaded a new release, 1.3.16, without blist. Can you try it? Unfortunately I still have a problem with MACOSX_DEPLOYMENT_TARGET so I didn't upload wheels for this release.

2. The master branch should always work, but I'm more careful with the versioned releases. You can see them in the git history, because they're named like "1.3.16". I upload them to PyPI (which is why pip3 install pheweb works), but not bioconda.

3. Would bioconda work better than PyPI? I don't know anything about it. I can add that to my release script. Should I?

4. Would it be helpful if I tagged releases using git? I don't see the benefit to make up for the slight amount of work, but I'm probably just missing it.

[abiddanda54gene]

Hi @pjvandehaar,

The newest version (1.3.16) worked great via pip install pheweb using a python environment with 3.9.5 on Mac OSX :) So no complaints there.

However, within a conda environment on Linux there are still some problems with finding the cffi files (much more cryptic). Is this able to work on servers at UM quite well?

name: pheweb
channels:
  - conda-forge
dependencies:
  - python=3.9.5

and then:

conda env crate -f pheweb.yaml
conda activate pheweb
pip install pheweb --no-cache --verbose 2>&1 | tee pheweb_install_Linux.log

pheweb_install_Linux.log

I think that the eventual gcc command expects build/temp.linux-x86_64-3.9/pheweb.load.cffi._x.cpp to be around, but this is only ever copied in as copying pheweb/load/cffi/x.cpp -> build/lib.linux-x86_64-3.9/pheweb/load/cffi.

Regarding point 4 - I definitely think having the tagged releases on github would be very helpful (although one can also version lock on pip as well). It would be useful on github to have to download source-code that is version locked as a tarball and trying to perform a local pip install.

[abiddanda54gene]

Hi @pjvandehaar,

I've recently solved this on linux for an installation via conda that worked great! The yaml was:

name: pheweb
channels:
  - conda-forge
dependencies:
  - python
  - gxx_linux-64>9
  - pip
  - pip:
    - pheweb

and then:

conda env create -f pheweb.yaml
conda activate pheweb
pheweb -help

This worked well for me (and was system agnostic as well so could work as a general solution on linux as well). Feel free to include this in recommendations to others as well!

[abiddanda54gene]

Closing this issue as its not active any more and there is a solution. Thanks very much!