Error: ModuleNotFoundError: No module named 'pandas._libs.interval'

[google]

Error: ModuleNotFoundError: No module named 'pandas._libs.interval'

[danielecook]

@gevro thank you for reporting. Can you share the command you are running?

[gevro]

singularity run -W /data -B /scratch/projects/bin/deepconsensus/model:/model -B `pwd` /scratch/projects/bin/deepconsensus/deepconsensus_1.0.0.sif deepconsensus run --batch_size=1024 --batch_zmws=100 --cpus 4 --max_passes 20 --subreads_to_ccs=subreads_to_ccs.bam --ccs_bam=ccs.bam --checkpoint=/model/checkpoint --output=output.deepconsensus.fastq

[danielecook]

@gevro please try running singularity with a clean environment. I think passing --cleanenv should work.

singularity run \
  -W /data \
  -B /scratch/projects/bin/deepconsensus/model:/model \
  -B `pwd` \
  --cleanenv \
  /scratch/projects/bin/deepconsensus/deepconsensus_1.0.0.sif \
  deepconsensus run \
    --batch_size=1024 \
    --batch_zmws=100 \
    --cpus 4 \
    --max_passes 20 \
    --subreads_to_ccs=subreads_to_ccs.bam \
    --ccs_bam=ccs.bam \
    --checkpoint=/model/checkpoint \
    --output=output.deepconsensus.fastq

For whatever reason, it appears you are using a shared version of the python library from your machine/HPC:

/share/apps/python/3.8.6/intel/lib/python3.8/site-packages/pandas/__init__.py

And this appears to be breaking things.

[gevro]

Now I'm getting this error:

2022-10-11 17:08:33.096036: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F AVX512_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
FATAL Flags parsing error: Unknown command line flag 'max_passes'
Pass --helpshort or --helpfull to see help on flags.

[danielecook]

We removed the --max_passes flag in this release because we have not provided any models with alternative numbers of passes. Please try removing this flag:

singularity run \
  -W /data \
  -B /scratch/projects/bin/deepconsensus/model:/model \
  -B `pwd` \
  --cleanenv \
  /scratch/projects/bin/deepconsensus/deepconsensus_1.0.0.sif \
  deepconsensus run \
    --batch_size=1024 \
    --batch_zmws=100 \
    --cpus 4 \
    --subreads_to_ccs=subreads_to_ccs.bam \
    --ccs_bam=ccs.bam \
    --checkpoint=/model/checkpoint \
    --output=output.deepconsensus.fastq

Also - the --max_passes, and --example_width flags are now dictated by the model params.json - so there is no need to set these flags.

[gevro]

Ok working now. Thanks.