read clustering error

[genomicsiter]

read clustering error

[hoohugokim]

I had the same issue under conda environment, in my case it seems to have stemmed from the version discrepancy between the ./conda_env/read_clustering/environment.yml file and the repository. I modified the version of hdbscan and umap-learn to the newest version found with conda search <package> and it is working fine now.

[bpenaud]

Hello,
I have exactly the same error do you resolve it ?
Regards,
Benjamin Penaud

[Thomieh73]

Hey, I tried the test run with conda and my run crashed at the same spot.

This is the error from the specific working directory:

UMAP(verbose=2)
Construct fuzzy simplicial set
Tue Feb 16 16:16:30 2021 Finding Nearest Neighbors
Tue Feb 16 16:16:33 2021 Finished Nearest Neighbor Search
Tue Feb 16 16:16:35 2021 Construct embedding
        completed  0  /  500 epochs
        completed  50  /  500 epochs
        completed  100  /  500 epochs
        completed  150  /  500 epochs
        completed  200  /  500 epochs
        completed  250  /  500 epochs
        completed  300  /  500 epochs
        completed  350  /  500 epochs
        completed  400  /  500 epochs
        completed  450  /  500 epochs
Tue Feb 16 16:16:42 2021 Finished embedding
Traceback (most recent call last):
  File "/cluster/work/users/thhaverk/nanoclust_tmp/fe/f4dc7167db2f6187bd0d5bf4ecc692/.command.sh", line 26, in <module>
    plt.figure(figsize=(20,20))
  File "/cluster/work/users/thhaverk/nanoclust_tmp/conda/read_clustering-800e1e27475cbaa0538f834c4aacc420/lib/python3.8/site-packages/matplotlib/pyplot.py", line 671, in figure
    figManager = new_figure_manager(num, figsize=figsize,
  File "/cluster/work/users/thhaverk/nanoclust_tmp/conda/read_clustering-800e1e27475cbaa0538f834c4aacc420/lib/python3.8/site-packages/matplotlib/pyplot.py", line 299, in new_figure_manager
    return _backend_mod.new_figure_manager(*args, **kwargs)
  File "/cluster/work/users/thhaverk/nanoclust_tmp/conda/read_clustering-800e1e27475cbaa0538f834c4aacc420/lib/python3.8/site-packages/matplotlib/backend_bases.py", line 3494, in new_figure_manager
    return cls.new_figure_manager_given_figure(num, fig)
  File "/cluster/work/users/thhaverk/nanoclust_tmp/conda/read_clustering-800e1e27475cbaa0538f834c4aacc420/lib/python3.8/site-packages/matplotlib/backends/_backend_tk.py", line 868, in new_figure_manager_given_figure
    window = tk.Tk(className="matplotlib")
  File "/cluster/work/users/thhaverk/nanoclust_tmp/conda/read_clustering-800e1e27475cbaa0538f834c4aacc420/lib/python3.8/tkinter/__init__.py", line 2261, in __init__
    self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
_tkinter.TclError: couldn't connect to display "158.36.42.36:25.0"

Any idea how to solve it?

[Thomieh73]

Okay, that did not work for me.
Can you explain why you found that that package was needed?

When I check the error, I see this:

Command error:
  Traceback (most recent call last):
    File ".command.sh", line 26, in <module>
      plt.figure(figsize=(20,20))
    File "/cluster/work/users/thhaverk/nanoclust_tmp/conda/read_clustering-165c04fe82a861f4b9dc6382a66f5ed7/lib/python3.8/site-packages/matplotlib/pyplot.py", line 671, in figure
      figManager = new_figure_manager(num, figsize=figsize,
    File "/cluster/work/users/thhaverk/nanoclust_tmp/conda/read_clustering-165c04fe82a861f4b9dc6382a66f5ed7/lib/python3.8/site-packages/matplotlib/pyplot.py", line 299, in new_figure_manager
      return _backend_mod.new_figure_manager(*args, **kwargs)
    File "/cluster/work/users/thhaverk/nanoclust_tmp/conda/read_clustering-165c04fe82a861f4b9dc6382a66f5ed7/lib/python3.8/site-packages/matplotlib/backend_bases.py", line 3494, in new_figure_manager
      return cls.new_figure_manager_given_figure(num, fig)
    File "/cluster/work/users/thhaverk/nanoclust_tmp/conda/read_clustering-165c04fe82a861f4b9dc6382a66f5ed7/lib/python3.8/site-packages/matplotlib/backends/_backend_tk.py", line 868, in new_figure_manager_given_figure
      window = tk.Tk(className="matplotlib")
    File "/cluster/work/users/thhaverk/nanoclust_tmp/conda/read_clustering-165c04fe82a861f4b9dc6382a66f5ed7/lib/python3.8/tkinter/__init__.py", line 2261, in __init__
      self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use)
  _tkinter.TclError: couldn't connect to display "158.36.42.36:25.0"

especially the last line, which is an IP address of a display, Why is that needed? I am working on a HPC cluster, so no other display arround then my terminal.

I will check the docker option

[Thomieh73]

Okay, I solved my issues by modifying the nextflow.config file to use singularity instead of docker. I added a singularity process to the processes part (see below). I work on a HPC cluster where we are not allowed to use docker. But I can use singularity with docker images.

This is the modified nextflow.config file for me:

profiles {
  test { includeConfig 'conf/test.config' }
  conda {
    process {
      withName: demultiplex { conda = "$baseDir/conda_envs/demultiplex/environment.yml" }
      withName: demultiplex_porechop { conda = "$baseDir/conda_envs/demultiplex_porechop/environment.yml" }
      withName: QC { conda = "$baseDir/conda_envs/qc_fastp/environment.yml" }
      withName: fastqc { conda = "$baseDir/conda_envs/fastqc/environment.yml" }
      withName: multiqc { conda = "$baseDir/conda_envs/fastqc/environment.yml" }
      withName: kmer_freqs { conda = "$baseDir/conda_envs/kmer_freqs/environment.yml" }
      withName: read_clustering { conda = "$baseDir/conda_envs/read_clustering/environment.yml" }
      withName: split_by_cluster { conda = "$baseDir/conda_envs/split_by_cluster/environment.yml" }
      withName: read_correction { conda = "$baseDir/conda_envs/read_correction/environment.yml" }
      withName: draft_selection { conda = "$baseDir/conda_envs/draft_selection/environment.yml" }
      withName: racon_pass { conda = "$baseDir/conda_envs/racon_pass/environment.yml" }
      withName: medaka_pass { conda = "$baseDir/conda_envs/medaka_pass/environment.yml" }
      withName: consensus_classification { conda = "$baseDir/conda_envs/consensus_classification/environment.yml" }
      withName: get_abundances { conda = "$baseDir/conda_envs/cluster_plot_pool/environment.yml" }
      withName: plot_abundances { conda = "$baseDir/conda_envs/cluster_plot_pool/environment.yml" }
      withName: output_documentation { conda = "$baseDir/conda_envs/output_documentation/environment.yml" }
    }
  }
  docker {
    docker.enabled = true
    //process.container = 'nf-core/nanoclust:latest'
    process {
      withName: demultiplex { container = 'hecrp/nanoclust-demultiplex' }
      withName: demultiplex_porechop { container = 'hecrp/nanoclust-demultiplex_porechop' }
      withName: QC { container = 'hecrp/nanoclust-qc' }
      withName: fastqc { container = 'hecrp/nanoclust-fastqc' }
      withName: multiqc { container = 'hecrp/nanoclust-fastqc' }
      withName: kmer_freqs { container = 'hecrp/nanoclust-kmer_freqs' }
      withName: read_clustering { container = 'hecrp/nanoclust-read_clustering' }
      withName: split_by_cluster { container = 'hecrp/nanoclust-split_by_cluster' }
      withName: read_correction { container = 'hecrp/nanoclust-read_correction' }
      withName: draft_selection { container = 'hecrp/nanoclust-draft_selection' }
      withName: racon_pass { container = 'hecrp/nanoclust-racon_pass' }
      withName: medaka_pass { container = 'hecrp/nanoclust-medaka_pass' }
      withName: consensus_classification { container = 'hecrp/nanoclust-consensus_classification'
                                           docker.temp = "$baseDir/" }
      withName: get_abundances { container = 'hecrp/nanoclust-plot_abundances' }
      withName: plot_abundances { container = 'hecrp/nanoclust-plot_abundances' }
      withName: output_documentation { container = 'hecrp/nanoclust-output_documentation' }
    }
    }
    singularity {
      singularity.enabled = true
      singularity.autoMounts = true
      //process.container = 'nf-core/nanoclust:latest'
      process {
        withName: demultiplex { container = 'docker://hecrp/nanoclust-demultiplex' }
        withName: demultiplex_porechop { container = 'docker://hecrp/nanoclust-demultiplex_porechop' }
        withName: QC { container = 'docker://hecrp/nanoclust-qc' }
        withName: fastqc { container = 'docker://hecrp/nanoclust-fastqc' }
        withName: multiqc { container = 'docker://hecrp/nanoclust-fastqc' }
        withName: kmer_freqs { container = 'docker://hecrp/nanoclust-kmer_freqs' }
        withName: read_clustering { container = 'docker://hecrp/nanoclust-read_clustering' }
        withName: split_by_cluster { container = 'docker://hecrp/nanoclust-split_by_cluster' }
        withName: read_correction { container = 'docker://hecrp/nanoclust-read_correction' }
        withName: draft_selection { container = 'docker://hecrp/nanoclust-draft_selection' }
        withName: racon_pass { container = 'docker://hecrp/nanoclust-racon_pass' }
        withName: medaka_pass { container = 'docker://hecrp/nanoclust-medaka_pass' }
        withName: consensus_classification { container = 'docker://hecrp/nanoclust-consensus_classification'
                                             singularity.temp = "$baseDir/" }
        withName: get_abundances { container = 'docker://hecrp/nanoclust-plot_abundances' }
        withName: plot_abundances { container = 'docker://hecrp/nanoclust-plot_abundances' }
        withName: output_documentation { container = 'docker://hecrp/nanoclust-output_documentation' }
      }
      }
}

[timyerg]

@hoohugokim
Thank you for your comment. In my case worked removing all package versions specified. Finally got through that step.