Final report not working

[Jmontoyamm]

Final report not working

[Dangertrip]

Hi,

Thank you for using LiBis as one of your options. Would you please paste the error message here?

Besides that, do you find any figures or HTML files in the RESULT folder?

[Jmontoyamm]

The interesting thing is that the RESULT folder is completely empty despite getting all the bam files correctly.

Here is the code:

2020-11-22 13:19:54.979499 command: bsmap -a /home/groups/CEDAR/montoyam/sc-WGBS/new_pipeline/trimmed_reads/case_14_R1_val_1.fq.gz -z 33 -d /home/groups/CEDAR/montoyam/WGBS-BC/genome/GRCh38.fa -o case_14_R1_val_1.bam -S 123 -n 1 -r 0 -U -u -p 8 1>>LiBis_log 2>>BAM_FILE/case_14_R1_val_1_originallog.record
Output:

2020-11-22 14:12:55.403280 command: bsmap -a case_14_R1_val_1.unmapped.fastq -d /home/groups/CEDAR/montoyam/WGBS-BC/genome/GRCh38.fa -o case_14_R1_val_1.unmapped.bam -n 1 -r 0 -U -R -p 8 1>>LiBis_log 2>>BAM_FILE/case_14_R1_val_1.unmapped_log.txt
Output:

2020-11-22 14:21:54.582573 command: samtools view -b -F 4 -@ 6 case_14_R1_val_1.bam > case_14_R1_val_1.filter.bam
Output:

2020-11-22 14:22:45.194401 command: samtools merge -f -@ 6 BAM_FILE/case_14_R1_val_1_combine.bam case_14_R1_val_1.bam case_14_R1_val_1_split.bam
Output:

2020-11-22 14:22:45.293876 command: mv case_14_R1_val_1.bam BAM_FILE/
Output:

2020-11-22 14:22:45.341107 command: mv case_14_R1_val_1_split.bam BAM_FILE/
Output:

2020-11-22 15:27:17.030359 command: bsmap -a /home/groups/CEDAR/montoyam/sc-WGBS/new_pipeline/trimmed_reads/case_14_R2_val_2.fq.gz -z 33 -d /home/groups/CEDAR/montoyam/WGBS-BC/genome/GRCh38.fa -o case_14_R2_val_2.bam -S 123 -n 1 -r 0 -U -u -p 8 1>>LiBis_log 2>>BAM_FILE/case_14_R2_val_2_originallog.record
Output:

2020-11-22 16:28:01.805575 command: bsmap -a case_14_R2_val_2.unmapped.fastq -d /home/groups/CEDAR/montoyam/WGBS-BC/genome/GRCh38.fa -o case_14_R2_val_2.unmapped.bam -n 1 -r 0 -U -R -p 8 1>>LiBis_log 2>>BAM_FILE/case_14_R2_val_2.unmapped_log.txt
Output:

2020-11-22 16:36:38.619305 command: samtools view -b -F 4 -@ 6 case_14_R2_val_2.bam > case_14_R2_val_2.filter.bam
Output:

2020-11-22 16:37:30.415537 command: samtools merge -f -@ 6 BAM_FILE/case_14_R2_val_2_combine.bam case_14_R2_val_2.bam case_14_R2_val_2_split.bam
Output:

2020-11-22 16:37:30.489801 command: mv case_14_R2_val_2.bam BAM_FILE/
Output:

2020-11-22 16:37:30.543002 command: mv case_14_R2_val_2_split.bam BAM_FILE/
Output:

[E::idx_find_and_load] Could not retrieve index file for 'case_14_R1_val_1.bam'
[E::idx_find_and_load] Could not retrieve index file for 'case_14_R1_val_1.unmapped.bam'
[E::idx_find_and_load] Could not retrieve index file for 'case_14_R2_val_2.bam'
[E::idx_find_and_load] Could not retrieve index file for 'case_14_R2_val_2.unmapped.bam'
[ 2020-11-22 12:23:22.367748 ] Running Alignment for samples...
[ 2020-11-22 12:23:22.415179 ] Temprary files cleaning...
[ 2020-11-22 12:23:22.415253 ] Begin the alignment process with clipping.
[ 2020-11-22 12:23:22.415374 ] Begin the 1st round full reads mapping.
[ 2020-11-22 13:19:54.987296 ] Filtering and clipping unmapped reads.
Temporary file doesn't exist: case_14_R1_val_1.unmapped.fastq !
[ 2020-11-22 13:23:35.741953 ] Begin the 2st round mapping for clipped fragments.
[ 2020-11-22 14:12:55.418405 ] Recombining uniquely mapped fragments...
[ 2020-11-22 14:21:16.832066 ] BAM file sorting...
Removing temporary file: case_14_R1_val_1.filter.bam ......
Removing temporary file: case_14_R1_val_1.bam ......
[ 2020-11-22 14:22:45.353635 ] FINISH!
Temporary file doesn't exist: case_14_R1_val_1.sort.bam !
Temporary file doesn't exist: case_14_R1_val_1.bam.bai !
Removing temporary file: case_14_R1_val_1.unmapped.fastq ......
Removing temporary file: case_14_R1_val_1.unmapped.bam ......
Temporary file doesn't exist: case_14_R1_val_1.unmapped.bam.bai !
[ 2020-11-22 14:22:46.935293 ] Temprary files cleaning...
[ 2020-11-22 14:22:46.935511 ] Begin the alignment process with clipping.
[ 2020-11-22 14:22:46.935641 ] Begin the 1st round full reads mapping.
[ 2020-11-22 15:27:17.044857 ] Filtering and clipping unmapped reads.
Temporary file doesn't exist: case_14_R2_val_2.unmapped.fastq !
[ 2020-11-22 15:31:02.698103 ] Begin the 2st round mapping for clipped fragments.
[ 2020-11-22 16:28:01.819906 ] Recombining uniquely mapped fragments...
[ 2020-11-22 16:36:02.450165 ] BAM file sorting...
Removing temporary file: case_14_R2_val_2.filter.bam ......
Removing temporary file: case_14_R2_val_2.bam ......
[ 2020-11-22 16:37:30.551089 ] FINISH!
Temporary file doesn't exist: case_14_R2_val_2.sort.bam !
Temporary file doesn't exist: case_14_R2_val_2.bam.bai !
Removing temporary file: case_14_R2_val_2.unmapped.fastq ......
Removing temporary file: case_14_R2_val_2.unmapped.bam ......
Temporary file doesn't exist: case_14_R2_val_2.unmapped.bam.bai !
[ 2020-11-22 16:37:31.736195 ] Finish Alignment for samples

[Dangertrip]

Hi! Please specify -g and -mcall in your command to get the methylation ratio of CpGs. Then figure and table files will show in the RESULT folder.

Example:
LiBis -n sample1_mate1.fq.gz sample1_mate2.fq.gz -r hg19.fa --plot -g hg19 -mcall

In your case, you can substitute hg19 to hg38.

Besides that, due to the browser update, CORS(cross-origin resource sharing) policy changed in firefox which leads to a failure of showing the summary table in the final report. But you can still find the summary data in datatable.txt. I will try to fix this asap.

[Jmontoyamm]

Hi,

Is there a way to be able to get the report table without -mcall if I don't need methylation calls?

Thanks!

[Jmontoyamm]

Hi again, I tried what you suggested and now gives me a different error regarding a bam file

LiBis -n 52_R1_val_1.fq.gz case_52_R2_val_2.fq.gz -c 1 -r GRCh38.fa -w 30 -s 5 -ft 45 --plot -g hg38 -mcall

[E::idx_find_and_load] Could not retrieve index file for 'case_14_R1_val_1.bam'
[E::idx_find_and_load] Could not retrieve index file for 'case_14_R1_val_1.unmapped.bam'
[E::idx_find_and_load] Could not retrieve index file for 'case_14_R2_val_2.bam'
[E::idx_find_and_load] Could not retrieve index file for 'case_14_R2_val_2.unmapped.bam'
[ 2020-11-23 16:06:12.336524 ] Running Alignment for samples...
[ 2020-11-23 16:06:12.488068 ] Temprary files cleaning...
[ 2020-11-23 16:06:12.488117 ] Begin the alignment process with clipping.
[ 2020-11-23 16:06:12.488177 ] Begin the 1st round full reads mapping.
[ 2020-11-23 17:07:21.073447 ] Filtering and clipping unmapped reads.
Temporary file doesn't exist: case_14_R1_val_1.unmapped.fastq !
[ 2020-11-23 17:10:13.600202 ] Begin the 2st round mapping for clipped fragments.
[ 2020-11-23 18:02:51.366916 ] Recombining uniquely mapped fragments...
[ 2020-11-23 18:09:19.705052 ] BAM file sorting...
Removing temporary file: case_14_R1_val_1.filter.bam ......
Removing temporary file: case_14_R1_val_1.bam ......
[ 2020-11-23 18:10:51.132918 ] FINISH!
Temporary file doesn't exist: case_14_R1_val_1.sort.bam !
Temporary file doesn't exist: case_14_R1_val_1.bam.bai !
Removing temporary file: case_14_R1_val_1.unmapped.fastq ......
Removing temporary file: case_14_R1_val_1.unmapped.bam ......
Temporary file doesn't exist: case_14_R1_val_1.unmapped.bam.bai !
[ 2020-11-23 18:10:53.125989 ] Temprary files cleaning...
[ 2020-11-23 18:10:53.126040 ] Begin the alignment process with clipping.
[ 2020-11-23 18:10:53.126089 ] Begin the 1st round full reads mapping.
[ 2020-11-23 19:23:45.610494 ] Filtering and clipping unmapped reads.
Temporary file doesn't exist: case_14_R2_val_2.unmapped.fastq !
[ 2020-11-23 19:27:20.423778 ] Begin the 2st round mapping for clipped fragments.
[ 2020-11-23 20:28:56.967063 ] Recombining uniquely mapped fragments...
[ 2020-11-23 20:33:41.488045 ] BAM file sorting...
Removing temporary file: case_14_R2_val_2.filter.bam ......
Removing temporary file: case_14_R2_val_2.bam ......
[ 2020-11-23 20:35:10.015594 ] FINISH!
Temporary file doesn't exist: case_14_R2_val_2.sort.bam !
Temporary file doesn't exist: case_14_R2_val_2.bam.bai !
Removing temporary file: case_14_R2_val_2.unmapped.fastq ......
Removing temporary file: case_14_R2_val_2.unmapped.bam ......
Temporary file doesn't exist: case_14_R2_val_2.unmapped.bam.bai !
[ 2020-11-23 20:35:11.942866 ] Finish Alignment for samples
[ 2020-11-23 20:35:11.942914 ] Running Methylation Calling for samples...
Traceback (most recent call last):
File "/home/users/montoyam/miniconda3/bin/LiBis", line 215, in
main()
File "/home/users/montoyam/miniconda3/bin/LiBis", line 212, in main
computeProcess(input_args())
File "/home/users/montoyam/miniconda3/lib/python3.7/site-packages/LiBis/computeProcess.py", line 147, in computeProcess
name,mcallresult = computeMcall(name,mcall,param)
File "/home/users/montoyam/miniconda3/lib/python3.7/site-packages/LiBis/computeProcess.py", line 99, in computeMcall
bn,ln=mcall.run(n)
File "/home/users/montoyam/miniconda3/lib/python3.7/site-packages/LiBis/moabs.py", line 45, in run
os.rename(name+g,self.path+name[name.rfind('/')+1:]+g)
FileNotFoundError: [Errno 2] No such file or directory: 'BAM_FILE/case_14_R1_val_1_combine.bam.G.bed' -> './BED_FILE/case_14_R1_val_1_combine.bam.G.bed'

Thanks

[Dangertrip]

Hi!
It seems that you don't have mcall correctly installed.
Besides that, I moved the result file generation to an earlier spot so you don't have to specify -mcall, -g and -plot to get the summary file. Here's my command:
LiBis-n sample1_mate1.fq.gz sample1_mate2.fq.gz -r hg19.fa
The summary file will locate at RESULT/datatable.txt

I've updated the new version to PyPI and conda. However, it may take days for conda to update to this new version. You can install the 0.1.5.8(new version) by PyPI if you want to test it now:
pip install LiBis --upgrade

BTW, we recommend users to use -w 40 -s 5 -ft 45 to reduce the false positive results.

Please let me know if you have further questions.