90% chimeric ambiguous reads on more than 10 experiments using standard enzymes & syntax.

[aidenlab]

90% chimeric ambiguous reads on more than 10 experiments using standard enzymes & syntax.

[nchernia]

There is a known problem depending on the read name. Could you send the first few lines of the fastqs (R1 and R2)?

On Wed, Oct 25, 2017 at 7:16 PM jamesdalgleishnih ***@***.***> wrote: Hello! The title says it all. Is there any way to discover why these reads are registering as chimeric ambiguous? None of the reference sets tend to have such odd stats. I have substituted the names of our conditions and genes in order to protect our ability to publish the results. Here is the syntax used to run juicer: module load juicer cd /scratch/Experiment1/ juicer.sh -p $JUICER/references/hg19.chrom.sizes -s HindIII -y /usr/local/apps/juicer/juicer-1.5/SLURM/restriction_sites/hg19_HindIII.txt Each folder has two fastq files and they are paired with the _R1.fastq.gz extension. -bash-4.1$ head .hic -n 14 HI69/usr/local/apps/juicer/juicer-1.5/SLURM//references/hg19.chrom.sizesstatisticsExperiment description: Sequenced Read Pairs: 51,225,101 Normal Paired: 4,837,169 (9.44%) Chimeric Paired: 0 (0.00%) Chimeric Ambiguous: 46,387,931 (90.56%) Unmapped: 0 (0.00%) Ligation Motif Present: 17,626,936 (34.41%) Alignable (Normal+Chimeric Paired): 4,837,169 (9.44%) Unique Reads: 4,371,746 (8.53%) PCR Duplicates: 460,436 (0.90%) Optical Duplicates: 4,987 (0.01%) Library Complexity Estimate: 23,718,686 Intra-fragment Reads: 41,555 (0.08% / 0.95%) Below MAPQ Threshold: 832,482 (1.63% / 19.04%) -bash-4.1$ head .hic -n 14 HI /usr/local/apps/juicer/juicer-1.5/SLURM//references/hg19.chrom.sizesstatisticsExperiment description: Sequenced Read Pairs: 53,112,216 Normal Paired: 4,531,213 (8.53%) Chimeric Paired: 0 (0.00%) Chimeric Ambiguous: 48,581,002 (91.47%) Unmapped: 0 (0.00%) Ligation Motif Present: 15,353,175 (28.91%) Alignable (Normal+Chimeric Paired): 4,531,213 (8.53%) Unique Reads: 4,165,313 (7.84%) PCR Duplicates: 361,219 (0.68%) Optical Duplicates: 4,681 (0.01%) Library Complexity Estimate: 26,831,778 Intra-fragment Reads: 51,098 (0.10% / 1.23%) Below MAPQ Threshold: 821,990 (1.55% / 19.73%) -bash-4.1$ head .hic -n 14 HIn▒/usr/local/apps/juicer/juicer-1.5/SLURM//references/hg19.chrom.sizesstatisticsExperiment description: Sequenced Read Pairs: 70,885,255 Normal Paired: 4,735,332 (6.68%) Chimeric Paired: 1 (0.00%) Chimeric Ambiguous: 66,149,921 (93.32%) Unmapped: 0 (0.00%) Ligation Motif Present: 16,650,157 (23.49%) Alignable (Normal+Chimeric Paired): 4,735,333 (6.68%) Unique Reads: 4,391,686 (6.20%) PCR Duplicates: 338,623 (0.48%) Optical Duplicates: 5,024 (0.01%) Library Complexity Estimate: 31,443,095 Intra-fragment Reads: 50,678 (0.07% / 1.15%) Below MAPQ Threshold: 807,014 (1.14% / 18.38%) Thanks, James D — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#43>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABWiW8mAMqznsQFvW7tNwbnEqtgwKYu4ks5sv8FGgaJpZM4QGzAY> .

-- Neva Cherniavsky Durand, Ph.D. Staff Scientist, Aiden Lab www.aidenlab.org

[jamesdalg]

Here is a couple of them, with their fastq file pairs.
-bash-4.1$ gzip -cd X_1_AHJNHNAFXX.1009_NEXTSEQ-2017-04-25.fq.gz | head -n 30
@NS500348:170:HJNHNAFXX:1:11101:16680:1036 1:N:0:1
CTCTANAGAGAAGCATTCTCAGAAGCTTCATTGGGATGTTTCAATTGAAGTCACAGTGTTGAACAGTCCCTTTCA
+
AAAAA#EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAAEEE
@NS500348:170:HJNHNAFXX:1:11101:13757:1036 1:N:0:1
TCCCTNGCTTGTTTAAGCAAAATCAACCTTGTGCCATTCTGCACGACAACAGCTCTGCCCAGCAGTCACCAATCA
+
AAAAA#EEEEEEEEEEEEEEEEEEEEEAEAEEEEEEEEAEEEEEEEEAEEAEEEEEA<EEEEEEEEEEEEEEAEE
@NS500348:170:HJNHNAFXX:1:11101:5644:1036 1:N:0:1
CCTCANGACGTAGCACCCTCCCAGCATATTGGTGTCTATTGGGTTAGCTCACCTAAGCTTTGAGTATCCAGTTTT
+
AAAAA#EEEEEEEEAEEEEEEEEEEEEEEEEEEEAEEEEEEEEEEEEEEEEEEEEEEEEAEEEEEEEE<EEEEEE
@NS500348:170:HJNHNAFXX:1:11101:16309:1036 1:N:0:1
ATAACNCTCCTTGCACACAAATCTCCAGCTCAGAGTTTTGTTCCCAGGGAAGTCATCCTATGACACCTGCTATCA
+
AAAAA#EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAEEEEAEEEE6<EEAEEEAAEEEEEAAEEEE
@NS500348:170:HJNHNAFXX:1:11101:15987:1036 1:N:0:1
GAGTANGTGACTTTAATTTTTCAGCTGTGCCAACAGAAAAAGTACTTCAACTAAGAATTTTTAAAAATTATTTCA
+
AAAAA#EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE
@NS500348:170:HJNHNAFXX:1:11101:12977:1036 1:N:0:1
GCGAANTGTAATGAGCACAAGAGCAAAGCAGATGTTTTAAAGAAACACTTTATAAACCTTTCCTCAACTTATCTC
+
AAAAA#EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE6EEEEEEEEEEEEEEEEEEEEEAEEEEEEEA
@NS500348:170:HJNHNAFXX:1:11101:16344:1036 1:N:0:1
TTAATNAATAATATGTCTCTGTTTGCTTCGGTAGCTCTTCTGTCTCTTATACACATCTCCGAGCCCACGAGACCG
+
AAAAA#EEEAEAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE6EAEEEEEAAAEE</EEEEEE/AE
@NS500348:170:HJNHNAFXX:1:11101:19625:1036 1:N:0:1
TAGCANACACGAGAGAGATGTGAACTATGGAAAGGCCAGCAGGGACCATGGACTGTCTCTTATACACATCTCCGA
-bash-4.1$ gzip -cd X_2_AHJNHNAFXX.1009_NEXTSEQ-2017-04-25.fq.gz | head -n 30
@NS500348:170:HJNHNAFXX:1:11101:16680:1036 2:N:0:1
NNNNNNNNNGNGNNCNNNANNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
+
#########E#E##E###E########################################################
@NS500348:170:HJNHNAFXX:1:11101:13757:1036 2:N:0:1
NNNNNNNNNGNCNNCNNNANNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
+
#########E#A##E###/########################################################
@NS500348:170:HJNHNAFXX:1:11101:5644:1036 2:N:0:1
NNNNNNNNNTNTNNANNNANNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
+
#########E#E##E###E########################################################
@NS500348:170:HJNHNAFXX:1:11101:16309:1036 2:N:0:1
NNNNNNNNNCNANNCNNNTNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
+
#########E#E##E###E########################################################
@NS500348:170:HJNHNAFXX:1:11101:15987:1036 2:N:0:1
NNNNNNNNNTNCNNGNNNTNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
+
#########E#E##E###E########################################################
@NS500348:170:HJNHNAFXX:1:11101:12977:1036 2:N:0:1
NNNNNNNNNANANNANNNCNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
+
#########E#E##E###E########################################################
@NS500348:170:HJNHNAFXX:1:11101:16344:1036 2:N:0:1
NNNNNNNNNCNANNCNNNCNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN
+
#########E#E##E###A########################################################
@NS500348:170:HJNHNAFXX:1:11101:19625:1036 2:N:0:1
NNNNNNNNNCNTNNTNNNCNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN

-bash-4.1$ gzip -cd X_1_AHJY2LAFXX.1005_NEXTSEQ-2017-04-18.fq.gz | head -n 30
@NB501402:27:HJY2LAFXX:1:11101:12561:1061 1:N:0:1
GNAATTGCCTGAAGGTATAGACTTAGAAATTTAACATTTAAAAACATTTTCTTCATTTTCTAAGCTAGCTTTTTT
+
A#AAAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAEEEAEEEEEE
@NB501402:27:HJY2LAFXX:1:11101:16276:1061 1:N:0:1
GNGTAAGACCACAAAAGCACAGGAAACAAAAGCAAAATAGACAAATGGTATTATATCATGCTAAAAAGCTAGCTT
+
A#AAAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAEEEEEEEEEEEE
@NB501402:27:HJY2LAFXX:1:11101:19495:1061 1:N:0:1
GNGTTCTAGGGCAAGCCAAATCTTTCCAGATCAACAATGACAACTAGATCCATTAAGACACTGGCCTGTTTAAGT
+
A#AAAEEEEEEEEEEEEEEEEEEEEEEEEEEEE/EAEEE/AEEE/EEEEAEEEEEAE/EAEE/A<E<E/EEEEEE
@NB501402:27:HJY2LAFXX:1:11101:17883:1061 1:N:0:1
CNCATAGACAGCTCCTGAATCAATGACTCCAGCCTGGCTCAAGCTAGCTTCCTCTGGCCTCCTCCCTCATTAAGG
+
A#AAAAEEEEEEEEEEEEEEEEEAEAEEEEAEEEEEEEEEEEEEEEEEEEEEEEEEE<EEEEAEEEAEE//EE/E
@NB501402:27:HJY2LAFXX:1:11101:5852:1061 1:N:0:1
GNAACCTTCTGTGTGCCAGAATGTCAGGATAAGGGGGTCACGGTCTCTTCTTTCTCTATCCCCTGTCTCTTATAC
+
A#AAAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAEEEEAEEEEEEEEEEEE
@NB501402:27:HJY2LAFXX:1:11101:9026:1061 1:N:0:1
GNGCAGGTAGAGGAGACCAGACAGTCCCAGGCTCAGTGGTAGAAGAGTCACCCAGGGCTACTCCAGCCCCTTCTC
+
A#AAAEEEEEAEEEEAEEEEEEEEEEEE/EEEEEEEEEEEEEEEEEEEE/EEE/EEEEAEEAEEAEEEEE<EEEA
@NB501402:27:HJY2LAFXX:1:11101:4690:1061 1:N:0:1
CNGTATCACTAGAGTTAAAATTATGAGCAACAGAAAACTGGTTCTCCATATCCTGGATGAGGGGCAGGGTTAGGG
+
A#AAAEEEEEEEEEEE6EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE/EEEEAEEEA
@NB501402:27:HJY2LAFXX:1:11101:10845:1061 1:N:0:1
CNATATGGAGGCAAACTTGAAAAACAAAAACTCAGTTTTGTTAAAATATGTGAGGAGGCAGCTTTAGGCCTGTCT
-bash-4.1$ gzip -cd X_2_AHJY2LAFXX.1005_NEXTSEQ-2017-04-18.fq.gz | head -n 30
@NB501402:27:HJY2LAFXX:1:11101:12561:1061 2:N:0:1
NNNNNNNAATTAAAATATATAAAAATTATTAAAAAATTAAAAAAAATTAAAATAATAATATAAGAAAAAATATTA
+
#######/E//////////////////A//A6///A//////E//////A/A//////////<//AA6A6/////
@NB501402:27:HJY2LAFXX:1:11101:16276:1061 2:N:0:1
NNNNNNNGGGTTTTGCTTATTTGAAAACTGTAAAAGGTACTCCAGTCTCCCCACATGGCAGCTTCAAGGCCCTGA
+
#######EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAEEEEEEEEEEEEEEEEEEEEEA
@NB501402:27:HJY2LAFXX:1:11101:19495:1061 2:N:0:1
NNNNNNNAAGGAACTCCTTTAGAAGGAACTCCTTGACTAGAACAACTTAAACAGGCCAGTGTCTTAATGGATCTA
+
#######EEEEEEEEEEEEEEEEEEEEEEEAEEEEEEEEAAEEEEEEEEEEEEEA<AEEEEEEE/EEE/EEEE<E
@NB501402:27:HJY2LAFXX:1:11101:17883:1061 2:N:0:1
NNNNNNNACCTGCAGCTCGTCCCTTCTCCTTAATGAGGGAGGAGGCCAGAGGAAGCTAGCTTGAGCCAGGCTGGA
+
#######EEEEEEEEEEEEEEEEEEE/EEE/EEEEEEEEEEEEEE<EE/EEEAEEEAEEEEEEAAE6AEAEEEE/
@NB501402:27:HJY2LAFXX:1:11101:5852:1061 2:N:0:1
NNNNNNNAGAAAGAAGAGACCGTGACCCCCTTATCCTGACATTCTGGCACACAGAAGGTTCCCTGTCTCTTATAC
+
#######AEEEEEEEEEEEEEEEE/EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAEA
@NB501402:27:HJY2LAFXX:1:11101:9026:1061 2:N:0:1
NNNNNNNCAGGGTGAGGCTGGGCAAGCGTGCTTCTGAGAAGGGGCTGGAGTAGCCCTGGGTGACTCTTCTACCAC
+
#######EEEEEEEEEEEEEEEEEAEEEEEEEEEEEEEEEEEEEEEEEEEEAEEAEEEEEEEAE<E/<E</EE/E
@NB501402:27:HJY2LAFXX:1:11101:4690:1061 2:N:0:1
NNNNNNNATATTAGTCAAGCTAGCTTATACCACCCATCCCTAACCCTGCCCCTCATCCAGGATATGGAGAACCAG
+
#######EEEEEEEEEE/EEEEEEEEEE/EEAEEEEEEEEE/6EEEEEEEEEE<E<EEEEEEAE/EEEE<//EAE
@NB501402:27:HJY2LAFXX:1:11101:10845:1061 2:N:0:1
NNNNNNNGCTGCCTCCTCACATATTTTAACAAAACTGAGTTTTTGTTTTTCAAGTTTGCCTCCATATAGCTGTCT

[jamesdalg]

What else can I give you that might help? What else can I look into to discover the cause of this issue?

[nchernia]

Hello - sorry for the late response. Juicer usually looks at the file name to align; read 1 should have “_R1” in the name and read 2 should have “_R2”. Your filenames don’t seem to be following this convention. You can either change the “read1str” in Juicer or rename your files so that all read 1s have “_R1” in the name. Here is the line from the top of the Juicer script: # fastq files should look like filename_R1.fastq and filename_R2.fastq # if your fastq files look different, change this value read1str="_R1" read2str="_R2" Best Neva ​

…

On Wed, Nov 1, 2017 at 10:12 AM, jamesdalg ***@***.***> wrote: What else can I give you that might help? What else can I look into to discover the cause of this issue? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#43 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABWiWx3Opep39hAZHIlctMXWz_js-E9Cks5syHxIgaJpZM4QGzAY> .

-- Neva Cherniavsky Durand, Ph.D. Staff Scientist, Aiden Lab www.aidenlab.org

[jamesdalg]

I'll give that a go and rerun. I have been looking through documentation and it's not clear where precisely the chimeric ambiguous reads end up. If it is indeed something experimental that might cause this, it would be worthwhile to see examine the reads and verify that everything is spaced as we expect and with the content we expect. Is that in abnormal.sam or is it in collisions.txt?

[nchernia]

The high quality ones end up in collisions.txt but these are a subset of the ones in abnormal.sam But I am pretty sure it's due to Juicer not understanding how to connect read1 and read2.

…

On Wed, Nov 1, 2017 at 2:23 PM, jamesdalg ***@***.***> wrote: I'll give that a go and rerun. I have been looking through documentation and it's not clear where precisely the chimeric ambiguous reads end up. If it is indeed something experimental that might cause this, it would be worthwhile to see examine the reads and verify that everything is spaced as we expect and with the content we expect. Is that in abnormal.sam or is it in collisions.txt? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#43 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABWiW3iQaFPRWWwKbs7FFiqjzqsXWzv5ks5syLc6gaJpZM4QGzAY> .

-- Neva Cherniavsky Durand, Ph.D. Staff Scientist, Aiden Lab www.aidenlab.org

[jamesdalg]

Fair enough. One more question. I'm trying to compare these to "normal paired" reads. What are the meanings of the columns in merged_nodups.txt so that I know what I'm looking at?

[nchernia]

It's described here: https://github.com/theaidenlab/juicer/wiki/Pre#file-format The long format

…

On Wed, Nov 1, 2017 at 4:42 PM, jamesdalg ***@***.***> wrote: Fair enough. One more question. I'm trying to compare these to "normal paired" reads. What are the meanings of the columns in merged_nodups.txt so that I know what I'm looking at? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#43 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABWiWydttyqvm-cw0PSuHMIzFi2V4nKjks5syNepgaJpZM4QGzAY> .

-- Neva Cherniavsky Durand, Ph.D. Staff Scientist, Aiden Lab www.aidenlab.org

[jamesdalg]

Thanks! This fixed things:
Sequenced Read Pairs: 159,166,775
Normal Paired: 145,519,585 (91.43%)
Chimeric Paired: 8,484,360 (5.33%)
Chimeric Ambiguous: 1,934,090 (1.22%)
Unmapped: 3,228,740 (2.03%)
Ligation Motif Present: 31,202,559 (19.60%)
Alignable (Normal+Chimeric Paired): 154,003,945 (96.76%)
Intra-fragment Reads: 78,985,214 (49.62% / 69.37%)
Below MAPQ Threshold: 7,553,693 (4.75% / 6.63%)
Hi-C Contacts: 27,315,825 (17.16% / 23.99%)
Ligation Motif Present: 18,987,324 (11.93% / 16.68%)
3' Bias (Long Range): 96% - 4%
Pair Type %(L-I-O-R): 25% - 25% - 25% - 25%
Inter-chromosomal: 3,237,353 (2.03% / 2.84%)
Intra-chromosomal: 24,078,472 (15.13% / 21.15%)
Short Range (<20Kb): 12,927,319 (8.12% / 11.35%)
Long Range (>20Kb): 11,151,153 (7.01% / 9.79%)