Failure to read reference stats from index

[biogo]

Failure to read reference stats from index

[kortschak]

Thanks. I've just looked at the code and the spec and it seems to me that we are doing the right thing, and further that if the stats are missing, they are missing because samtools did not right them. The index reader reads them if they exist and the stats accessor returns them if they are not nil. I'm not sure there is anything to do here; even documenting it would just be "they aren't returned if they don't exist".

[kortschak]

Can you post a reproducer?

[bsipos]

Okay, I think I figured it out: the statistics are deemed invalid if are no reads map to a reference. I guess this is the intended behavior so no fix is needed. Maybe a hint in the documentation would help.

In any case, here is the code to reproduce:

package main

import (
        "bufio"
        "fmt"
        "github.com/biogo/hts/bam"
        "os"
)

func main() {
        var err error
        file := "test_sorted.bam"
        idx := file + ".bai"
        _, err = os.Stat(idx)
        if err != nil {
                fmt.Fprintf(os.Stderr, "Index file not found for file %s! Please run samtools index on the sorted file!\n", file)
                os.Exit(1)
        }
        fh, err := os.Open(idx)
        if err != nil {
                panic(err)
        }
        i, err := bam.ReadIndex(bufio.NewReader(fh))
        if err != nil {
                panic(err)
        }
        fmt.Println(i.NumRefs())
        stats, ok := i.ReferenceStats(0)
        if !ok {
                panic("Failed to read reference stats!")
        }
        fmt.Println(stats)
        stats, ok = i.ReferenceStats(1)
        if !ok {
                panic("Failed to read reference stats!")
        }
        fmt.Println(stats)
}

Input as SAM:

@HD	VN:1.6	SO:coordinate
@SQ	SN:SIRV101	LN:1561
@SQ	SN:SIRV106	LN:973
@SQ	SN:SIRV103	LN:1363
@SQ	SN:SIRV102	LN:1300
@SQ	SN:SIRV105	LN:670
@SQ	SN:SIRV108	LN:702
@SQ	SN:SIRV107	LN:744
@SQ	SN:SIRV109	LN:464
@SQ	SN:SIRV201	LN:2051
@SQ	SN:SIRV202	LN:1971
@SQ	SN:SIRV205	LN:523
@SQ	SN:SIRV204	LN:740
@SQ	SN:SIRV203	LN:686
@SQ	SN:SIRV206	LN:424
@SQ	SN:SIRV308	LN:479
@SQ	SN:SIRV306	LN:2373
@SQ	SN:SIRV301	LN:2467
@SQ	SN:SIRV302	LN:1807
@SQ	SN:SIRV303	LN:2018
@SQ	SN:SIRV304	LN:1083
@SQ	SN:SIRV307	LN:779
@SQ	SN:SIRV305	LN:436
@SQ	SN:SIRV311	LN:161
@SQ	SN:SIRV310	LN:589
@SQ	SN:SIRV309	LN:796
@SQ	SN:SIRV409	LN:1567
@SQ	SN:SIRV410	LN:950
@SQ	SN:SIRV406	LN:617
@SQ	SN:SIRV404	LN:592
@SQ	SN:SIRV403	LN:670
@SQ	SN:SIRV408	LN:570
@SQ	SN:SIRV405	LN:626
@SQ	SN:SIRV505	LN:2029
@SQ	SN:SIRV506	LN:552
@SQ	SN:SIRV511	LN:546
@SQ	SN:SIRV508	LN:2085
@SQ	SN:SIRV502	LN:1984
@SQ	SN:SIRV507	LN:533
@SQ	SN:SIRV510	LN:2474
@SQ	SN:SIRV501	LN:1890
@SQ	SN:SIRV512	LN:229
@SQ	SN:SIRV503	LN:526
@SQ	SN:SIRV509	LN:885
@SQ	SN:SIRV504	LN:2473
@SQ	SN:SIRV601	LN:1435
@SQ	SN:SIRV612	LN:1528
@SQ	SN:SIRV604	LN:1537
@SQ	SN:SIRV602	LN:574
@SQ	SN:SIRV607	LN:574
@SQ	SN:SIRV605	LN:1088
@SQ	SN:SIRV609	LN:485
@SQ	SN:SIRV611	LN:454
@SQ	SN:SIRV617	LN:276
@SQ	SN:SIRV606	LN:545
@SQ	SN:SIRV616	LN:531
@SQ	SN:SIRV618	LN:189
@SQ	SN:SIRV610	LN:1163
@SQ	SN:SIRV614	LN:459
@SQ	SN:SIRV608	LN:377
@SQ	SN:SIRV613	LN:1311
@SQ	SN:SIRV603	LN:1969
@SQ	SN:SIRV615	LN:783
@SQ	SN:SIRV702	LN:2247
@SQ	SN:SIRV703	LN:2498
@SQ	SN:SIRV701	LN:2462
@SQ	SN:SIRV705	LN:2462
@SQ	SN:SIRV704	LN:428
@SQ	SN:SIRV706	LN:949
@SQ	SN:SIRV708	LN:889
@PG	ID:minimap2	PN:minimap2	VN:2.17-r941	CL:minimap2 -t 30 -ax map-ont reference/sirv_transcriptome.fas.mmi -
d2623acc-c51f-4bb5-a8ab-7af65dce5296	0	SIRV101	1	60	106S11M1D12M1D7M2D6M1D16M1D5M1D22M1D15M2D9M1I4M1I19M1D8M3D39M1D4M1D30M1I16M2D2M1D31M3I3M3D6M2I17M1D5M2I6M2D77M1I22M1I10M1D2M1I3M3I36M1D15M3D1M1D24M1D17M2D37M1D37M1D7M1D36M1D37M2D9M4I5M1D15M2D16M1D2M1D4M5D7M3I19M1I9M2D6M1D32M1I26M4D6M1D6M1D5M3I26M1I4M1I46M1I6M2I27M2D6M1D61M1D49M1D25M2I9M1D11M1I1M2D3M1D26M1D11M1I17M1D2M1D4M1I3M2D27M1D11M3D38M3D89M1D25M2I12M1D5M2D16M2D14M2D17M2D2M3D13M2D12M3I8M2I19M72S	*	0	0	CGATCATTAGTTTATGGTTTTACTACTACCGTGACAAAAGTTGTCATTGTCTTTGTGTTTCTGTTGGTGCTGATATTGCTTTAGGATTATCGGGTTCAGATTTGGGGCTTGTCCGCGGTGTCAACCGGGTTTACAAGAGGGCTCCAACATCAGTGTATGGATGTTCAAACACCCCCAGAGACGGAACAATAGTGTGGTCATGCATTTAGAGAGGAATAAAAGCAACAACAACAGAGGTGTGAGCTGGGTACATGCTCTTACCCCACTCTCAGCACGTAGGGGCAGGTGTGGAAGACCAATAAGCCCGTCACCTCTGCCCAAATACAGCTATCGATTGCCAGGTTAGAATAGGCCCTACCACATCTGGCAAGAGAAACTTGGTCTGTTCCTCGTGTCAGTTCTTTTTGTCATTATACCGCCATCTCTCGTTGTGGTACAAGAGTTGGTTACTCTCAGTCCCATCCCCAACACATCACTGAGAAGTATGAGGGCCTAGTGGTCATTAGAGGAGTCAACATGGGTAAGGAATTGTTTAGGTGTGTCGGAGGGCAGAAACCAACGAAAGAAGGAAACGATCAACTTATAGTGTCCTGTCAGACCCAAATCTAGAAGACCTTAGGTTCCACATTAGTACTAAACAGCCTGTTGTATACCATGAGCCCCACTTCGATCTGTAAATGGTAATCCCTTGTAAAACTCATCTCTAAAACATTATACCAAGGTACAGCTGATGTGATCCCTCTACTAGTCTTCTCTGAGGAACCATGAGATGGATAAGGGGACCTACCCAAGTCGCCTGGGGTAACTTCCCATGTTGACATCACTCTGCGTGGGGACGGTAGTAGTCTGTGGGAAATCTACCCCTAGTGAAAAGGTAGAGAATGTCTGAACATTACCTGTAGACCCAGAGTACTCTCAGAACAACTACTCTCACAGAAAGGTTGGGTCTTGGCGATCAAGGCTTTTTACTGACTCTGCGGAGCGATGGAGTATCTGAACCAACACTACTATCAAAGATGTTCGACTAGACCTCGTGATGGTATAGGGCCTGGAAGTCAAAAGACTTCTCTGGGTATAGGTAGGTAGTTAGAGTACGGTCCCTGGGGTGTCTCAGAGTCGAACGTTCGGGGGAAGTTCTAAACATGTTAGGATGTTATGATAACAAGTCCAAACGTGGACGTCGTCATGTCACCTCTCGGAACACTAGCTGGCAGGTGGCTTAAAGCATTCCGCTTTGGTGGTCTGCAGGGGTGGCACATGCTGGGGTTCTACCTATAACAGGAAGGTGAAACCTCAATAATCTGAAAGGACTAGTTGAGATAGCCCAAATGAATCCGGAGGAGGAGAACTTGGAAAACGTTGTACGACGTCATTAATAATAACGTTCTTCCCGTTCGTGTCTATAAGGATATTCTTTCATTGCGAAGGTAAGCCCTTTTGGGTCTAGATACGGACTATACTCAAAAACCACAGACCTTCCACTTGGAGTTGTTGTTTAGGAGACAGGTAGTAGAAAACTTGTGGCACTACCAGTCGGTTCCACAGACTCTCCTTAGTATCAGATCGTCGGATTACGACGACATTGAACTCCCGACTTCGACGTAGAGGCATTACTTTAAAAAAACCAAAAAAAAAAAAAAAAAGAAGATAGAGCGACAGGCAGAGTCACAAAGACACCGACAGCTTTCT	$%)#$#"$$+#)-(%&'''%%$#$#$%&')%%%$%*,'%%#'&''$#&164@BC,>;A<+354::,17:=;=;36446%%9:%''&87))+*+<@5/%$-237;:BECF;://7''')9/'921?>55CBC?<?C>7.()/,,'2%&/322,;>A=>83550/31759(:9;9:80%&$%'(((%,=;<896<:.10,*%&&6:=>=/&'%2&&&2/*2&&$&$&)()&./11.+/894,)$0,'(&%),,-----,+))0012($23&5+,,-.58:@;.2*+3:>:2:8(-,+$$*/013,:54005$////,%%)--//+,89793,%*&$$&8?>::?=:9<;=5//+<;:-;8?1<60:.'*1-540$$'$''':==?A<96,4:5<>.'7:1A?<BBA=0&%0$&$%$9==:BE>9:&),66320*-2::E,EA7<35./682342(:6><=AC=;;@B@=<77<911,,.12&&2+33(((&%,-**&$3%39>777/-*&&'102023364.72*14.&&&&69:6*.4.4-14423,551++*)+223%))*238561&&%&$&2358(+$$$).))*))*091*1$*$,++55768.9'(2785555586D=(7/4353*+,0:=8;)*7'))-'0,-,0/.*+$%%%#....-13424:A=C>579=636?@?88@1080/-/496-8969*)$+-+&')$'.('%)$-7:<>9;75233--011,,/4,197=><<>?FA<6?692--,1)-+&',30112.%%&.-:;766*&&,,(+,36((&&''',&-+$)*$&%&'$")5652/<($+&%#%#%$(0@8,/31*)(569;84<8/-DED=?::://.15:++&+/.*-,1+3,1.000012,*''&)*%%%./115)*%('$'(*..800,/,),**),)).0$3)-(%#$%*58986-'-'%&$"&(%$$)*,+++,,+,-/643456.%%%$%..-00/)'%'*.57<BC;?<9C<??==;8:AE:6?;<?7:977312.%(3-4)&$$'('$$%-,*)%'//271-82>+++5110+&0*(./),/1/<<<=60282:=84?C?118=AC7=764786?@??9:33*6,099=B57(*:<:4:=B08099%128:;;2>59>.8:8332<878365''47:*25:7>.$.;;879*2/2;665862=45(33+866><97)#%+/91?==<7<2+6;@>,/'&')3%')%*3%$&%%3/1271-%%')))///09??880)$$$%#('%%8;?2.633-,677++))$$$(&.+84;5=?:89@>@@6<-''&%&'.0'#))6%%7;:59,2:/<389<=>A@16D94;=<:66:?;67<B>BB>EC;:1./.;@?<H@;=A;@:4?>>6?8878:=BAAB@@BHHGA9:9D:979A20247AB==FC@@>;;911182268=B4.><;A>:78LC5365:;;><@A?.6&'((*))043'%%7;<92;/&,%$$%+997+=/0-7,/0//4042(&&%$&((*8*.=<'1%'''8=@;0'/'+.(1(*'%%@::30611&++$$/)%%&&(#$%$%$'$+-/3$$$&.4566=:<&;9>>B@CFF8CDA:74087BC=68598:9;8;9;;;%%-,5**++:88>8;::.A%<753-3::7:9821,5/(-$+/14*(	NM:i:205	ms:i:1918	AS:i:1918	nn:i:0	tp:A:P	cm:i:84	s1:i:641	s2:i:540	de:f:0.0973	rl:i:0
0e057898-ac47-41fb-8ba2-c0a28660c9c2	0	SIRV101	1	60	7S6M2D22M1D26M1D5M1D8M1D69M1I23M1D44M1D45M5I3M3D56M6D12M2I10M1I16M1I4M1D17M1D1M4D50M2D26M5D16M1D13M1I4M2D96M1I90M1I14M1I6M1I5M3D30M2D26M1D45M1I93M1I47M2D7M1I35M2D4M1D3M1D37M1I6M1I8M1I23M1D8M1I14M1I77M2I3M2D32M1I25M2I6M1D19M1I1M1I10M1D82M2D16M1D1M2D15M1D25M1D5M1D23M2I18M3D5M1D5M1D11M1D2M1D37M69S	*	0	0	AGTTGGAGCTTGTGCGAGTGTCAACCGGGTTTTACAAGGAGAACTTCCAACATCGGTGTATGGATGTTCAAACACCCTAGAGACGGAGACAATAGTGTGGTCACGTACATTTAGGAAGATAAAAATAACAACAACAGAAGGTGCAGAGGCGCAAGGTGCATGCTCTTGCACTCTCTCAGCACGTAGGGAGCGGAGTGTGGAAGACCAACAACCCGTCGCCTCTACCAAATGCAGCTATCCTGAGGTGCCAGGTTAGGAGCAAATTTCTACCACATCTAAGCCTTGAAACGGTCTGTTCCTCGTGTCAAGTTCTTTGTCCGCCGCCATCTCTCAATTTGTGTTGCAAAGAGTTGGTTACTCTCCAGTCCATTCCCAACACATCATAGTATAGAGCCTAGTGGTCATTAGAAGGTCAACATGGAGTAGGTGTTTAGGTGTCGGAGGGCAGAAACCAACGAAAGAAACGATCAACACTTGCCAGTGTCCTGTCCGGGCTGTTCCTAAGAGACCTTGGGTTTACCACATTAGTGCTAAACAGCCTGTTGTATACCGTGAGCCCCCACTTCGATCTGTAAATGGTAATCCCTTGTAAAAACCTCATCTCCAAAAACAGTATACGCAGGTACAGCTGACGTGATCCTCTCTACTAGTCTTCTCTGAGGAACCATGAGATGGATAATGTGAGGACTAGAATCACTACTAGAGGTAACTTATATGGTGACATCACTCCTACCGTAGAGGGGACGGTAGTCTGTGGGAAATCTACCTAGTGAAGAAAGGTAAGAGAATGTCTGAACATCACCTGTAGACAGAAGGTACTCCCAGAACAACTACTCTCACATACGGAAAGGTTTGGGTGCTTGGTCAAAGGACTTTACTGACTCTGCAGACGATGAATATCACAGGCCCTTACTACTATCAAAGATGTTCGACTAGACCTCTCGTGGTAGGGTACAAGTCCAAAAGACTTCTCTATGGGTACTAGGTAGGTAGTTAGTAGGTCCTGGGGTGTCTCAGAGTCGAACGTTCGGGGGAAGTTCCCCAAAGCATGTTAGAGATGTTATGATAACAAGTCCAAATTGGACGTCCGTCGCTGTCACCTACTCGGAACACTAGCTGAGGTGGCTTAAAAGCATTCCGTAACTGAGTGGTCACCGGGGGTGAAACATGCTGAGGGTTCCATACTAACGGGAAGGTGAAACTCTCCAATGTCTTTGAAAAGGACTAGTTGAGATAACCCAAACCCCGAATCGGAGGAAGAGGAGAACTTAGAGAAAACGTTGGCGACGTCATTAATAACGATTCCGTTCTTCCCGTTCGTGTCTATAAGGATATTCTTTCATTGCGAAGGTAAGCCCTTTTGGGTAGATACGGATTATGCCAAGCCCACAGACCTTCACTTGGAGTTGTTTTAGGAGACGGGTAGTGAGAAAACTTGTGGCACCTTCCCCGAGTCGGTTCCCTACAGACCCTCATAGCTCTAAGATCGTCAGTACACGACGACATTGTTCCGACTGACGTAGAGGCATTGCTCAAAAAAAAAAAAAAAAAAAGATAGAGCGACAGGCAAGTCACAAAGACACCGACAACTTTCTTGTC	/&++****892-'&#$&&##+*01:>FBB9:;0,335456<FB>@A8=:;B3-A9@AD?A776://12$%&$%%%%$&779;<542:1A>?:12?CC<A6/A2((10/@GF9599:@<EDIKMLMGCGG:85..))((01-/)&++,64*'/-18;92)(+*34454)0*$)1*&)5-7775=<@C??0?;;;+956469&1999.'&'$%99<3:;=DA>+1354-./'')+(764..3781?<C@DA7?@49,7%'%'+-+04-0$#''$<;8@<=83=@@A;37A*+=9;@?AA+4/0,.-(@25?FA>GFF0%%7%$)463-67:/14)**14*('%%,1<>:6)*;<9.5-+*2+1&><:../2<>9BD99<CA?:@)+*.;6/00264139<:<?AD;?8F,,5B4*D:;>??C</:5><B?B>?;3>9;6.:;;5--59:4497A;')$%41>11/)3;?=E96--(()%$%(*;;77<A>''<.,+0(&+/351+<7>=<..))&'>AA-);;.6/.-,%$$)000($#,127655=<,$$.-)5-BCIEJ>BADE9746663;@597540*8:=>>6785///.4446=,0+62(=67MF1++7;<77&&'*.--,274900/,+&*+++-)87:>?><><B>A72,'.47;<819;@9:;6?:;:4,%'%%#$#5760)&$&'%)%$&&%%'')..-,-((+0++&(*226>@[email protected]).55>;+*%/-4BBCB:;.%?@A??B?BAB@?AAEG=?4?<DA9A:=<>A;1(A-44>=0+;+&(36///,&&))02979;<:?6B8C@B8655578;)<@=?:<5662333;B@1'')+$"%&%'&4:<B:??>?B3.2354/4:@7::;;9?A???>61)&##$&&-,##$&*9;A?95@?@ECB>?=<>@>=DDFA1568>><<7+*)2*,++,+08&&@AC>>C@<=B;4?8:)))B@?<:=73,.-8=?;>3%('(3((')',-20,+51+@46*/*?9==9?G<6/++.2059?<02/.442&.::;<;./%'33350.*0'757@/7?))<35%&(46-%%**2;=?/1,+(-&&%*''8>@@@@<<<163**.$3383/**))-))$,.7:0-&6732</)14205/(+3.0@=:;?A@B;A>?@F0.%'@=8$%$+$#(89236E?64374??99B>>$D@A<CB@CC:;@9869=;743@396/2:587777=@80,,*'(-%>;;<9;.83398//&&$%$%&)'57D?;>D61(&,(+,8:./1234886=DB>:==KGCOB;=@=FC5+<1:;=EDGH>@FBB=>?HCEB668?871.96=>?3BB@CB<3&&$%)-+,-*+,*&'-''%,.0&87535;=8:BD4?(&''(((2//4::A@?D:,8.)''/()$*(/(00E;=>=:9<7??::IB786.&%$+%%+26&>===;GCB><91.'88<=/<,2@@@=<634*+98>6;<;887-,33:@?%,&)$#+&%$#&&5883334$11+:<7697:622:8;8:775+,1(((*;<9>77A@;4;:8;+:@D:7CADA>=;888=@@B==:+(-	NM:i:143	ms:i:2264	AS:i:2264	nn:i:0	tp:A:P	cm:i:124	s1:i:939	s2:i:786	de:f:0.0695	rl:i:0