Test successful NCBI Cat 8.0 to UCSC

[ncbi-hackathons]

Test successful NCBI Cat 8.0 to UCSC

[childers]

$ head -n 20 test_cat8.gff3 
ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/181/335/GCF_000181335.2_Felis_catus_8.0/GCF_000181335.2_Felis_catus_8.0_assembly_report.txt
##gff-version 3
#!gff-spec-version 1.21
#!processor NCBI annotwriter
#!genome-build Felis_catus_8.0
#!genome-build-accession NCBI_Assembly:GCF_000181335.2
#!annotation-date 31 December 2016
#!annotation-source NCBI Felis catus Annotation Release 103
##sequence-region NC_018723.2 1 240380223
##species https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=9685
chrA1	RefSeq	region	1	240380223	.	+	.	ID=id0;Dbxref=taxon:9685;Name=A1;breed=Abyssinian;chromosome=A1;gbkey=Src;genome=chromosome;isolate=Cinnamon;mol_type=genomic DNA;sex=female
chrA1	Gnomon	gene	25237	30258	.	+	.	ID=gene0;Dbxref=GeneID:102899061;Name=LOC102899061;gbkey=Gene;gene=LOC102899061;gene_biotype=lncRNA
chrA1	Gnomon	lnc_RNA	25237	30258	.	+	.	ID=rna0;Parent=gene0;Dbxref=GeneID:102899061,Genbank:XR_002147070.1;Name=XR_002147070.1;gbkey=ncRNA;gene=LOC102899061;model_evidence=Supporting evidence includes similarity to: 100%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 6 samples with support for all annotated introns;product=uncharacterized LOC102899061%2C transcript variant X3;transcript_id=XR_002147070.1
chrA1	Gnomon	exon	25237	25514	.	+	.	ID=id1;Parent=rna0;Dbxref=GeneID:102899061,Genbank:XR_002147070.1;gbkey=ncRNA;gene=LOC102899061;product=uncharacterized LOC102899061%2C transcript variant X3;transcript_id=XR_002147070.1
chrA1	Gnomon	exon	25993	26083	.	+	.	ID=id2;Parent=rna0;Dbxref=GeneID:102899061,Genbank:XR_002147070.1;gbkey=ncRNA;gene=LOC102899061;product=uncharacterized LOC102899061%2C transcript variant X3;transcript_id=XR_002147070.1
chrA1	Gnomon	exon	27897	30258	.	+	.	ID=id3;Parent=rna0;Dbxref=GeneID:102899061,Genbank:XR_002147070.1;gbkey=ncRNA;gene=LOC102899061;product=uncharacterized LOC102899061%2C transcript variant X3;transcript_id=XR_002147070.1
chrA1	Gnomon	lnc_RNA	25237	30258	.	+	.	ID=rna1;Parent=gene0;Dbxref=GeneID:102899061,Genbank:XR_002147044.1;Name=XR_002147044.1;gbkey=ncRNA;gene=LOC102899061;model_evidence=Supporting evidence includes similarity to: 100%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 30 samples with support for all annotated introns;product=uncharacterized LOC102899061%2C transcript variant X2;transcript_id=XR_002147044.1
chrA1	Gnomon	exon	25237	25514	.	+	.	ID=id4;Parent=rna1;Dbxref=GeneID:102899061,Genbank:XR_002147044.1;gbkey=ncRNA;gene=LOC102899061;product=uncharacterized LOC102899061%2C transcript variant X2;transcript_id=XR_002147044.1
chrA1	Gnomon	exon	27897	30258	.	+	.	ID=id5;Parent=rna1;Dbxref=GeneID:102899061,Genbank:XR_002147044.1;gbkey=ncRNA;gene=LOC102899061;product=uncharacterized LOC102899061%2C transcript variant X2;transcript_id=XR_002147044.1

[childers]

Rerunning test, now it is failing...

$ time seqconv convert  --out gb --ref Mm_Celera alt_Mm_Celera_top_level.gff3 >mouse_Celera_nogz.gff3
Converting from None to gb
...
...
...
Cannot convert id: NW_001037322.1
Cannot convert id: NW_001030938.1
Cannot convert id: NW_001035636.1
Cannot convert id: NW_001032658.1
Cannot convert id: NW_001035887.1
Cannot convert id: NW_001072727.1
Cannot convert id: NW_001033972.1
Cannot convert id: NW_001075011.1
Cannot convert id: NW_001034251.1
Cannot convert id: NW_001034379.1
Cannot convert id: NW_001037486.1
Cannot convert id: NW_001033807.1
Cannot convert id: NW_001033809.1
Cannot convert id: NW_001031014.1
Cannot convert id: NW_001075320.1
Cannot convert id: NW_001071984.1
Cannot convert id: NW_001033738.1
Cannot convert id: NW_001075955.1
Cannot convert id: NW_001037425.1
Cannot convert id: NW_001033761.1
Cannot convert id: NW_001071974.1
Cannot convert id: NW_001034374.1
Traceback (most recent call last):
  File "/usr/local/bin/seqconv", line 11, in <module>
    load_entry_point('seqconv==0.0.1', 'console_scripts', 'seqconv')()
  File "/usr/local/lib/python2.7/site-packages/seqconv-0.0.1-py2.7.egg/cli/command.py", line 98, in main
    comm()
  File "/usr/local/lib/python2.7/site-packages/seqconv-0.0.1-py2.7.egg/cli/command.py", line 52, in __init__
    getattr(self, args.command)()
  File "/usr/local/lib/python2.7/site-packages/seqconv-0.0.1-py2.7.egg/cli/command.py", line 94, in convert
    id_to=args.out)
  File "/usr/local/lib/python2.7/site-packages/seqconv-0.0.1-py2.7.egg/cli/assembly.py", line 340, in converter
    except (KeyboardInterrupt, SystemExit, BrokenPipeError):
NameError: global name 'BrokenPipeError' is not defined

real	1m57.201s
user	0m14.091s
sys	0m0.862s

[childers]

Looks like this is resolved with current codebase:

$  time  seqconv convert --ref Felis_catus_8.0 --out uc ref_Felis_catus_8.0_top_level.gff3 >test_cat8.gff3
Converting from None to uc
Starting Conversion
FORMAT detected: rs
real	5m23.860s
user	0m16.260s
sys	0m2.069s
arsmdbe22209001:NCBI cchilders$ 
arsmdbe22209001:NCBI cchilders$  time  seqconv convert --ref Felis_catus_8.0 --out uc ref_Felis_catus_8.0_top_level.gff3 >test_cat8_run2.gff3
Converting from None to ucStarting Conversion
FORMAT detected: rs
real	0m24.684s
user	0m15.357s
sys	0m1.907s

Also, VPN makes getting the lookup table VERY slow