What are the major differences between 2019-02-11 and 2019-08-01?

[ncbi]

What are the major differences between 2019-02-11 and 2019-08-01?

[azat-badretdin]

This might be related to issue #30.

Could you please post

• Command line
• Output of command tail cwltool.log (use find . -name cwltool.log if it's not in current directory)
• Output of command grep -P '2019.*\[.*\].*\$' cwltool.log | tail
• Output of command head cwltool.log

Thanks for your report!

[wangdi2014]

Thanks for your response! Here are the output from cwltoo.log

==
== output of command tail cwltool.log

21761/004/3319/RB 759D5501D4524281 19487/4689 2019-08-03T06:05:46.535382 eef0fa7478f2 UNK_CLIENT UNK_SESSION prosplign_wnode request-start ncbi_app_version=0.0.0&ncbi_app_sc_version=22&ncbi_app_vcs_revision=590519
21761/004/3319/R 759D5501D4524281 19488/4690 2019-08-03T06:05:46.535485 eef0fa7478f2 UNK_CLIENT UNK_SESSION prosplign_wnode extra job_id=3153&affinity=lcl|L43967.2&input=<job+file%3D"/tmp/tmp4kzqxabp/stg2213d48f-8d5c-48cc-9221-729452e381fe/compartments.asn"+offs%3D"2650334"+len%3D"553">
21761/004/3319/R 759D5501D4524281 19489/4691 2019-08-03T06:05:46.535600 eef0fa7478f2 UNK_CLIENT UNK_SESSION prosplign_wnode extra allow_alt_starts=true&compartment_penalty=0.10&cut_flank_partial_codons=true&cut_flanks_with_posit_drop=true&cut_flanks_with_posit_dropoff=40&cut_flanks_with_posit_gap_ratio=18&cut_flanks_with_posit_max_len=48&cut_flanks_with_posit_window=150&cut_trailing_Ns=false&fill_holes=true&flank_positives=49&frameshift_opening=30&gap_extension=1&gap_opening=10&input_file=/tmp/tmp8w57bhhc/stg978f4754-9654-4527-bd0c-1ede9f205beb/compartments.asn&intron_AT=25&intron_GC=20&intron_GT=15&intron_non_consensus=34&inverted_intron_extension=1000&max_bad_len=45&max_extent=100&max_intron=100&max_overlap=60&maximize=coverage&min_compartment_idty=0.05&min_exon_ident=30&min_exon_positives=55&min_flanking_exon_len=41&min_good_len=41&min_hole_len=200&min_intron_len=30&min_positives=15&min_singleton_idty=0.05&no_introns=true&nogenbank=true&score_matrix=BLOSUM62&start_bonus=8&stop_bonus=8&subj_mol=guess&total_positives=55
21761/004/3319/R 759D5501D4524281 19490/4692 2019-08-03T06:05:46.535886 eef0fa7478f2 UNK_CLIENT UNK_SESSION prosplign_wnode extra query=gi|504037966&subject=lcl|L43967.2&query_length=320&from=111758&to=112803&num_covered_aa=286&bit_score=200.936
21761/007/3316/R 759D5501D4524281 19491/4759 2019-08-03T06:05:46.543682 eef0fa7478f2 UNK_CLIENT UNK_SESSION prosplign_wnode extra alignments=1
21761/007/3316/RE 759D5501D4524281 19492/4760 2019-08-03T06:05:46.544019 eef0fa7478f2 UNK_CLIENT UNK_SESSION prosplign_wnode request-stop 200 0.028477907 553 509
21761/007/3320/RB 759D5501D4524281 19493/4761 2019-08-03T06:05:46.544093 eef0fa7478f2 UNK_CLIENT UNK_SESSION prosplign_wnode request-start ncbi_app_version=0.0.0&ncbi_app_sc_version=22&ncbi_app_vcs_revision=590519
21761/007/3320/R 759D5501D4524281 19494/4762 2019-08-03T06:05:46.544219 eef0fa7478f2 UNK_CLIENT UNK_SESSION prosplign_wnode extra job_id=3154&affinity=lcl|L43967.2&input=<job+file%3D"/tmp/tmp4kzqxabp/stg2213d48f-8d5c-48cc-9221-729452e381fe/compartments.asn"+offs%3D"2651439"+len%3D"553">
21761/007/3320/R 759D5501D4524281 19495/4763 2019-08-03T06:05:46.544361 eef0fa7478f2 UNK_CLIENT UNK_SESSION prosplign_wnode extra allow_alt_starts=true&compartment_penalty=0.10&cut_flank_partial_codons=true&cut_flanks_with_posit_drop=true&cut_flanks_with_posit_dropoff=40&cut_flanks_with_posit_gap_ratio=18&cut_flanks_with_posit_max_len=48&cut_flanks_with_posit_window=150&cut_trailing_Ns=false&fill_holes=true&flank_positives=49&frameshift_opening=30&gap_extension=1&gap_opening=10&input_file=/tmp/tmp8w57bhhc/stg978f4754-9654-4527-bd0c-1ede9f205beb/compartments.asn&intron_AT=25&intron_GC=20&intron_GT=15&intron_non_consensus=34&inverted_intron_extension=1000&max_bad_len=45&max_extent=100&max_intron=100&max_overlap=60&maximize=coverage&min_compartment_idty=0.05&min_exon_ident=30&min_exon_positives=55&min_flanking_exon_len=41&min_good_len=41&min_hole_len=200&min_intron_len=30&min_positives=15&min_singleton_idty=0.05&no_introns=true&nogenbank=true&score_matrix=BLOSUM62&start_bonus=8&stop_bonus=8&subj_mol=guess&total_positives=55

===
=== output of command grep -P '2019.[.].*$' cwltool.log | tail

[2019-08-03 06:04:36] INFO [job gpx_qsubmit_5] /tmp/rh8tte00$ gpx_qsubmit
[2019-08-03 06:04:38] INFO [job tblastn_wnode_2] /tmp/umye_efs$ tblastn_wnode
[2019-08-03 06:05:17] INFO [job gpx_make_outputs_2] /tmp/3m3v2nn3$ gpx_make_outputs
[2019-08-03 06:05:18] INFO [job cat] /tmp/o3mr1xvp$ cat
[2019-08-03 06:05:19] INFO [job Sort_Seed_Hits] /tmp/1mvd7xl2$ align_sort
[2019-08-03 06:05:20] INFO [job Filter_Full_Coverage_Alignments_I] /tmp/8t26qf_c$ bacterial_protalign_filter
[2019-08-03 06:05:23] INFO [job Filter_Protein_Seeds_I_prosplign_compart] /tmp/8180d9kc$ prosplign_compart
[2019-08-03 06:05:26] INFO [job Filter_Protein_Seeds_I_prosplign_compart_filter] /tmp/igyuzjgc$ prosplign_compart_filter
[2019-08-03 06:05:27] INFO [job Find_ProSplign_Alignments_gpx_qsubmit] /tmp/3x0ksynw$ gpx_qsubmit
[2019-08-03 06:05:28] INFO [job Find_ProSplign_Alignments_I_prosplign_wnode] /tmp/7hxr9549$ prosplign_wnode \

==
== output of command head cwltool.log

Original command: ./pgap.py -n -o mg37_results test_genomes/MG37/input.yaml

Docker command: /usr/bin/docker run -i --user 1000:1000 --volume /home/wangdi/work/2019-08-01/input-2019-08-01.build3919:/pgap/input:ro,z --volume /home/wangdi/work/2019-08-01/test_genomes/MG37:/pgap/user_input:z --volume /home/wangdi/work/2019-08-01/test_genomes/MG37/pgap_input.yaml:/pgap/user_input/pgap_input.yaml:ro,z --volume /home/wangdi/work/2019-08-01/mg37_results:/pgap/output:rw,z ncbi/pgap:2019-08-01.build3919 cwltool --timestamps --outdir /pgap/output pgap.cwl /pgap/user_input/pgap_input.yaml

--- Start YAML Input ---
fasta:
class: File
location: ASM2732v1.annotation.nucleotide.1.fasta
submol:
class: File

[azat-badretdin]

Thanks for posting this, we will look at this at earliest opportunity!

[slottad]

Note that running with smaller machines has not been extensively tested by us. However, the general rule of thumb is to have at least ~4GB of ram per cpu. We should probably add an option to pgap.py to allow throttling the container resources. When we have added such a feature (soon), we will let you know.

[wangdi2014]

2019-08-22 version is back to normal. I can run it on a smaller machine with 8GB memory.