get_cns error

[nextomics]

get_cns error

[moold]

Could you send the following two files to me by email ?

• /nextdenovo/01.correct/01.raw_align/03.sort_align.sh.work/sort_align006/input.seed.088.sorted.ovl
• /nextdenovo/01.correct/01.raw_align/.input.seed.088.idx

[JHAO12321]

HI, Thank you for your immediate reply. I suspect that the ‘sorted.ovls’ may not have the correct structure due to some problems of the disk and trying to regenerate the files. I will rerun the tasks again with the new ovls and will send you the files if the errors still exist. Thank you! Best Regards 发件人: Hu Jiang <[email protected]> 答复: Nextomics/NextDenovo <[email protected]> 日期: 2019年7月17日 星期三 12:28 收件人: Nextomics/NextDenovo <[email protected]> 抄送: "王佳昊(Jiahao Wang)" <[email protected]>, Author <[email protected]> 主题: Re: [Nextomics/NextDenovo] get_cns error (#7) Could you send the following two files to me by email ? * /nextdenovo/01.correct/01.raw_align/03.sort_align.sh.work/sort_align006/input.seed.088.sorted.ovl * /nextdenovo/01.correct/01.raw_align/.input.seed.088.idx — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#7?email_source=notifications&email_token=AMUIB4RRTFSO7KWIPVNDWZ3P72NYBA5CNFSM4IEK6OOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2C7XVY#issuecomment-512097239>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMUIB4VJ7QE544J2VUW4NLTP72NYBANCNFSM4IEK6OOA>.

[moold]

OK

[JHAO12321]

HI, I have upload the two files, https://pan.genomics.cn/ucdisk/s/MJr2ii. Besides, I also need to know the reason of other two errors. Thank you very much!

…

________________________________ 发件人: Hu Jiang <[email protected]> 发送时间: 2019年7月17日 18:24 收件人: Nextomics/NextDenovo 抄送: 王佳昊(Jiahao Wang); Author 主题: Re: [Nextomics/NextDenovo] get_cns error (#7) OK ― You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#7?email_source=notifications&email_token=AMUIB4SUJ6REV5CA3MNIIZDP73XOLA5CNFSM4IEK6OOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2DX6RY#issuecomment-512196423>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMUIB4R6IUPTZWKAF6JDCDTP73XOLANCNFSM4IEK6OOA>.

[moold]

Hi, I have checked files that your uploaded and it seems this is a old version bug, I have fixed in the new version some weeks ago, so pls download the newest version and try again.

[JHAO12321]

OK, I will try. Thanks! Best Regards Jiahao [email protected] 发件人: Hu Jiang <[email protected]> 答复: Nextomics/NextDenovo <[email protected]> 日期: 2019年7月22日 星期一 13:38 收件人: Nextomics/NextDenovo <[email protected]> 抄送: "王佳昊(Jiahao Wang)" <[email protected]>, Author <[email protected]> 主题: Re: [Nextomics/NextDenovo] get_cns error (#7) Hi, I have checked files that your uploaded and it seems this is a old version bug, I have fixed in the new version some weeks ago, so pls download the newest version and try again. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#7?email_source=notifications&email_token=AMUIB4WU62ZIQEXSVNWW7NDQAVBUHA5CNFSM4IEK6OOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2O2FNY#issuecomment-513647287>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMUIB4UKSWGRISGQMTYXTO3QAVBUHANCNFSM4IEK6OOA>.

[JHAO12321]

I download the NextDenovo from github yesterday and rerun the get_cns step but the error still exists. Maybe it is caused by some other reasons. Could you please check again? Thank you very much! Best Regards Jiahao [email protected] 发件人: "王佳昊(Jiahao Wang)" <[email protected]> 日期: 2019年7月22日 星期一 14:52 收件人: Nextomics/NextDenovo <[email protected]> 主题: Re: [Nextomics/NextDenovo] get_cns error (#7) OK, I will try. Thanks! Best Regards Jiahao [email protected] 发件人: Hu Jiang <[email protected]> 答复: Nextomics/NextDenovo <[email protected]> 日期: 2019年7月22日 星期一 13:38 收件人: Nextomics/NextDenovo <[email protected]> 抄送: "王佳昊(Jiahao Wang)" <[email protected]>, Author <[email protected]> 主题: Re: [Nextomics/NextDenovo] get_cns error (#7) Hi, I have checked files that your uploaded and it seems this is a old version bug, I have fixed in the new version some weeks ago, so pls download the newest version and try again. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#7?email_source=notifications&email_token=AMUIB4WU62ZIQEXSVNWW7NDQAVBUHA5CNFSM4IEK6OOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2O2FNY#issuecomment-513647287>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMUIB4UKSWGRISGQMTYXTO3QAVBUHANCNFSM4IEK6OOA>.

[JHAO12321]

HI, if it is not easy to solve the problem, could you help me to ignore the reads like this ? Thank you. Best Regards Jiahao [email protected] 发件人: "王佳昊(Jiahao Wang)" <[email protected]> 日期: 2019年7月23日 星期二 09:39 收件人: Nextomics/NextDenovo <[email protected]> 主题: Re: [Nextomics/NextDenovo] get_cns error (#7) I download the NextDenovo from github yesterday and rerun the get_cns step but the error still exists. Maybe it is caused by some other reasons. Could you please check again? Thank you very much! Best Regards Jiahao [email protected] 发件人: "王佳昊(Jiahao Wang)" <[email protected]> 日期: 2019年7月22日 星期一 14:52 收件人: Nextomics/NextDenovo <[email protected]> 主题: Re: [Nextomics/NextDenovo] get_cns error (#7) OK, I will try. Thanks! Best Regards Jiahao [email protected] 发件人: Hu Jiang <[email protected]> 答复: Nextomics/NextDenovo <[email protected]> 日期: 2019年7月22日 星期一 13:38 收件人: Nextomics/NextDenovo <[email protected]> 抄送: "王佳昊(Jiahao Wang)" <[email protected]>, Author <[email protected]> 主题: Re: [Nextomics/NextDenovo] get_cns error (#7) Hi, I have checked files that your uploaded and it seems this is a old version bug, I have fixed in the new version some weeks ago, so pls download the newest version and try again. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#7?email_source=notifications&email_token=AMUIB4WU62ZIQEXSVNWW7NDQAVBUHA5CNFSM4IEK6OOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2O2FNY#issuecomment-513647287>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMUIB4UKSWGRISGQMTYXTO3QAVBUHANCNFSM4IEK6OOA>.

[moold]

You should re-run all pipeline, not only the get_cns step.

[moold]

But, if you only have few tasks with this error, you can check the input.fofn for the sort task, such as: /nextdenovo/01.correct/01.raw_align/03.sort_align.sh.work/sort_align006/input.fofn, and rerurn the minimap task which produced the ovl files in input.fofn. and then re-run the corresponding sort task and cns task.

[JHAO12321]

It will cost a lot of time to check whther the new version is ok. Is nextdenovo suitable for large dataset of pacbio？I know that small dataset is ok, but I haven't seen people to deal with large dataset of pb use nextdenovo. 获取 Outlook for Android<https://aka.ms/ghei36>

…

________________________________ From: Hu Jiang <[email protected]> Sent: Wednesday, July 24, 2019 9:33:22 AM To: Nextomics/NextDenovo <[email protected]> Cc: 王佳昊(Jiahao Wang) <[email protected]>; Author <[email protected]> Subject: Re: [Nextomics/NextDenovo] get_cns error (#7) But, if you only have few tasks with this error, you can check the input.fofn for the sort task, such as: /nextdenovo/01.correct/01.raw_align/03.sort_align.sh.work/sort_align006/input.fofn, and rerurn the minimap task which produced the ovl files in input.fofn. and then re-run the corresponding sort task and cns task. ― You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#7?email_source=notifications&email_token=AMUIB4WF6VU6WCRLAUUJXQDQA6WOFA5CNFSM4IEK6OOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2U4RQA#issuecomment-514443456>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMUIB4UUMZFVV5J4PMGWQADQA6WOFANCNFSM4IEK6OOA>.

[moold]

We have used NextDenovo for lots of large genomes with genome size from 10G to 30G, for large genome size, you can check FAQ IV (use usetempdir option and adjust -k -w for minimap2). Actually, for large genomes , if you want to correct the raw reads and then do the assembly, you almost have no other choices, that is why we developed NextDenovo.

[moold]

I think you can re-run one sort task, and check if this is ok, and then re-run all other unfinished tasks.

[JHAO12321]

Yes, I am trying one task now. I hope this will work. Is this bug specifically for pacbio data? I know many people do large genome data analysis using nextdenovo but they all used nanopore for sequencing. They have used an older version than me but nothing wrong happened. And I have ssh to the compute node of one get_cns task, there are 24 subtasks but almost all of them are in S stat. Despite I set “-p 24”, the load of cpus is about 1~2. Is there anything wrong of my job? Does this mean, whatever the “-p” is, the task is actually using one cpu when running. Best Regards Jiahao [email protected] 发件人: Hu Jiang <[email protected]> 答复: Nextomics/NextDenovo <[email protected]> 日期: 2019年7月24日 星期三 13:07 收件人: Nextomics/NextDenovo <[email protected]> 抄送: "王佳昊(Jiahao Wang)" <[email protected]>, Author <[email protected]> 主题: Re: [Nextomics/NextDenovo] get_cns error (#7) I think you can re-run one sort task, and check if this is ok, and then re-run all other unfinished tasks. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#7?email_source=notifications&email_token=AMUIB4VVMRCQVBWB66YDWB3QA7PRTA5CNFSM4IEK6OOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2VFZGY#issuecomment-514481307>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AMUIB4QFCVOKKM5M4IK3HFTQA7PRTANCNFSM4IEK6OOA>.

[moold]

1.No, not only for pacbio data.
2. It means IO wait, please use usetempdir option, especially if usetempdir is on a SSD driver.

[moold]

I will close this issue, but if you still have this problem, feel free to reopen it.