How do I run NextDenovo on my SLURM / SGE / PBS / LSF / Torque system?

[nextomics]

How do I run NextDenovo on my SLURM / SGE / PBS / LSF / Torque system?

[moold]

Please see the option doc, and you will find the option: job_type = local can be used to set running nextdenovo on a local system.

[zijiangyang]

Please see the option doc, and you will find the option: job_type = local can be used to set running nextdenovo on a local system.

@moold
Hi Hu:

I checked the option doc and only sge cluster_options are provided. May I ask the settings for cluster_options when using PBS system? Shall I put the nodes name in the nodelist?

Thx!

[moold]

I do not test PBS system. but you can use PBS template to define the resource requirements for each job, it should be ok, see here for more details. The nodelist is only used with the option: 'usetempdir'.

[Rachel-pku]

How do I run NextDenovo on my SLURM system? Change the job_type = slurm ?

[moold]

yes, also should change cluster_options to set the resource requirements for each job.

[Rachel-pku]

sorry, I don't know the meaning of option in cluster_options when I want to run nextDenovo on slurm cluster? Can you give the detail information for it.Thx

[moold]

NextDenovo will replace {vf}, {cpu}, {bash} in cluster_options with specific values needed for each jobs. For slurm system, may be you can try -–cpus-per-task={cpu} -–mem-per-cpu={vf}.

[Rachel-pku]

I do not work it successfully on slurm clusters when I try to modify the cluster_option. The err report the script in nextDenovo and kit.py may be have some invalid syntax.

[moold]

Becasue I do not have a slurm clusters , I can not test it by now. But the cluster_option is the same with nativeSpecification in drmaa.

[moold]

Hi, you can try to download the lastest version which have been increased more compatibility to run on a slurm system. I have ran it on a slurm system successfully.

[shehongbing]

Hi, how can I run nextdenovo on PBS system.

[moold]

Try this -l nodes=1:ppn={cpu}:mem={vf}, but I do not test.

[shehongbing]

hi hu, i test it but it does not work. thx

[bmbbms]

Hi,when i use pbs system with params cluster_options = -l -q cu nodes=1:ppn={8}:mem={20} ,it does not work,so what can i do ?

[moold]

Try to use use -l -q cu nodes=1:ppn={cpu}:mem={vf}, or you can manually submit subtasks to the computer cluster.

[ggoodstudydaydayup]

@zijiangyang
Hi Yang,
May I ask you how to run nextdenovo on pbs system finally?
Thanks.
Liu

[zijiangyang]

@zijiangyang
Hi Yang,
May I ask you how to run nextdenovo on pbs system finally?
Thanks.
Liu
@ggoodstudydaydayup
Hi Liu:
Unfortunately I failed to run nextdenovo with pbs settings due to drmaa error. Still it is possible to run nextdenovo on pbs systems by submitting jobs manually: you can submit the main script (nextDenovo run.cfg) with job_type set as local and wait for the generation of scripts for sub tasks, once the generation of scripts for sub tasks has finished (normally took a second), qdel the main script and submit the sub scripts manually. When all the sub scripts are finished, a file with suffix "done" will appear, then activate the main scripts for the next step and repeat the above procedures. By the way, if you want to get fasta output for each run, change the parameter -a 0 to -a 1 for every nextDenovo.sh in 03.ctg_graph/01.ctg_graph.sh.work/*/

I hope this helps you!

[ggoodstudydaydayup]

@zijiangyang
Hi Yang,
May I ask you how to run nextdenovo on pbs system finally?
Thanks.
Liu
@ggoodstudydaydayup
Hi Liu:
Unfortunately I failed to run nextdenovo with pbs settings due to drmaa error. Still it is possible to run nextdenovo on pbs systems by submitting jobs manually: you can submit the main script (nextDenovo run.cfg) with job_type set as local and wait for the generation of scripts for sub tasks, once the generation of scripts for sub tasks has finished (normally took a second), qdel the main script and submit the sub scripts manually. When all the sub scripts are finished, a file with suffix "done" will appear, then activate the main scripts for the next step and repeat the above procedures. By the way, if you want to get fasta output for each run, change the parameter -a 0 to -a 1 for every nextDenovo.sh in 03.ctg_graph/01.ctg_graph.sh.work/*/

I hope this helps you!

Thanks very much!

[moold]

Hi, I have written a tutorial about how to configure drmaa in different computer clusters and how to set cluster_options, anyone who has questions about it can visit here.

[christinawu2008]

Hi, you can try to download the lastest version which have been increased more compatibility to run on a slurm system. I have ran it on a slurm system successfully.

Hey, could you please paste your config file here? with the cluster options for slurm? Thank you!

Cheers,
Chen

[moold]

see here

[christinawu2008]

see here

I'm also trying to run it on slurm. i was asking whether you could paste your config file that you used for slurm system. such as where to add cluster options relating to slurm etc...

Thanks

Cheers,
Chen

[moold]

Just set job_type = slurm, if you want to change some cluster options, see Paralleltask or set the submit option.