Timing out on minimap-nd tasks

[wgallin]

Timing out on minimap-nd tasks

[moold]

Two solutions

1. It seems that your system limits the running time of a job, so you can reduce blockize and increase seed_cutfiles to reduce the size of each subfile and speed up the map task. But the total runing time maybe will longer.
2. see here or here to adjust the submit command.

[DaniPaulo]

Hi @wgallin . I'm still trying to figure out how to run NextDenovo in a HPC environment using SLURM. Would you be able to share your script.slurm.sh with me?

[wgallin]

Hi, So I ddi figure out why I was having a problem, and worked around it. The basic problem was that when running in Grid mode SLURM allows to system administrators to set the wall time for jobs that are submitted without an explicit wall time value. In my case about 10% of jobs in one step ran over that time, so the whole job crashed because these jobs would not complete. The solution that I used was to run the job in LOCAL mode on a single node with 32 cpus, 256G RAM and a wall time that turned out otherwise be much longer than the job actually took (I requested 7 days but the job finished in less than 4 days). If I had wanted to run the job in Grid mode I would have needed to be able to set the wall times for at least some of the individual sub-jobs, but I could not find a way to do that in the submission script. So I guess the solution to my problem would have been to allow the submission script or the configuration file to feed a user-defined wall-time (and probably memory allocation) to the individual sub-jobs that the parent job spawns onto the grid. Warren Gallin

On Apr 22, 2024, at 2:41 AM, DaniPaulo ***@***.***> wrote: Hi @wgallin <https://github.com/wgallin> . I'm still trying to figure out how to run NextDenovo in a HPC environment using SLURM. Would you be able to share your script.slurm.sh with me? — Reply to this email directly, view it on GitHub <#203 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEW55K3DYL4ZKPKZBXMIPODY6TEJ7AVCNFSM6AAAAABFRYWY5SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRYHAZDINBYG4>. You are receiving this because you were mentioned.



[DaniPaulo]

Hi @wgallin,

Thanks for your response. Let's see if I understand.
So basically, you set up your script.slurm.sh to:

#SBATCH --nodes 1
#SBATCH --ntasks 1
#SBATCH --cpus-per-task 32
#SBATCH --mem 256G
#SBATCH --time 7-00:00:00

# MODULES
module load nextdenovo

# MAIN
nextDenovo run.cfg

And your run.cfg to use local, one parallel job and -t / -p to 32:

[General]
job_type = local
parallel_jobs = 1

[correct_option]
sort_options = -t 32
minimap2_options_raw = -t 32
pa_correction = 3
correction_options = -p 32

[assemble_option]
minimap2_options_cns = -t 32

Could you please verify this?
I appreciate your help,
Dani.