HADDOCK local crashing during run.

[haddocking]

HADDOCK local crashing during run.

[mieczyslaw]

It means that the number of system's file descriptors in use exceeds the limit. This can be changed with 'ulimit' in linux.

It may also be an issue with the code when files are not closed, or temporary files are created on low level (I created a bug report to Python developers as the tempfile documentation doesn't mention that when creating a temporary file, it should be both removed after use and file descriptor closed even though the file was never opened).

[amjjbonvin]

Most like a Python limitation. It is probably more efficient to run fewer complexes at the same time rather than starting all 100 at the same time.

[iamysk]

Thanks for the prompt response @amjjbonvin . I'll check my implementation. Btw I need to run 735 complexes instead of the above mentioned 100, and it ran pretty well till around 400 complexes, after which I got these issues.

Anyways, thanks again for your response. And also for building and providing such a powerful tool.

[amjjbonvin]

May-be interesting for your case: https://github.com/haddocking/haddock-pilot <https://github.com/haddocking/haddock-pilot> Work in progress.

…

On 25 Nov 2021, at 14:06, Yogesh Kalakoti ***@***.***> wrote: Thanks for the prompt response @amjjbonvin <https://github.com/amjjbonvin> . I'll check my implementation. Btw I need to run 735 complexes instead of the above mentioned 100, and it ran pretty well till around 400 complexes, after which I got these issues. Anyways, thanks again for your response. And also for building and providing such a powerful tool. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#26 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABOH5IHB5WAGPXTSND2MKC3UNYYE3ANCNFSM5IXTVNKA>.

[iamysk]

Most like a Python limitation. It is probably more efficient to run fewer complexes at the same time rather than starting all 100 at the same time.

Thanks for the suggestion @amjjbonvin . I am not running 100 complexes simultaneously. My setup looks like the following.

# Input PDB directory
set z="processed_pdb_batch2/*"

@ run=1
# Iterate over all the proteins in the directory
foreach i (`ls -d $z`)
 # Make run.param file for current complex
 python makeParam.py --pdb $i --run $run
 # 1st haddock initialization
 haddock2.4
 # Enter run directory
 cd run$run
 # Edit the run.cns file to use 128 cores
 python ../editCNS.py --filename run.cns
 # 2st haddock initialization
 haddock2.4
 cd ..
 @ run+=1
 echo $i
 echo $run
end

It runs well for the initial few complexes, but arbitrarily freezes with the mentioned error or skips the current run and moves over to the next complex (Inferred from the small size ~400MB of such run directory in comparison to a complete run that weighs exactly 1.1GB). I suspect some temp files are not being closed properly during the three step (it0, it1, water) docking process for these incomplete runs, leading to the eventual bottleneck. Pls suggest a way to overcome or an alternative to automate HADDOCK runs for multiple complexes.

Thanks.

[amjjbonvin]

This is really an issue of your system and not haddock. We are running on our server behind the web portal up to 125 // docking runs without running into those issues. Most files are created and written by CNS and not python.

…

On 25 Nov 2021, at 13:57, Yogesh Kalakoti ***@***.***> wrote: Most like a Python limitation. It is probably more efficient to run fewer complexes at the same time rather than starting all 100 at the same time. Thanks for the suggestion @amjjbonvin <https://github.com/amjjbonvin> . I am not running 100 complexes simultaneously. My setup looks like the following. # Input PDB directory set z="processed_pdb_batch2/*" @ run=1 # Iterate over all the proteins in the directory foreach i (`ls -d $z`) # Make run.param file for current complex python makeParam.py --pdb $i --run $run # 1st haddock initialization haddock2.4 # Enter run directory cd run$run # Edit the run.cns file to use 128 cores python ../editCNS.py --filename run.cns # 2st haddock initialization haddock2.4 cd .. @ run+=1 echo $i echo $run end It runs well for the initial few complexes, but arbitrarily freezes with the mentioned error or skips the current run and moves over to the next complex (Inferred from the small size ~400MB of such run directory in comparison to a complete run that weighs exactly 1.1GB). I suspect some temp files are not being closed properly during the three step (it0, it1, water) docking process for these incomplete runs, leading to the eventual bottleneck. Pls suggest a way to overcome or an alternative to automate HADDOCK runs for multiple complexes. Thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#26 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABOH5IBXIVWWCZPPMJFKB5DUNYXEFANCNFSM5IXTVNKA>.