Comments (3)
That sounds weird. I haven't heard of any other instances of this happening. It could be many many things not related to gwf at all (the filesystem, Slurm reporting intermittent states, jobs not being completely finished, but being counted as "completed" by gwf for a short while).
Have you experienced this since you reported it?
from gwf.
Hi Dan,
I haven't experienced it since no. Not sure what went wrong. I see how it could be a problem in slurm and not GWF.
I have recently started using the following instead of gwf status
since it returns instantly (gwf status
takes 45min with my giant workflow :-) ):
alias sqs='squeue -u $USER -o "%.10i %.9P %.8j %.8u %.2t %.10M %.6D %.6R" | awk '\''
NR > 1 {
state = $5
if (state == "R") state = "Running"
else if (state == "PD") state = "Pending"
else if (state == "CG") state = "Completing"
else if (state == "CD") state = "Completed"
else if (state == "F") state = "Failed"
else if (state == "TO") state = "Timeout"
# Add more state mappings as needed
count[state]++
}
END {
for (status in count) print status, count[status]
}'\'
from gwf.
I'll close this for now then :-)
(gwf status
needs to ask for metadata on every single file in your workflow to determine if everything is up-to-date, while squeue
just gives you the list of jobs and their status).
from gwf.
Related Issues (20)
- time.clock() not available in python 3.9 HOT 1
- time HOT 1
- collections.abc
- After update to 1.8.0: Targets ending in "Shouldrun" even though they finish correctly HOT 2
- Import problem with gwf 1.8.1 HOT 2
- percentage of dependencies complete for targets is always 0.00% HOT 1
- Scheduler resubmits completed jobs (gwf v1.8.5) HOT 1
- gwf touch: unexpected keyword argument 'group' HOT 2
- Allow gwf touch to only touch parts of the graph
- Add "failed" to status -s HOT 4
- Target that should not run yet fails HOT 4
- Update documentation
- Failed targets are not recognized as failed HOT 1
- gwf does not work with python 3.12 HOT 1
- Handle too long target names HOT 2
- Support PathLib paths in targets HOT 1
- Running gwf on SGE HOT 3
- (Edited) Option to log context HOT 6
- Add argument to run for submitting targets without IO checks HOT 5
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gwf.