Make jobname match number in batchtools_XXXXXXXXXX by default when using SLURM?

[kendonB]

Make jobname match number in batchtools_XXXXXXXXXX by default when using SLURM?

[HenrikBengtsson]

The format of that path is:
.future/<timestamp>_{{R-SESSION-ID}}/batchtools_{{FUTURE_ID}}/logs/job{{BATCHTOOLS-JOB-ID}}.log

I agree that having ../batchtools_{{FUTURE_ID}}_{{BATCHTOOLS-JOB-ID}}/... instead of ../batchtools_{{FUTURE_ID}}/... could be handy. Unfortunately, I'm not sure this can be done, because that batchtools_{{FUTURE_ID}} part must be set up before submitting a batchtools job (because that is actually the batchtools registry name). Only later is the {{BATCHTOOLS-JOB-ID}} created when the job is created/submitted.

When you say you "want to peek", do you mean from outside of R while the job is running? That is, it would not be helpful to provide accessor functions to the BatchtoolsFuture class?

[kendonB]

I have been looking from outside of R while the job is running, yes. However, if the best way to achieve the peeking would be from within a second R session, then, of course, that would also work great.

[HenrikBengtsson]

I do that at times as well. Also, when there's an error, I find myself navigating these temporary future+batchtools repositories by hand from within R (the default behavior of future.batchtools is to not clean them up if there's an error).

Having an R API to peek into/browse around in an existing (and possibly even a live/hot) batchtools job registry would be neat. If possible, it's not hard to imagine of pulling out an overview of current jobs, e.g. classical stdout, fancy color-annotated stdout, or even a GUI/Shiny dashboard. I think that such a feature request for a basic querying API is better address to the batchtools project itself.

[kendonB]

Ref: mllg/batchtools#157

[HenrikBengtsson]

Note to self: good news from mllg/batchtools#157 (comment):

To clarify this feature request: Is it possible (and safe) to set up a pointer to a batchtools registry from a separate R session (by only knowing its path on the file system) and use the find*() functions to query it's status?

Yes, it should be possible if you use loadRegistry("<path>", writeable = FALSE). If you encounter problems while only monitoring, it is a bug in batchtools.