No job statistics in BatchExplorer

[azure-samples]

No job statistics in BatchExplorer

[matthchr]

Hi @wirehell can you expand more on this? Where are you looking for the JobStats and not seeing them? Have you tried turning on "use stats during list" in the options menu?

[wirehell]

For tasks the statistics is there:

But not for jobs:

[matthchr]

@wirehell Are you refreshing your resources after enabling the "use stats during list" (either by refreshing the individual item or by refreshing the list of jobs?). If you do not refresh after checking the "use stats during list" box then the statistics don't show up.

Also, JobStatistics takes a bit to aggregate, you won't see it immediately after the first task in the job is run, it will take some time (I think 15m).

I looked into this a bit more on our server though and it seems your job doesn't have stats even now. We are looking into why that is.

[matthchr]

@wirehell Sorry for the long response -- there was a bug in the service where some jobs were not having statistics aggregated for them correctly (so you would see task level statistics but not job level statistics).

A fix has been rolled out, so all new jobs statistics will work (new jobs should have statistics). There is no issue in BatchExplorer, this was an issue with the underlying service.

[wirehell]

Ah, I see. Thanks.

[Malondron]

Hi,

the issue seems to persist. Both using the BatchExplorer and the usual batch rest API, the tasks have statistics, but the jobs don't.

[nmontmarquette]

@matthchr Probably not directly related to the original issue, how do you get statistics for jobs or tasks? I am using the Python API. When querying jobs with the list method, I feel like you have to supply something in the JobListOptions in the expand parameter. I have some hard time finding up-to-date documentation on that.

[darylmsft]

We will get you a clear answer in Python, but because it is late on a Friday I will add enough non-Python information in the hopes you can be unblocked. The <job, task> statistics are considered "associated entities" and such entities are more expensive to fetch and, thus, are not returned by default. In the context of C# this is discussed here. The REST API for ListJobs shows the expand clause here.

These sources implicate the JobListOptions in Python (with expand set to stats).

We will get you a sample fragment in Python to show how this is done but until then hopefully the above will help.

[annatisch]

Hi @nmontmarquette,
In the Python SDK this can be done with the JobGetOptions (or taskGetOptions) where you can set the expand clause that @darylmsft described:

options = batch.models.JobGetOptions(expand='stats')
job = batch_client.job.get(job.id, job_get_options=options)
print(job.stats)

The same syntax is used for the select clause.
If you want to select or expand multiple attributes, you can specify comma-separated values:

options = batch.models.JobGetOptions(expand='stats', select='id,state')
job = batch_client.job.get(job.id, job_get_options=options)