Pytorch Lightning run is marked as finished after .fit loop

[Michael-Tanzer]

Pytorch Lightning run is marked as finished after .fit loop

[mihran113]

Hey @Michael-Tanzer! Thanks for the report.
The run is being closed, because pytorch lightning is calling .finalize() method on the logger. But when the test loop starts, and the trainer logs any additional metrics during the test loop aim.Run will be reopened.
I've tested it on our example(https://github.com/aimhubio/aim/blob/main/examples/pytorch_lightning_track.py) and the test loss is successfully tracked.
Can you please double-check if the test metrics are tracked?

[Michael-Tanzer]

Hi, I'm glad it's working on this example, but there is also another ticket with pretty much the same issue. Could it be related to the fact that I am using a remote server? My current fix is to disable finalize and later finalize the run manually.

[Michael-Tanzer]

#3097

[mihran113]

Oh, yeah, remote tracking is actually causing this.
I've just opened a PR which should address that:
#3134
We'll release a patch version today or tomorrow which will include the fix for this issue.

[Michael-Tanzer]

Thank you! This is awesome news! I will close this issue then