dougsouza/pytorch-sync-batchnorm-example

How to use Cross Replica / Synchronized Batchnorm in Pytorch

pytorch-sync-batchnorm-example's Issues

Example

your explaination is very helpful. But I still could not implement it. Could you please give an example? Thanks.

About validating

Hi, thanks for the nice guide here. Really saves me lots of time.
I have another question about saving/loading the model and validating.

From the imagenet example > https://github.com/pytorch/examples/blob/master/imagenet/main.py
We only need to save our model once on only rank0 device, right?
And I wrote another standalone script for validating on one gpu only (with batchsize=1), do we still need to warp it up with distributed parallel and convert the model to use syncbn?

Many thx!

can you give a complete example?

for example, I want to use gpu2, gpu4, gpu5 to train the net, how can I set ?