help needed for reproducing FID

[youngjung]

help needed for reproducing FID

[HelenMao]

Hi, thanks for your interest in our work.
In our paper, we use all the training samples and the generated samples in all classes to compute FID (not per-class FID and average the result).
We find most of the previous papers report the FID in this way.
The mean and the standard derivation are calculated based on five independent trials. Please see the appendix in our paper.

[youngjung]

Wow super prompt reply! Thanks!

I misunderstood the sentence in the appendix 'We use all the training samples and the generated samples to compute FID.'. Now I see the point. 5K samples per class and 50K in total and all 50K samples are used to compute a single stats.

(What a waste of time writing per-class FID..... stupid me lol)

Further explanation on five trials is also valuable.
Could you explain whether it means

• five different models from five training trials (probably this one is the right one but just for double check..)
• or five testing for one fixed model (randomness from sampling z from Gaussian)
  ?

[HelenMao]

We just used one fixed model and generated five times of samples.
Hope it helps.

[youngjung]

Thanks!

Finally I got FID of 28.60 from your pretrained model.

However, the model trained by myself reports 35.66.

Did you observe variation among multiple training trials?

I will re-run the training for several times anyway but any comments will be helpful since training for 200K iters takes about 10 hours on my machine.

Is weight_decay on optimizer is accidentally not mentioned in the paper or not erased from your code?
Or, do I have to train for more than 200K iters?

[HelenMao]

We just obtained the pre-trained model using the codes released and reported the five times of the results using that model.

[youngjung]

Thanks it means that I can keep working on the same code!

I will train again several times and share my experience.

Thank you again :)

[youngjung]

I ran the same code for four more trials and got FIDs below.

trial0: 29.93222174258068
trial1: 30.902965756971412
trial2: 29.98750439731532
trial3: 35.25630393016536

Probably measure of the provided pre-trained model can be reached in a few more trials.

[HelenMao]

Hi, I find I set the max_iter=200,000 in my code. In fact, I remembered I trained 200 epochs of the pre-trained model.

[youngjung]

Oh, now I see that!

My 00199.pth are not actually from after 200 epoch hahahahaha.

Thanks I will try without if total_it >= max_it: block :)