Problem about the training result

[chenf99]

Problem about the training result

[chenf99]

In instruction you decayed the learning rate by 90% when the validation loss stopped improving for long periods, so how many epochs should I wait for this long periods? @sgrvinod

[sgrvinod]

Hi @chenf99, are you using the same datasets that I used? The parameters are the same as in the paper, except for the batch size. I trained using a batch size of 8, but for the same number of epochs as in the paper.

I think I waited around 15 epochs to see if the loss improved before early-stopping. But I realize that this just complicates the training. So I removed the early-stopping/validation and simply followed the procedure in the paper. (A learning rate of 1e-3 for N epochs, 1e-4 for M epochs, 1e-5 for M epochs. N, M are the number of epochs corresponding to 80k, 20k iterations respectively at the paper's batch size of 32.) This also slightly improved my mAP (from 77.1 to 77.2), so I guess it's just simpler and better to not do any early-stopping/validation and to simply follow the above schedule. I have now updated the code and tutorial to reflect this.

I don't know why you are getting a loss of 3.5 though. You could run eval.py to check what your mAP is, since loss can be measured differently. Also, there have been some breaking changes since PyTorch version 0.4, so did you solve those?

[chenf99]

@sgrvinod Thank you for your reply, I tried again and I can get good result this time.