Weight Decay

[megvii-basedetection]

Weight Decay

[yinchimaoliang]

The weight_decay was set to 1e-7 in the beginning. But when we conduct experiments on the cbgs exps, it may cause loss expansion during training, so we change it to 1e-2.

[Divadi]

Thank you for the clarification!

[zw615]

The weight_decay was set to 1e-7 in the beginning. But when we conduct experiments on the cbgs exps, it may cause loss expansion during training, so we change it to 1e-2.

Only the weight_decay in bev_depth_lss_r50_256x704_128x128_20e_cbgs_2key_da_ema.py is set to 1e-2. The weight_decay in bev_depth_lss_r50_256x704_128x128_20e_cbgs_2key_da.py is still 1e-7. I wonder what is the relation between weight_decay and ema?