why large training/validation loss difference between VGG16 vs inception_v2?

[titu1994]

why large training/validation loss difference between VGG16 vs inception_v2?

[bobby-chiu]

I also train TID2013 dataset on inception_v2/mobile_net with the same manner , pre-trained results are with 0.1/0.2 loss on inception_v2/mobile_net, but the loss on the fine-tune procedure is almostly unavailable and unchanged (got 0.23/0.5 loss at the beginning of training epoch and 0.45 finally). I have no idea why the loss suddenly became such large than pre-trained results?

[titu1994]

Hmm, I haven't tried tid2013 dataset, so I can't give a definitive answer as to why VGG doesn't match the results. I believe they used a slightly different scoring scheme for tid since it's score range is 0-9 rather than 1-10 like AVA. Perhaps that is causing some error in score calculation, though I highly doubt it.

VGG 16 is also very easy to overfit with so little data, and harder to train compared to other models. Are you using the same validation set that the paper is using to score your model ? I believe they used a 20 % split of the data as validation set.

Tid is a relatively small dataset. It is easy to overfit, even with cross validation. So your 25 models may just have overfit for a few of the harder cases. I see the same thing in the pretrained models such as NASNet.

[bobby-chiu]

@titu1994, i have noticed that tid2013 is easy overfitting if only 3000 data used for training and validation. So data augmentation with randomly flip and crop is applied as followed your code. The paper used 20% of the data as validation, but still unknown for me which 20% data will remove from training data. As i know, leave-one-distorted-level-out is useful to make training and testing data with similar distribution (all have same scene and distorted types). My experiments have validated this works on testing data. For leave-one-scene-out method, training model has never seen the scene in testing data, this may be challenging prediction for pre-trained model. Now i am searching more available datasets and simulating more distorted images/types to enlarge training data. Hopes it will work for distorted image scoring application.

[JustinhoCHN]

@bobby-chiu hi, have you enlarge your training data?