Questions about optimal values of the lambdas, data split and epsilon values

[ck-amrahd]

Questions about optimal values of the lambdas, data split and epsilon values

[ck-amrahd]

Hi Mo,
Thank you for your interest in our paper.

1. Yes, you are correct. it should be train_images, I was experimenting with the smaller dataset and forgot to update to previous values.
2. For lambda_1 and lambda_2, we selected them by training multiple models as mentioned in the paper and selecting the model that performs best for a given value of epsilon.
3. validation set was used to select models among different trained models and the accuracies mentioned in the paper are for the test dataset.
   I hope it clears your confusion. Thank you.

[mohanhanmo]

Hi Dharma,

Thank you very much for your quick response! Your answers are really helpful and I appreciate it a lot. Continuing with my questions:

1. Thank you very much for your clarification!

2. I understand this part that the lambda_1 and lambda_2 were selected by grid search according to the performance of each parameter pair, and we may have different optimal results of lambda_1 and lambda_2 since the dataset was split randomly in each of our side. But I was wondering what is the best selection of lambda_1 and lambda_2 in your side when epsilon=0, so that I could compare them with the optimal selection on my end to see how different the optimal selections could be for different data split.

3. For the validation accuracies with different epsilons, I also tested the model on the test dataset, and the results were similar with those of the validation dataset (around 30% accuracy for the normal model case with epsilon=0.175), since the dataset was split randomly into testing and validation set without any specification. Is there any other possible reason that I could not get similar accuracies with different epsilons as the curve in the paper, please?

Thank you very much for your kind help!!

[ck-amrahd]

Hi Mo,
I looked at the results and it seems like lamnda_1=1 and lambda_2=4.64 gave good values for epsilon equals 0 [for bbox training]. For a normal model, we should use lamnda_1=0 and lambda_2=0 [that's normal CNN training where we don't do any penalization]. I used the train_val set to select the best model during training and then use the val dataset to select lambda_1 and lambda_2. If you use lambda_1=0 and lambda_2=0 and do normal training, it should produce test acc around 50% on the test set with epsilon=0.175, which is the value that you can see in the graph in the paper. Thank you.

[mohanhanmo]

Hi Mo,
I looked at the results and it seems like lamnda_1=1 and lambda_2=4.64 gave good values for epsilon equals 0 [for bbox training]. For a normal model, we should use lamnda_1=0 and lambda_2=0 [that's normal CNN training where we don't do any penalization]. I used the train_val set to select the best model during training and then use the val dataset to select lambda_1 and lambda_2. If you use lambda_1=0 and lambda_2=0 and do normal training, it should produce test acc around 50% on the test set with epsilon=0.175, which is the value that you can see in the graph in the paper. Thank you.

Thank you so much for your suggestion! I will try them out. For now I will close the issue.