Running on Office31

[RayWangWR]

Running on Office31

[RayWangWR]

I am adaption from amazon to webcam. During training, the validation accuracy on webcam first increased a little bit, then it began to drop very fast after about 20 iterations.

[LamForest]

One possible solution is to use a small lr for target encoder(e.g. 1e-5) and reduce training epoch.
My setting:
ResNet-50 for source and target encoder.
lr for source encoder : 1e-3 this doesnot matter
lr for discriminator: 1e-3
lr for target encoder: 1e-5
trainin epoch : 6

[LamForest]

BTW, I use https://github.com/corenel/pytorch-adda not this offical Tensorflow. My result (A->W) is 81% using above setting.

[RayWangWR]

Hi Tianlin-Gao,

Thanks for your reply. May I know which layer are you matching and do you fix some layers in the source/target encoder to avoid overfitting? Also, can you tell me the number/size of the hidden layer of your discriminator and which optimizer are you using, parameters for the optimizer? Or would you mind sharing your code for adda on Office?

I am sorry that this has been killing me for days. I just want to get adda work and build my model based on it. I would be really thankful if you can provide more information on that implementation.

Thanks,
Rui
[email protected]

[RayWangWR]

Hi, may I have a look at your specific setting regarding the results in your last post? On Oct 31, 2018, at 04:03, Tianlin-Gao <[email protected]<mailto:[email protected]>> wrote: BTW, I use https://github.com/corenel/pytorch-adda<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_corenel_pytorch-2Dadda&d=DwMCaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=KUScIrWv9DcqHOllgaD0uQ&m=7ghePg-YJ1SoumiNtku-yZNINB1H7o1Kvt-YtJIVNQg&s=r-yT-qiAalhw6wb-kWIG2mO2cRLO0bT411figYpfwaQ&e=> not this offical Tensorflow. My result (A->W) is 81% using above setting. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_erictzeng_adda_issues_11-23issuecomment-2D434595245&d=DwMCaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=KUScIrWv9DcqHOllgaD0uQ&m=7ghePg-YJ1SoumiNtku-yZNINB1H7o1Kvt-YtJIVNQg&s=Of8bAk0XKzTWJYsQg--ZN0J1HmGxU6UeZ1q65jZMNQk&e=>, or mute the thread<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AqGue8durfBU1cC7bpBrWoub1dVbryX3ks5uqVllgaJpZM4XbfUB&d=DwMCaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=KUScIrWv9DcqHOllgaD0uQ&m=7ghePg-YJ1SoumiNtku-yZNINB1H7o1Kvt-YtJIVNQg&s=rjoiZgzTYawpKPO-mLOz5rtYXIc8m1-Yqi7mvI_6a8E&e=>.

[LamForest]

Hi Tianlin-Gao,

Thanks for your reply. May I know which layer are you matching and do you fix some layers in the source/target encoder to avoid overfitting? Also, can you tell me the number/size of the hidden layer of your discriminator and which optimizer are you using, parameters for the optimizer? Or would you mind sharing your code for adda on Office?

I am sorry that this has been killing me for days. I just want to get adda work and build my model based on it. I would be really thankful if you can provide more information on that implementation.

Thanks,
Rui
[email protected]

Sorry for the delay.
I use ResNet50 for source encoder and target encoder. For both of them, layers prior to Res4a are freezed. Target encode is initialized from source encoder. The output of avgpooling is used as the input for discriminator. The discriminator has 2 hidden layer. Each of them has 500 units.
The pytorch code for discriminator is as follow:

input_dims = 2048
hidden_dims = 500
output_dims = 2
         self.layer = nn.Sequential( 
             nn.Linear(input_dims, hidden_dims),
             nn.ReLU(),
             nn.Linear(hidden_dims, hidden_dims),
             nn.ReLU(),
             nn.Linear(hidden_dims, output_dims),
             nn.LogSoftmax()
         )

Discriminator is from scratch. Source encoder is initialized from ImageNet ResNet50