new with on stage is even better than which with two stage

[marekkowalski]

new with on stage is even better than which with two stage

[MarekKowalski]

Hi,

Those results are a bit surprising, as the error for the IBUG dataset you obtained seems to be much lower than any paper I have seen so far. Are you sure you are measuring this correctly?

If you want, you can place the models you have in some public data storage like Dropbox and send me the link, so I can re-evaluate it. This would help rule out the possibility of measurement error.

Best regards,

Marek

[mariolew]

@MarekKowalski Hi, in the training phase, did you hold the learning rate fixed to be 0.001? If I use learning rate 0.001, then the loss in stage 2 cannot drop.

[MarekKowalski]

Yes, the learning rate was kept the same. When you are training the second stage are you updating the parameters for both stages or just stage 2?

In my experience, if you are updating parameters of both stages at the same time, the error takes much longer to drop.

[mariolew]

@MarekKowalski Actually, I only update parameters for stage 2, but I can see error drop only when I lower down the learning rate.

[MarekKowalski]

That's surprising, for how long are you trying to see if the error starts droping? Also, is that in your implementation or in the one in this repo?

[mariolew]

@MarekKowalski I mean in my implementation, I only observe error drop with my learning rate lower down. And, someone has found almost the same problem in another tensorflow implementation. I've double checked the functionality of the self defined layer and found no problem, I don't know if the Adam is implemented differently in theano, or Batch Norm is implemented differently.

[MarekKowalski]

I agree that it might be because TF has a different implementation of some part of the learning algorithm or the network.

[zjjMaiMai]

@MarekKowalski @mariolew Hi
In my implementation on tensorflow, I have not found this problem.
i use a private dataset , and epoch is [15,45]

[jnulzl]

i want to know if both stages share parameters in the same layer?

[MarekKowalski]

No the parameter values are not shared between the stages.

Marek