yu1ut/openset-da

Code for 'Open Set Domain Adaptation by Backpropagation'

openset-da's Issues

some thoughts for testing

I was just reading the paper and the code and it seems that this code may not be able to catch up with the accuracy presented in the paper.
I wonder if it is possible to replace the classification result from choosing the largest log-probability to: if the probability of Unknown(softmaxed) > (#Unknown Samples/ #Known Samples) then choose Unknown as output; else choose the largest log-probability of the known classes as the output.
For example, if we get classes [0, 1, 2, Unknown] (with 1000 samples of 0,1,2 and 3000 samples of Unknown in the test set) and their probabilities are [0.1, 0.2, 0.3, 0.4]. In this code, we would choose Unknown as our result. Instead, as the probability for Unknown is 0.4, which is lower than 0.5 (3000/6000), we should choose class 2 as our result.
I wonder if this could improve the classification accuracy.

Large gap on "usps -> mnist"

I find that there is a big gap on u2m. I have tried some tricks, but the performance is still below the paper by 4%-5%. Do you have any ideas to fix the gap?

how to choose the result

Hi, I am confused about the way to choose experiments result. In your code, you select the best one as the reported result. Is it correct? I also wonder how to choose the experiments results in deep learning. Thank you very much.

loss function problem

Thanks for your work! I find that your code of loss function have some problem which only uses "Loss_cel + Loss_abv". However, the paper proposed "Loss_cel + Loss_abv" and "Loss_cel - Loss_abv". And I get very very low accuracy for sample 4 of task "s2m". It was just about 0.005 and even lower. And task "u2m" and task “m2u” have bad results. How should I deal with it? Do you have any advice?

Very low accuracy of sample 4 for s2m

Hi,
Thanks for your work firstly! I run your code for s2m but I get very very low accuracy for sample 4. It was just about 0.008 and even lower. The accuracy for sample 0 is also low. How should I deal with it?

Access A Variable from torch.autograd.function.GradReverseBackward

I am getting a “AttributeError: ‘GradReverseBackward’ object has no attribute ‘lambd’” on the backward propagation. I modified line 65 in models.py to GradReverse(lambd).apply(x).

question about the setting of lambd(constant) for grad_reverse

In the office code of OSBP the p(constant) is set to 1 and then -1*grad to inverse the grad, but in your code this seems to be converge to 1 by the code in 88,89 in train.py
p = global_step / total_steps
constant = 2. / (1. + np.exp(-10 * p)) - 1
What is the reason for this operation and what is the meaning of grl-rampup-epochs?
thanks for your reply!

Problem about the implement of Office31

Nice work!

I try to add the implement of Office31 dataset beyond your code. But something wrong in Alexnet that I can not reach the performance as paper provide. I think the key lies in the Alexnet's poor generalization and I set the wrong super-parameter in training such as lr. Have you encountered this situation before?

Thanks much!