Question about the handwritten digit experiment

[mil-tokyo]

Question about the handwritten digit experiment

[MindExp]

In Table 1 the paper shows very high scores (more than 90%) in the digit classification tasks. So the question is that the testing set is merely the target-domain testing set or the combination of both testing sets in source and target domains. For example, in the unsupervised domain adaptation task of SVHN--> MNIST, the classification score can reach as high as 96.2%. Is the testing set for getting 96.2% ONLY MNIST TEST SET or the combination of SVHN TEST SET and MNIST TEST SET? Thanks in advance.

the testset in target domain

[AlanLuSun]

In Table 1 the paper shows very high scores (more than 90%) in the digit classification tasks. So the question is that the testing set is merely the target-domain testing set or the combination of both testing sets in source and target domains. For example, in the unsupervised domain adaptation task of SVHN--> MNIST, the classification score can reach as high as 96.2%. Is the testing set for getting 96.2% ONLY MNIST TEST SET or the combination of SVHN TEST SET and MNIST TEST SET? Thanks in advance.

the testset in target domain

I know the test set is the target domain! The major issue is the components of this test set. Just answer the test set is ONLY MNIST TEST SET or the combination of SVHN TEST SET and MNIST TEST SET? Because as for other methods, it is quite hard to reach 96.2% for the transfer task of SVHN-->MNIST and most of the methods only can reach about 80%. I guess there is some difference in the testing configurations.

[MindExp]

To the best of my knowledge, if the transfer task is SVHN to MNIST, the testset ONLY refers to MNIST TESTSET, and this can be verified in many papers and corresponding released sources code.

There is also one problem bothered me, the mnist dataset(mnist_data.mat) seems to be provided by the author, and it contains only 55000 training samples and 10000 testing samples, which is different with most related paper in SVHN to MNIST(60000 training samples, 10000 testing samples) task.

[AlanLuSun]

Yes, this is a major concern. Therefore, it is inappropriate to compare the results obtained by this experiment with those results cited from previously published paper which uses the full test set of MNIST.

[AlanLuSun]

@ksaito-ut

[Siddhinita]

I am not even getting the 67% source only baseline in the transfer task SVHN -> MNIST. Could someone please guide me how to obtain it?

[xyldmc]

I cannot download the mnist dataset ，who can help me？thanks