Thank you for your work and code!
After running the command
python train.py --dataset cifar10 --target_label 0 --gpu 0
we have tried to evaluate the performance of your detector with
python Beatrix.py --dataset cifar10 --gpu 0
limiting ourselves to only checking the effect of poisoning label 0.
In particular, we have changed this code to the following
if __name__ == "__main__":
for seed_ in range (10):
print('-'*50+'seed:', seed_)
seed = seed_
torch.manual_seed(seed)
torch.cuda.manual_seed(seed)
np.random.seed(seed)
random.seed(seed)
torch.backends.cudnn.deterministic = True
opt = config.get_argument().parse_args()
os.environ["CUDA_VISIBLE_DEVICES"] = opt.gpu
for k in range(1): # range(10):
main(k)
to study the effect of different seeds on the performance.
From the attached log file, we have noticed that for some seeds, namely [3, 5, 7, 9] the value of the anomaly index for the target class 0 is not the highest.
Moreover, for some seeds, namely [0, 2, 3, 5, 7, 9], the anomaly index for class 0 appears to be below the threshold $e^2$ reported in the paper, resulting in missed detections.
These phenomena seem to appear more often than we expected.
Could you help interpreting this, and suggest what to change in case we are doing something wrong?