a1600012888/yopo-you-only-propagate-once

Code for our nips19 paper: You Only Propagate Once: Accelerating Adversarial Training Via Maximal Principle

YOPO (You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle)

Code for our paper: "You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle" by Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, Bin Dong.

Our paper has been accepted by NeurIPS2019.

Prerequisites

• Pytorch==1.0.1, torchvision
• Python 3.5
• tensorboardX
• easydict
• tqdm

Intall

git clone https://github.com/a1600012888/YOPO-You-Only-Propagate-Once.git
cd YOPO-You-Only-Propagate-Once
pip3 install -r requirements.txt --user

How to run our code

Natural training and PGD training

• normal training: experiments/CIFAR10/wide34.natural
• PGD adversarial training: experiments/CIFAR10/wide34.pgd10 run python train.py -d <whcih_gpu>

You can change all the hyper-parameters in config.py. And change network in network.py Actually code in above mentioned director is very flexible and can be easiliy modified. It can be used as a template.

YOPO training

Go to directory experiments/CIFAR10/wide34.yopo-5-3 run python train.py -d <whcih_gpu>

You can change all the hyper-parameters in config.py. And change network in network.py Runing this code for the first time will dowload the dataset in ./experiments/CIFAR10/data/, you can modify the path in dataset.py

Miscellaneous

A C++ implementation by Nitin Shyamkumar is provided here! Thank you Nitin for your work!

The mainbody of experiments/CIFAR10-TRADES/baseline.res-pre18.TRADES.10step is written according to TRADES official repo

A tensorflow implementation provided by Runtian Zhai is provided here. The implemetation of the "For Free" paper is also included. It turns out that our YOPO is faster than "For Free" (detailed results will come soon). Thanks for Runtian's help!

Cite

@article{zhang2019you,
  title={You Only Propagate Once: Accelerating Adversarial Training via Maximal Principle},
  author={Zhang, Dinghuai and Zhang, Tianyuan and Lu, Yiping and Zhu, Zhanxing and Dong, Bin},
  journal={arXiv preprint arXiv:1905.00877},
  year={2019}
}

yopo-you-only-propagate-once's Issues

About training with multi-GPUs.

Hello. It doesn't use DataParallel in train.py in experiments/CIFAR10/wide34.yopo-5-3. Does it mean this can only use one gpu to run this code?

About the experiment settings of Free-m (For Free) in your paper.

Hi, your work is great! I have read your paper YOPO. I have not seen the experiment settings of Free-m (Adversarial Training for Free) in your paper. Could you please share the settings?

Gradient of first layer updated twice?

With Line 99 and Line 102 of https://github.com/a1600012888/YOPO-You-Only-Propagate-Once/blob/master/experiments/CIFAR10/pre-res18.yopo-5-3/training_function.py#L99, it seems that you are trying to reset the gradient of the first layer to its original value before BP through the whole network. However, this is ineffective, since Line 99 only keeps a reference to net.conv1.weight.grad with wgrad. As a result, Line 102 does not change the value of net.conv1.weight.grad after BP. Are these two lines just redundant or are they bugs?

CIFAR10-YOPO5-3 not Converge

Hi, I tried the code of pre-resnet yopo 5-3 on cifar 10. It seems not converge which has expanding loss. Could you help me check with it?