stevetsui/rebnn

Resilient Binary Neural Network (ReBNN)

Pytorch implementation of our paper "Resilient Binary Neural Network" accepted by AAAI2023 as oral presentation.

Tips

Any problem, please contact the first author (Email: [email protected]).

Our code is heavily borrowed from ReActNet (https://github.com/liuzechun/ReActNet).

Dependencies

• Python 3.8
• Pytorch 1.7.1
• Torchvision 0.8.2

ReBNN with two-stage tranining

We test our ReBNN using the same ResNet-18 structure and training setttings as ReActNet, and obtain 66.9% top-1 accuracy.

Methods	Top-1 acc	Top-5 acc	Quantized model link	Log
ReActNet	65.9	-	Model	-
ReCU	66.4	86.5	Model	-
RBONN	66.7	87.0	Model	-
ReBNN	66.9	87.1	-	-

To verify the performance of our quantized models with ReActNet-like structure on ImageNet, please do as the following steps:

1. Finish the first stage training using ReActNet.
2. Use the following command:

cd 2_step2_rebnn 
bash run.sh

If you find this work useful in your research, please consider to cite:

@inproceedings{xu2023resilient,
  title={Resilient Binary Neural Network},
  author={Xu, Sheng and Li, Yanjing and Ma, Teli and Lin, Mingbao and Dong, Hao and Zhang, Baochang and Gao, Peng and Lu, Jinhu},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={37},
  number={9},
  pages={10620--10628},
  year={2023}
}

rebnn's Issues

代码疑问

你好，我对Binarization类有些疑问的地方希望您能解答一下。
①rebnn.py line79 请问weight_bin = torch.sign(weight) * bin中二值化后乘以bin是出于什么原因呢?
②rebnn.py line104 代码里的round_pass函数在输入任何数的情况下输出都是2e-4，和论文里的公式(14)不符，这块需求我猜应该是把公式(14)得到的阈值p截断到[1e-5, 2e-4]之间? 所以line67的round_pass函数这块是代码编写错误了吗?
③rebnn.py line117 论文里的公式(16)是两个梯度相加，为什么代码gradscale = grad_scale_1 - para_loss * grad_scale_2是用的减号呢?