cornell-zhang/fracbnn

FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations

fracbnn's Issues

Regenerate ImageNet FracNet bitstream

Hi @ychzhang , you didn't release vivado files(such as design_1_wrapper.v and tcl file) when I re-build ur imagenet accelerator from HLS to Vivado. Would u mind sharing these bitstream prepare files or u can provide vivado diagram which shows PS-PL connection bewteen Francnet and ZYNQ process? thank you

Training parameter

张老师，您好！首先非常感谢您的开源，我最近阅读了您的Fracbnn论文，将网络进行训练并部署在了HLS上，但我目前还有以下一些问题，希望能得到您的解答：
（1）首先我使用了您提供的硬件参数将网络部署到HLS上，并达到了不错的准确率（95.1%）。接着我在软件上重新训练了一遍网络，参数均采用默认，epoch=260，得到了新的网络参数，并将卷积权重按照输入通道的不同拼接在一起，末位补0，形成uint64的形式，将参数装载到HLS上，发现网络对于所有输入图片的十分类结果都一样，结果为-20.5625 -6.83594 -7.55469 -5.4375 -15.1016 3.5625 -25.4453 -9.15625 -23.2344 -18.3984 ，且软件新训练的thrsehold参数值在0.1左右（与给出的threshold=2.0左右不同）我想知道其中的问题出在了哪里，是否是需要将软件训练的超参数进行一定的修改才能将其部署在HLS上。我在之后的实验发现，仅将layer3_2中的所有bn参数和全连接层参数替换为新训练参数，其余均使用预先给好的参数，就会出现以上问题。
（2）在给出的代码q.InputEncoder(input_size=(N,3,32,32), resolution=8)中resolution=8是否指的是不同bit位之间的间隔为8，那么给出的示例是否是将像素值进行 32bit量化（与论文中给出的8bit量化不同）。
（3）最后想了解在参与硬件测试的input.bin和label.bin文件的输入形式是怎样的。
期待您的答复

how to calculate LSB in code

张老师，您好，我最近阅读了您的FracBNN的论文，首先感谢您的开源，我也已在cifar-10上复现了精度。但我这里还有两个问题不太明白，向您请教下。
1：您在论文中是将一个2bit的数目分成了高1bit的数，以及低1bit的数，然后分别和1bit的权重进行XNOR，得到激活值，但是程序中貌似您计算MSB是用反量化后的2bit的输入进行了sign函数变成了-1和1，然后与1bit的权重进行卷积，最后结果乘以2/3得到MSB，而得到LSB则是反量化的2bit的输入与1bit的权重直接卷积减去MSB的值，我想问您下论文中的XNOR和代码中的卷积是如何对应的（我已阅读了PG这篇论文）。
2：如何在程序中实现将2-bit input直接split成高1bit的数和低1bit的数目，从而使得计算LSB和MSB都可以变成1bit的输入和1bit的权重相乘。
期待您的答复

软硬件参数适配

张老师，您好！首先非常感谢您的开源，我最近阅读了您的Fracbnn论文，将网络进行训练并部署在了HLS上，但我目前还有以下一些问题，希望能得到您的解答：
（1）首先我使用了您提供的硬件参数将网络部署到HLS上，并达到了不错的准确率（95.1%）。接着我在软件上重新训练了一遍网络，参数均采用默认，epoch=260，得到了新的网络参数，试图将其部署在HLS上。目前将卷积权重按照输入通道的不同拼接在一起，末位补0，形成uint64的形式，将权重参数与其余参数装载到HLS上，发现网络对于所有输入图片的十分类结果都一样，结果为-20.5625 -6.83594 -7.55469 -5.4375 -15.1016 3.5625 -25.4453 -9.15625 -23.2344 -18.3984 ，我想知道其中的问题出在了哪里，是否是需要将软件训练的超参数进行一定的修改才能将其部署在HLS上。我在之后的实验发现，仅将layer3_2中的所有bn参数和全连接层参数替换为新训练参数，其余均使用预先给好的参数，就会出现以上问题。
（2）最后想了解在参与硬件测试的input.bin和label.bin文件的输入形式是怎样的，不知道您是否方便提供。
期待您的答复

Conv_weights method? Vivado/Vitis version

Due to my limited knowledge as I am still learning, I have encountered problems when implementing the FracBNN model onto the FPGA Ultra96v2 with a custom dataset.

Specifically, what technique did you use for encoding the convolutional weights in conv_weights.h? Is it Thermometer encoding the same as the input? Can you please go into details/sample code of your encoding process in 2-bit with hex representation? as the shape is [45][16][3][3], I am quite confused into which layers' weights did you use for the conv_weights.h?

Second question, which Vivado version did you use for implementation? Was it 2019.1 or 2019.2? I tried using 2022.2 version but I have run into a lot of problems with tcl.

Thanks in advance!
Winston

Question about C-Sim simulation results

First I would like to thank you for this open source code.
I ran the C-Sim simulation using your code and the software and hardware results were both 0.
Can you explain this reason?

different result

Thanks for the great paper.

training step

Hi,
I'm trying to reproduce your work. But I have some problems. You use a two-step training in the README, but Step 3 is mentioned in your paper: "To train the FracBNN, the first two steps are the same except that the activations are quantized to 2 bits. We add a third step that initializes the weights from the second step, and applies the fractional convolutional layer". According to the paper, what is the difference between the "activation quantization to 2 bits" in the first two steps and the PGBinaryConv2d function? According to README, is the function "PGBinaryConv2d" used in each step of the two-step training? The training tricks of BNN are important, I am not sure whether I understand correctly, looking forward to your reply.

weight dimensions in pgconv.h

Dear @ychzhang,

Assume we use the same dimension notations in tb.cc for input[CHANNEL_IN][HEIGHT_IN][WIDTH_IN] and weight[CHANNEL_OUT][CHANNEL_IN][3][3]. In pgconv.h, CHANNEL_IN for inputs and weight input is 1. While such dimensions should be 16, 32, and 64 in resnet20_cifar (e.g., input[1][16][32][32], weight[16][16][3][3]) in layer1.0_conv).

I am not sure whether my understanding is incorrect, and could you please provide a brief explanation of it? Thanks in advance!

About separation MSB and LSB and quantity input？

After reading your paper, I have some doubts and hope to get your help.
First of all, what is the role of delta in the quantization process of the quantSign function? (Because your comment is to quantize the input into arbitrary bits, the result obtained by using delta here is still a float).
Then use FastSign in PGBinaryConv2d to binarize the input (based on quantSign).
At this time, have the high bits of the binarized input been separated? Why is the result of out_msb multiplied by two thirds.
Moreover, in the process of calculating FracBNN, I did not understand the separation process of MSB and LSB. Whether it is the shift operation described in the paper or the sparse operation of LSB, there is no clear expression in the code. The end result seems to be just choosing between out_msb and out_full。

Imagenet deployment

Hi @ychzhang , could u provide imagenet version of ur francBNN? As I compiled this project, it shows invalid results. Thanks,

BN layer network parameter export

Hello.
When I derived the network parameters from the pre-trained model (binput-resnet20-pg.pt), I found that the network parameters of bn layer directly derived were different from those given in HLS, and the other parameters were consistent. I wonder if additional processing is required when the bn layer parameters are exported to the hardware.

ImageNet HLS Model is Missing

You only show how to deploy the CIFAR-10 variant. Can you also add how to test with the ImageNet dataset?

Weight conversion