buaabai/ternary-weights-network

Ternay-Weights-Network/Pytorch

Ternary-Weights-Network

This is a Pytorch implementation of Ternary-Weights-Network for the MNIST dataset.The model structure is LeNet-5. The dataset is provided by torchvision. And here are two ways I used to ternarize the weights, which correspond to main.py and second_main.py.

Requirements

• Python, Numpy
• Pytorch 0.3.1

Usage

$ git clone https://github.com/buaabai/Ternary-Weights-Network
$ python main.py --epochs 100
$ python second_main.py --epochs 100

You can use

$ python main.py -h

to check other parmeters.

How to get Ternary Weights

Here are two ways to get ternary weights. In the main.py, I use TernaryLinear layer and TernaryConv2d layer in the model.py. Both these two layers ternarize their weights during forward computing. In the second_main.py, I first use a normal LeNet-5 model, but the weights in the model were ternarized before the forward computing, and after the forward computing, the full precision weights are restored for the update operation. However, the run time of these two ways are both long(second_main.py is faster than main.py). And both of them run on a M40 gpu of NVIDIA. I don't know the reason.

ternary-weights-network's Issues

Enquiring about the model's performance on standard datasets.

Hello @buaabai. I would like to ask about the real time performance stats of your implementation. Does this converge to a satisfactory solution? What about the extent of reduction in size of the model, can you give details about that?

请问， 论文中的forward 和 backward过程中量化，但是参数更新不量化，是怎么实现的啊？

您好，在论文中说到，正如XNOR-Net一样，forward 和 backward过程中量化，但是参数更新不量化，这个过程您是怎么实现的呢？

In Python3 here should be converted into number by tensor.item()

Just a note for others

Ternary-Weights-Network/main.py

Line 107 in ccc7f94

100. * batch_idx / len(train_loader), loss.data[0]))

Ternary-Weights-Network/main.py

Line 120 in ccc7f94

test_loss += criterion(output,target).data[0]

loss.data[0] -> loss.item()
test_loss += criterion(output, target).data[0] -> test_loss += criterion(output, target).item()

Ternarize中有代码错误

Update pytorch version

It seems pytorch version is not the latest. Could you update your pytorch version?

Introduce Ternary Weight Networks (TWNs) - neural networks with weights constrained to +1, 0 and -1.

Hi. I try to get the weight (-1,0,1) based on your code and the TWNs paper, I can't get the (-1,0,1) weight, but can get the (-a,0,a). Could you give me any suggestions? Thank you so much for your time.