chengshengchan/model_compression

Implementation of model compression with knowledge distilling method.

model_compression's Issues

迭代100次后loss不在发生变化

你好。我在使用您的代码，用imagenet训练数据集，训练resnet（teacher） / queezenet(student)网络。
但是不管lr设置成0.1还是0.001，（batch_size）都是迭代140次左右loss稳定在一个值，并且不同的lr稳定的loss值都相同。 以您的经验能否指导下出现这种问题的可能原因？

Wrong lose function in Hinton method

Here instead of teacher_tau I think it should be y (placeholder variable). Because we want to give this value as input to the loss function. And in order to give input we have to use placeholder variable, isn't it.

there have a problem when i run the code

hi , when run the code ,as follow:

python teacher-student.py --task test --model trained_model
i want to know what is the folder of trained_model, and i change the code :
python teacher-student.py --task test --model trained_model to >python teacher-student.py --task test --model savemodel, but didn't make it.

Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization

I implement a TensorFlow.keras version for ICML2019 paper Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorization. Hope this would help
https://github.com/Adamdad/Samesame.git

使用KD训练student network的问题

想请问，在训练过程中，代码里面是如何保证只更新student network的权重的？
另外，在训练过程中，数据集的label应该是没有用到吧？

Teacher model performance

Hi. Thanks for sharing your code. I am wondering why your teacher's model (NIN) has 78% accuracy, which is lower than the one reported in the paper.

不管lr设置多少，迭代100次后loss不在发生变化。

你好，尝试了你的模型压缩中的知识蒸馏，的确知识蒸馏能提高学生网络的精度，但是训练保存的model文件还是很大呀，而且教师网络越大，最后蒸馏模型训练后得到的权重文件也越大。

我尝试查看了得到的学生模型权重文件，发现模型文件里依然保存有教师网络的权重。请问这算是模型压缩嘛？