Comments (3)
dorefa paper says :'Here we assume the output of the previous layer has passed through a bounded activation function h, which ensures r ∈ [0, 1].'
But the paper does not specify what a bounded activation h is. I think multiplying activation by 0.1 can reduce the dynamic range of parameters and make the model perform better.I had an internship in megvii and they dealt with activation functions in this way.
from pytorch_quantization.
I think, DoReFa is fully integer calculation on the layers(without first, last layer).
multiplying activation by 0.1 is not quantized format, it need floating point calculation on feature map(my guess).
how do you think about it?
from pytorch_quantization.
yes, I think so.
in training time, we use simulation quantization, the activation layer inputs is the dequantize result. it's float format.
from pytorch_quantization.
Related Issues (8)
- 关于量化精度的请教 HOT 5
- 量化后的模型大小 HOT 1
- 支持转换成onnx吗? HOT 1
- 大佬,想请教一些量化方面的问题,可以留个联系方式吗?^.^ HOT 1
- ResNet50 在ImageNet数据集 HOT 4
- 你好,请问一下文中的梯度量化在哪里实现的呀,在代码中没有看到 HOT 1
- yolov5的4bit量化 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pytorch_quantization.