quic/aimet

AIMET is a library that provides advanced quantization and compression techniques for trained neural network models.

aimet's Issues

Backward compatible simplified Tensor Quantizer API updates

Add a button to get to the discussion-forums from the README

Currently the link to the discussion forums is somewhat hidden deep within the page. Most folks have been using github issues to ask questions instead.

googletest not included in ThirdParty

The CMakeLists.txt in ThirdParty looks for a googltest library. Downloading this is not mentioned as part of the Installation guide and the build fails.

Add AIMET feature guidebook and references to existing rst

Installation requires too much human input

It would be highly appreciated if you could provide a shell script that automates all the commands required to build the libraries, install and set paths. Also, it would be highly appreciated if you could include library installation procedures within the Dockerfile so that the Docker container with AIMET could be spun up in one go for experiments (instead of manually typing and debugging every line).

wrong gamma and beta for no bn case?

for bias_correction.py#L192, gamma is zero and beta is one?

Add support to handle multiple inputs in data subsampling

Update tensor quantizer session after QuantSim save and load

Refactor SpatialSvdCostCalculator to support linear layers

Test issue

This is a test.

Winnower code cleanup and docstring updates

Fix max-rank calculation in spatial svd

There is a bug when the input-channels * kernel-height > output-channels * kernel-width

Use default config file in QuantSim if none is provided

Update NOTICE file to remove unnecessary license

Usage

I used the buildandtest docker file. After it gave an interactive terminal i tried importing aimet packages but that didn't work. It threw an import error. I hope I followed the right steps. Please help me with this.

Tweaks to README and User Guide

Update link to the forums
Update references to AIMET in the docs to make it explicit that AIMET is an acronym

What is the suggested way to load the exported quantized model?

There are a .onnx file, a .pth file and a .decodings file after the export
Can a QuantizationSimModel be built from them?
Thanks!

Is there support for BatchNorm?

I get the following error 2020-05-21 16:16:35,240 - Winnow - ERROR - Unsupported op_type Mul, dotted net/Conv2d/BatchNorm/batchnorm_1/mul_1, input_ops: net/Conv2d/Conv2D]

Add support for additional ops in quantsim config file

Add code and documentation for Using Keras model with AIMET

Update Fused Batchnorm param read unit tests validation criterion

Add properties to QuantizerInfo to support TF Quantsim configuration

Export func error in OnnxSaver phase

Add support for string representation of object for TF QuantizerInfo class

Add TF quantsim utility to save and load checkpoints

does the 1% loss of resnet require fine-tuning?

Hi, I've tried the snippet of code from Compressing using Channel Pruning in auto mode on torchvision resnet50.

**********************************************************************************************
Compressed Model Statistics
Baseline model accuracy: 0.761580, Compressed model accuracy: 0.001480
Compression ratio for memory=0.731370, mac=0.432465
**********************************************************************************************

The model accuracy before (~76%) after pruning (~0.1%) is very different. Is the reported ~1% loss achieved with fine-tuning? And what is the result we should expect without fine-tuning?

Thanks!

Log AIMET package information

hot to export int8 onnx model?

I need to import onnx into the unity engine, but aimet only exports the 32 float version.
In theory, it can export int8 model by adding the information from encoding json directly to the model inference section, right?

Aimet is a great optimization tool, but I don't use the Qualcomm SDK. Can you support the above features.

Problem during Build code and install

When running: cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=ON ../aimet && make -j8

I get the following error:

-- Could NOT find GTest (missing: GTEST_LIBRARY GTEST_MAIN_LIBRARY) 
-- Configuring done
-- Generating done
-- Build files have been written to: /home/goel39/AIMET/build
[  2%] Built target gtest
[  5%] Built target MoDlCompressionCuda
[ 10%] Built target TrainingExtensionsTfCuda
[ 16%] Built target MoDlQuantizationCuda
[ 22%] Built target MoDlCompression
[ 40%] Built target MoDlEqualization
[ 59%] Built target MoDlQuantization
[ 62%] Built target gtest_main
[ 67%] Built target PyModelOptimizations
[ 70%] Built target MoDlCompressionTest
[ 83%] Built target MoDlQuantizationTest
[ 86%] Built target MoDlEqualizationTest
[ 88%] Generating build/timestamp, build
No such file or directory
TrainingExtensions/torch/CMakeFiles/TorchCppOps.dir/build.make:61: recipe for target 'TrainingExtensions/torch/build/timestamp' failed
make[2]: *** [TrainingExtensions/torch/build/timestamp] Error 1
CMakeFiles/Makefile2:760: recipe for target 'TrainingExtensions/torch/CMakeFiles/TorchCppOps.dir/all' failed
make[1]: *** [TrainingExtensions/torch/CMakeFiles/TorchCppOps.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 95%] Built target TrainingExtensionsTf
Makefile:162: recipe for target 'all' failed
make: *** [all] Error 2```

Am I missing something? I setup the docker container using the docker file successfully. 

Thanks!

Make encodings parameter part of post training tensor quantizer

Update default mode for min and max during TF quantize op creation

TF cp support for additional ops and dynamic shapes

Fix failing tf quantsim nightly test

QuantSim API error when model is on GPU

Add support to enable or disable TF Quantize ops in QuantSim model

Add quantsim config info to aimet user guide and api docs

compute scalingMatrix23

file: ModelOptimizations/DlEqualization/src/ScaleFactorCalculator.cpp:
According to the equation on line 102, the calculation on line 132 should be:
reScalingMats->scalingMatrix23.at(s) = (cubeRootMat.at(s)) * (1.0f / rangeVec2.at(s));
???

Make detecting ops in module identifier data driven

modify api to get quantizer config for TF Quantize op

Remove bn ops from update_ops when bn ops are folded

Error when using Dockerfile

Building the Docker image directly from the given Dockerfile results in an error. The error got alleviated when I replaced "pip==9.0.1" with "pip" in the Dockerfile.

Link the feature guidebook to the rest of the user guide

Issue with creating docker image

Following the instructions of aimet/USAGE.md, while building the docker, I need to specify the docker's name, and I have defined it as :
docker_image_name='img:f14'

But getting the error that the argument is invalid. I have searched about the issue, and I believe the format of the definition is correct!
Appreciate any help!

evaluate_model and train_model functions not available in the video tutorials

Hello, I'm learning about how to use the Qualcomm aimet toolkit.

I'm watching the video tutorials to reproduce the examples (specifcally this one about quantization: https://www.youtube.com/watch?v=0-i64DelNkI&t=1107s ).
In these videos, the author uses the evaluate_model and train_model functions but I did not found it for download.

Here is the code to import the functions used in the videos:

from pipelines import evaluate_model, train_model

Could you provide these functions?

Thanks in advance.

.tar file mentioned in aimet/packaging/INSTALL.txt is not found!

Trying to install aimet, followinf the install.txt instructions, but could not find the following .tar file:

aimet-release-1.7.0_build-.tar.gz

Add sampling logic for linear layers for Pytorch compression

Fix beta gamma values for first conv during analytical bias correction

Support for DepthwiseConv2d

Will there be support for DepthwiseConv2d in the future for Spatial/Weight SVD pruning and Channel Pruning?

updates to simplify Tensor Quantizer state

Documentation Related Issue

Does the library supports RNN based model, as you haven't mentioned anything about it and also not mentioned any benchmarks on a RNN based model??