Invalid constructore argument StaticGridPerTensorQuantizer

[escorciav]

Invalid constructore argument StaticGridPerTensorQuantizer

[quic-mtuttle]

Hi @escorciav, I'm having some trouble reproducing this error on my end after building from develop; it might be a bit difficult to pinpoint the issue without more details on your forked repo/build changes. Do you see issues when pulling the latest develop branch and building with torch 2.1?

[quic-mtuttle]

Ah I see, if you are in a conda environment perhaps there are several other differences in your packages compared to what we build with. I think our build process is very sensitive to package versions and build flags, so probably the safest way to go would be if you could build our pytorch 2.1 docker and then build aimet inside of that.

https://github.com/quic/aimet/blob/develop/packaging/docker_install.md#build-docker-image-locally

Otherwise, we should have pytorch 2.1-compatible release out very soon.

Edit: The pytorch 2.1 dockerfile was just changed to be the default torch-gpu docker

[quic-mtuttle]

Update:
You can find torch 2.1-compatible AIMET whl files here (look for aimet_torch_<gpu/cpu>_pt21...)

You should also be able to pip install the torch 2.1-compatible release if you have Ubuntu 22.04 w/ python 3.10

apt-get install liblapacke
python3 -m pip install aimet-torch

[escorciav]

Thanks mate. I forgot to report that I managed to built it with conda succesfully.

Recipe that worked in my server:

conda install pytorch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 pytorch-cuda=11.8 -c pytorch -c nvidia
conda install pkg-config cmake
conda install -c conda-forge liblapacke
conda install eigen==3.3.7
pip install onnx==1.12.0 onnxruntime==1.15.1 jsonschema==4.19.0 spconv==2.3.6 bokeh onnxsim
echo pip install your project dependencies....

[escorciav]

Prep and QuantSim looks as:

logging.info('Preparing model from AIMET...')
model = prepare_model(model_fp32)
# out = model(input_tensor)

## Actual quantization & tweaking
quantsim_kwargs = dict(
    quant_scheme=QuantScheme.training_range_learning_with_tf_init,
    rounding_mode='nearest',
    in_place=True,
    default_param_bw=8,  # args.bit_width[0],
    default_output_bw=8,  # args.bit_width[1],
    default_data_type=QuantizationDataType.int,
)

# logging.info(f'Inferring quantsim config...')
# aimet_common_path = Path(aimet_common.__path__[0])
# quant_config = (aimet_common_path / 'quantsim_config' /
#                 'default_config_per_channel.json')
# quantsim_kwargs['config_file'] = quant_config

logging.info('Setting up QuantizationSimModel wrapper...')
# Create a new dictionary with modified keys, where '_1' is appended to each key except 'x'
model_sim = QuantizationSimModel(model=model, dummy_input=dummy_input,
                                **quantsim_kwargs)

As you can see, I even commented the per-channel quantization

[escorciav]

I also found the same issue after using:

• latest branch develop
• as well as v1.30.0

What sort of details would be relevant?

[quic-mtuttle]

Hi @escorciav, it seems likely that the failure is pybind11 related. I experimented a bit and found a couple ways to reproduce the failure in my environment:

1. Downgrading my pybind11 version below 2.6
2. Removing the cmake flag -DPYBIND11_BUILD_ABI=\\\"_cxxabi1011\\\"" in the top-level CMakeLists file

Could you check what pybind11 version you have installed in your environment?

[escorciav]

from conda list
pybind11 2.12.0 pypi_0 pypi

Confirmed: python -c "import pybind11; print(pybind11.__version__)"
2.12.0

Likewise, after cmake I find
C++ flags: ... -D_GLIBCXX_USE_CXX11_ABI=0 -DPYBIND11_BUILD_ABI=\"_cxxabi1011\" ...

[escorciav]

Does it make any sense to use this?

# set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++11 -O0 -ggdb -fPIC -D_GLIBCXX_USE_CXX11_ABI=0 -DPYBIND11_BUILD_ABI=\\\"_cxxabi1011\\\"")
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c++17 -O0 -ggdb -fPIC -D_GLIBCXX_USE_CXX11_ABI=1 -DPYBIND11_BUILD_ABI=\\\"_cxxabi1017\\\"")

It worked locally but for some reason, while building in a server those flags are ignored 😖

[escorciav]

BTW, Do you know if AIMET Pro 1.32 support Pytorch 2.1?
From QMP, afair the released date is lower than the corresponding issues.

Feel free to close the issue :)

[quic-mtuttle]

Awesome, thanks for the update. Yes AIMET pro 1.32 should support pytorch 2.1 as well.