Implement optimizations as in `ane_transformers`

[huggingface]

Implement optimizations as in `ane_transformers`

[hollance]

Good question! Hugging Face has a library, Optimum, that can do optimizations such as pruning and quantization. It seems to me that these kinds of optimizations that require "model surgery" really belong into Optimum, but I'm not aware of any plans to add these particular optimizations. But it's definitely something worth considering (and it could be prototyped in Exporters).

[dimitry12]

these kinds of optimizations that require "model surgery" really belong into Optimum, but I'm not aware of any plans to add these particular optimizations. But it's definitely something worth considering (and it could be prototyped in Exporters)

Makes sense! I can't commit to a particular timeline but I definitely plan to work in this direction. If that's ok, I will keep this issue open to post updates.