A self compiled version of HuggingFace's bitsandbytes package for windows - CUDA 12.3
Quantizes large models into lower precision data types (8-bit) to run models hosted on HuggingFace locally at lower memory footprints.
Current: v0.44
Download from the release section.