Error on PIP install

[victorbuttner]

Error on PIP install

[dcferreira]

I don't get this error on MacOS, but do get it in WSL (ubuntu in windows).

I noticed that another workaround is to install from master (then I got tokenizers==0.13.3) with pip install git+https://github.com/unitaryai/detoxify.

[Toffaa]

Hi,

I faced the same issue, that is due to the older version (0.12.1) of tokenizers used by detoxify, which does not have built version for Python 3.11 or higher, and seems to have compilation issue.

The issue seems to be fixed in higher version of tokenizers according to this

As a workaround, I downgraded my conda env to Python 3.10.

Maybe the dependancies of Detoxify should be updated ?

[dcferreira]

I couldn't reproduce this error, could you give more details?
Could you please specify which operating system, python version, and maybe the output of pip freeze if you're using pip?

[atrifat]

Confirmed, what @Toffaa said is true. @victorbuttner , you can downgrade or use Python 3.9 or Python 3.10 in your virtual environment as workaround.

[DanielDaCosta]

I encountered a similar error on macOS Monterey Version 12.5:
ERROR: Failed building wheel for tokenizers Failed to build tokenizers ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects

@dcferreira's solution of installing from master also worked: pip install git+https://github.com/unitaryai/detoxify

[laurahanu]

Updated pypi version now as well so this should be fixed!