Finetuning script broken?

[mosaicml]

Finetuning script broken?

[dakinggg]

I believe that triton flash attention will not work on P100s. Could you try uninstalling flash_attn_triton before running anything? I think then it will fall back to torch attention properly instead of trying to use flash attention and failing.

[mscherrmann]

Thank you for your quick response! Unfortunately I do not have a flash_attn_triton package installed. I only find flash_attn, but uninstalling it doesnt help.

[dakinggg]

Apologies, I think I got the package wrong, and it's actually triton you want to uninstall. flash_attn_triton is a file in our repo. We have a try/catch around importing it, which would disable the triton attention implementation, but I guess for you the import succeeds and then it fails when it starts actually running. So I want to make that import fail so that triton is disabled.

[mscherrmann]

Did also not work for me unfortunately. However, I just switched to pretrain hf-bert, that works fine.

Thank you for your help!