Xformers does not work in Colab for Automatic1111

[thelastben]

Xformers does not work in Colab for Automatic1111

[gquarles]

@TheLastBen

Hey! I installed the repo on colab with a V100, I tossed the files in google drive

_C.so
https://drive.google.com/file/d/1Gl_9nxbom02ntBRVpEt8bYYDsBz6Wzvt/view?usp=sharing

_C_flashattention.so
https://drive.google.com/file/d/1EegDE6zEMKsqFBvTdDOFATttqdR6HUPO/view?usp=sharing

[taylorharrison]

I just got another A100. I'll run the install again and try and retrieve the files when I have a moment

[TheLastBen]

@tusharbhutt wait a few minutes before trying

[tusharbhutt]

Just snagged an A100, so will probably not work :)

I'll let you know if I can save those two files you mentioned

[taylorharrison]

@tusharbhutt @TheLastBen Just got an A100 too, ha. I'll send a link to the files when it's done, Ben. Running it now. Took me three runtimes before I got a less substantial GPU...

[TheLastBen]

Hello,
The supported GPUs are T4 and P100, but if you care to add yours (V100), you can run
pip install git+https://github.com/facebookresearch/xformers@51dd119#egg=xformers

after around 40min, and the installation is done, navigate to /usr/local/lib/python3.7/dist-packages/xformers
save the two files : "_C_flashattention.so" and "_C.so", upload them to any host and send me the link and I will integrate them in the Colab for V100 users.

Note: for V100 or equivalent, the speed increase is almost 100%

[TheLastBen]

Thank you very much, I'll integrate them immediately

[tusharbhutt]

Hello, The supported GPUs are T4 and P100, but if you care to add yours (V100), you can run pip install git+https://github.com/facebookresearch/xformers@51dd119#egg=xformers

after around 40min, and the installation is done, navigate to /usr/local/lib/python3.7/dist-packages/xformers save the two files : "_C_flashattention.so" and "_C.so", upload them to any host and send me the link and I will integrate them in the Colab for V100 users.

Note: for V100 or equivalent, the speed increase is almost 100%

Excellent, will give that I try, thank you!

[TheLastBen]

let me know if it works

[tusharbhutt]

Three times?!?! Wow, I dropped from an A100 to a P100 in one go... can't wait to see what the limits are when they're imposed tomorrow by Google.

No appearance of the xformers folder under the dist-packages yet.

[taylorharrison]

My install finished but I don't see anything with those file names. https://imgur.com/a/OIQvUN5

[TheLastBen]

@taylorharrison try the command : !ls -al [path to xformers folder] to see if the files exist

[tusharbhutt]

Same results, no folder. Not sure how to run a Unix command on a Google Web Colab

[TheLastBen]

@tusharbhutt make a new cell and run this code : !ls -al /usr/local/lib/python3.7/dist-packages/xformers/

[TheLastBen]

files beginning with "." or "_" don't show in the colab file explorer

[gquarles]

@tusharbhutt Colab file manager hides files with certain file extensions, I had to rename those files to .py to be able to see them.

I already uploaded the compiled libraries for the V100 card on colab in a message above if you want to use those, I think @TheLastBen has already included them in the repo

[TheLastBen]

@gquarles the V100 are included, but he got an A100, would be good to have the files as an addition

[gquarles]

Oh I see, I had to rename my files to get them to show up

!cp /usr/local/lib/python3.7/dist-packages/xformers/_C.so /usr/local/lib/python3.7/dist-packages/xformers/C.py
!cp /usr/local/lib/python3.7/dist-packages/xformers/_C_flashattention.so /usr/local/lib/python3.7/dist-packages/xformers/C_flashattention.py 

[tusharbhutt]

!ls -al /usr/local/lib/python3.7/dist-packages/xformers/

No go...

ls: cannot access '/usr/local/lib/python3.7/dist-packages/xformers/': No such file or directory

[TheLastBen]

@tusharbhutt it needs to be reinstalled in case you disconnected from the colab

[RaphaelMaschinsen]

Just came here because I got lucky and got an A100 myself. Thx @taylorharrison!

[taylorharrison]

@RaphaelMaschinsen don't thank me yet. Still building wheels over here. Will be nice to play with the A100 eventually though, ha...
Update: well, no luck. It disconnected on me. Got assigned another A100 though and done with work. Will do it again...

@TheLastBen finished the download. here you go: https://drive.google.com/drive/folders/15K3AGvhSfe15NIRE-rkNWYu7gKpx7oSW?usp=sharing