DefaultCPUAllocator: not enough memory

[blinkdl]

DefaultCPUAllocator: not enough memory

[BlinkDL]

Please see #40 :)

Will be much better soon.

Currently it requires huge amt (probably 40G+?) of CPU RAM to load 14B.

[BlinkDL]

Update ChatRWKV v2 & pip rwkv package (0.7.0):
Use v2/convert_model.py to convert a model for a strategy, for faster loading & saves CPU RAM.
@936384885xy

[nanonomad]

I've updated to the latest source and reinstalled the pip package. Unfortunately, convert_model.py appears to act similarly to the v2/chat.py script in terms of when it terminates. The message is "Killed" rather than a segmentation fault this time.

Does it need to be run on a machine with 40+gb ram? I attempted to run it on Google Colab as well, and it behaved the same (but iirc that is only 25gb ram)

Example launch command:
RWKV_CUDA_ON=1 python v2/convert_model.py --in RWKV-4-Pile-14B-20230313-ctx8192-test1050.pth --out RWKV-4-Pile-14B-20230313-ctx8192-test1050.converted.pth --strategy 'cuda fp16 *8+'

[BlinkDL]

I've updated to the latest source and reinstalled the pip package. Unfortunately, convert_model.py appears to act similarly to the v2/chat.py script in terms of when it terminates. The message is "Killed" rather than a segmentation fault this time.

Find a machine with lots of RAM to convert it :) Then you can load using much less RAM.