ztmidgo/rwkv-android

使用Android cpu 运行 RWKV V4 ONNX

RWKV-Android

适用于 RWKV-World-0.4B 的模型，运行在CPU上进行推理

使用方式：下载模型 https://huggingface.co/TIEMING/rwkv-world-0.4B-onnx/tree/main ，把模型复制到Assets文件夹

下载APP安装体验(RWKV-0.4B-World-CHNtuned-INT8): https://huggingface.co/TIEMING/rwkv-world-0.4B-onnx/blob/main/world-CHNtuned-int8.apk

下载APP安装体验(RWKV-0.4B-Pile-INT8): https://drive.google.com/file/d/1Rrx8SlErId6TLCCL1SKhA3ba9o0rJCVz/view?usp=sharing

Model for RWKV-World-0.4B, running on CPU for inference

How to use: Download the model https://huggingface.co/TIEMING/rwkv-world-0.4B-onnx/tree/main and copy the model to the Assets folder

Download APP Installation Experience (RWKV-0.4B-World-CHNtuned-INT8): https://huggingface.co/TIEMING/rwkv-world-0.4B-onnx/blob/main/world-CHNtuned-int8.apk

Download APP Installation Experience (RWKV-0.4B-Pile-INT8): https://drive.google.com/file/d/1Rrx8SlErId6TLCCL1SKhA3ba9o0rJCVz/view?usp=sharing

rwkv-android's Issues

Can different models be used?

Hi, I want to use another model like redpajam, vicuna, lama, alpca? The current model is good but can we use rwkv 3b?

大佬，要在Android上跑rwkv 3b（甚至7b 量化）模型，步骤是什么样的？谢谢

首先感谢您的代码，跑起来很有用。只是我不清楚参数该如何设置？（这个问题， 可以以后再解决？）
“At present, the code only supports RWKV. Currently, due to the size limit of onnx 2gb, larger models cannot be run. However, I will try to use pytorch for Android instead of onnxruntime in the future. However, I am currently doing the porting of rwkv midi, and I have no time to make the Android version of pytorch”
————这段话的意思，要跑大小超过2gb的模型文件，必须要首先在Android上安装pytorch？我想试试这个，请问 在Android上安装pytorch 的步骤是什么样的啊？有参考链接吗？谢谢。另外，rwkv都有QQ交流群，方便留一下您的qq号码吗？谢谢

可以使用rwkv5-world模型吗，我尝试了3B的模型

转为onnx放在assets下面了，但是却得不到结果

3B的模型，转为onnx会有两个文件
quantized_RWKV_24_2048_32_18.onnx.data RWKV_24_2048_32_18.onnx
app启动后看起来无法推理