laitathei/yolov8-onnx-rknn-horizon-tensorrt-segmentation

Inference YOLOv8 segmentation on ONNX, RKNN, Horizon and TensorRT

YOLOv8-ONNX-RKNN-HORIZON-TensorRT-Segmentation

Remark: This repo only support 1 batch size

Video source: https://www.youtube.com/watch?v=n3Dru5y3ROc&t=0s

git clone --recursive https://github.com/laitathei/YOLOv8-ONNX-RKNN-HORIZON-TensorRT-Segmentation.git

0. Environment Setting

# For onnx, rknn, horizon
torch: 1.10.1+cu102
torchvision: 0.11.2+cu102
onnx: 1.10.0
onnxruntime: 1.10.0

# For tensorrt
torch: 1.11.0+cu113
torchvision: 0.12.0+cu113
TensorRT: 8.6.1

1. Yolov8 Prerequisite

pip3 install ultralytics==8.0.147
pip3 install numpy==1.23.5

2. Convert Pytorch model to ONNX

Remember to change the variable to your setting.

python3 pytorch2onnx.py

3. RKNN Prerequisite

Install the wheel according to your python version

cd rknn-toolkit2/packages
pip3 install rknn_toolkit2-1.5.0+1fa95b5c-cpxx-cpxx-linux_x86_64.whl

4. Convert ONNX model to RKNN

Remember to change the variable to your setting To improve perfermance, you can change ./config/yolov8x-seg-xxx-xxx.quantization.cfg layer type. Please follow official document hybrid quatization part and reference to example program to modify your codes.

python3 onnx2rknn_step1.py
python3 onnx2rknn_step2.py

5. RKNN-Lite Inference

python3 rknn_lite_inference.py

6. Horizon Prerequisite

wget -c ftp://[email protected]/ai_toolchain/ai_toolchain.tar.gz --ftp-password=xj3ftp@123$%
tar -xvf ai_toolchain.tar.gz
cd ai_toolchain/
pip3 install h*

7. Convert ONNX model to Horizon

Remember to change the variable to your setting include yolov8seg_config.yaml and get onnx file from python3 pytorch2onnx.py and replace

model.export(format="onnx", imgsz=[input_height,input_width], opset=11)

sh 01_check.sh
sh 02_preprocess.sh
sh 03_build.sh

8. Horizon Inference

python3 horizion_simulator_inference.py
python3 horizion_onboard_inference.py

9. Onnx Runtime Inference

python3 onnxruntime_inference.py

10. Convert ONNX model to TensorRT

Remember to change the variable to your setting

python3 onnx2trt.py

11. TensorRT Inference

python3 tensorrt_inference.py

12. Blob Inference

Convert model from onnx to blob format via https://blobconverter.luxonis.com/

python3 blob_inference.py

Reference

https://blog.csdn.net/magic_ll/article/details/131944207
https://blog.csdn.net/weixin_45377629/article/details/124582404#t18
https://github.com/ibaiGorordo/ONNX-YOLOv8-Instance-Segmentation

yolov8-onnx-rknn-horizon-tensorrt-segmentation's Issues

Training the yolov8 neural network

Good afternoon I tried converting rknn to a .pt model trained on the original ultralitics repository. But after converting the model to rknn, there were a large number of false and incorrect predictions (this was observed on the rockchip board. On the intel processor everything worked fine). Tell me, is there any special technique for training yolov8, in which the model is successfully transformed?

TensorRT版本的代码运行bus.jpg检不出任何东西是怎么回事，没有改任何代码，模型用的是yolov8m-seg-480-640.engne

How to test accuracy

Blogger, after I complete the conversion of the rknn file, how do I test the map on a certain data set?

YOLOv8 pose

hey! i am working with yolov8pose-onnx-rknn. i don't have any experience before. Can you give me some sample as your YOLOv8--Segmentation when you have time to do.Thanks a lot!!!