qengineering/yolov5-npu

YoloV5 NPU for the RK3566/68/88

YoloV5 NPU

YoloV5 for RK3566/68/88 NPU (Rock 5, Orange Pi 5, Radxa Zero 3).

Paper: https://towardsdatascience.com/yolo-v5-is-here-b668ce2a4908

Special made for the NPU, see Q-engineering deep learning examples

---

Model performance benchmark(FPS)

All models, with C++ examples can be found on the SD images.

Rock 5 with Ubuntu 22.04, OpenCV, ncnn and NPU

Radxa Zero 3 with Ubuntu 22.04, OpenCV, ncnn and NPU

All models are quantized to int8, unless otherwise noted.

demo	model_name	RK3588	RK3566/68
yolov5	yolov5s_relu	50.0	14.8
	yolov5n	58.8	19.5
	yolov5s	37.7	11.7
	yolov5m	16.2	5.7
yolov6	yolov6n	63.0	18.0
	yolov6s	29.5	8.1
	yolov6m	15.4	4.5
yolov7	yolov7-tiny	53.4	16.1
	yolov7	9.4	3.4
yolov8	yolov8n	53.1	18.2
	yolov8s	28.5	8.9
	yolov8m	12.1	4.4
yolox	yolox_s	30.0	10.0
	yolox_m	12.9	4.8
ppyoloe	ppyoloe_s	28.8	9.2
	ppyoloe_m	13.1	5.04
yolov5_seg	yolov5n-seg	9.4	1.04
	yolov5s-seg	7.8	0.87
	yolov5m-seg	6.1	0.71
yolov8_seg	yolov8n-seg	8.9	0.91
	yolov8s-seg	7.3	0.87
	yolov8m-seg	4.5	0.7
ppseg	ppseg_lite_1024x512	27.5	2.4
RetinaFace	RetinaFace_mobile3201	243.6	88.5
	RetinaFace_resnet50_3201	43.4	11.8
PPOCR-Det	ppocrv4_det2	31.5	15.1
PPOCR-Rec	ppocrv4_rec3	35.7	17.3

¹ Input size 320x320
² Input size 480x480
³ Input size 48x320, FP16

• Due to the pixel-wise filling and drawing, segmentation models are relatively slow

---

Dependencies.

To run the application, you have to:

• OpenCV 64-bit installed.
• Optional: Code::Blocks. ($ sudo apt-get install codeblocks)

Installing the dependencies.

Start with the usual

$ sudo apt-get update 
$ sudo apt-get upgrade
$ sudo apt-get install cmake wget curl

OpenCV

Follow the Raspberry Pi 4 guide.

RKNPU2

$ git clone https://github.com/airockchip/rknn-toolkit2.git

We only use a few files.

rknn-toolkit2-master
│      
└── rknpu2
    │      
    └── runtime
        │       
        └── Linux
            │      
            └── librknn_api
                ├── aarch64
                │   └── librknnrt.so
                └── include
                    ├── rknn_api.h
                    ├── rknn_custom_op.h
                    └── rknn_matmul_api.h

$ cd ~/rknn-toolkit2-master/rknpu2/runtime/Linux/librknn_api/aarch64
$ sudo cp ./librknnrt.so /usr/local/lib
$ cd ~/rknn-toolkit2-master/rknpu2/runtime/Linux/librknn_api/include
$ sudo cp ./rknn_* /usr/local/include

Save 2 GB of disk space by removing the toolkit. We do not need it anymore.

$ cd ~
$ sudo rm -rf ./rknn-toolkit2-master

---

Installing the app.

To extract and run the network in Code::Blocks

$ mkdir *MyDir* <br/>
$ cd *MyDir* <br/>
$ git clone https://github.com/Qengineering/YoloV5-NPU.git <br/>

---

Running the app.

You can use Code::Blocks.

• Load the project file *.cbp in Code::Blocks.
• Select Release, not Debug.
• Compile and run with F9.
• You can alter command line arguments with Project -> Set programs arguments...

Or use Cmake.

$ cd *MyDir*
$ mkdir build
$ cd build
$ cmake ..
$ make -j4

Make sure you use the model fitting your system.

More info or if you want to connect a camera to the app, follow the instructions at Hands-On.

---

yolov5-npu's Issues

Custom Yolo Model

Hi team,

Are you aware of how this can be implemented with custom models?

I tried converting my custom yolo models to onnx and then to rknn, but the inference logic fail because of mismatch in output layers.

Are you aware of how this can be setup for custom number of classes?

Awesome! What about yolov8?

How hard would be to have it also for Yolov8?

Where did the rknn model come from?

Hi! Can you tell me where you got the rknn model from? (yolov5s-640-640.rknn) Or if you converted a .pt model, how was it converted and where did this .pt model come from?