doleron/yolov5-opencv-cpp-python

Example of using ultralytics YOLO V5 with OpenCV 4.5.4, C++ and Python

yolov5-opencv-cpp-python

Example of performing inference with ultralytics YOLO V5, OpenCV 4.5.4 DNN, C++ and Python

Looking for YOLO V4 OpenCV C++/Python inference? Check this repository

Code explanation

This code is explained in this medium article

Prerequisites

Make sure you have already on your system:

• Any modern Linux OS (tested on Ubuntu 20.04)
• OpenCV 4.5.4+
• Python 3.7+ (only if you are intended to run the python program)
• GCC 9.0+ (only if you are intended to run the C++ program)

IMPORTANT!!! Note that OpenCV versions prior to 4.5.4 will not work at all.

Running the python script

The python code is here.

git clone https://github.com/doleron/yolov5-opencv-cpp-python.git
cd yolov5-opencv-cpp-python
python python/yolo.py 

If your machine/OpenCV install are CUDA capable you can try out running using the GPU:

git clone https://github.com/doleron/yolov5-opencv-cpp-python.git
cd yolov5-opencv-cpp-python
python python/yolo.py cuda

Running the C++ program

The C++ code is here.

git clone https://github.com/doleron/yolov5-opencv-cpp-python.git
cd yolov5-opencv-cpp-python
g++ -O3 cpp/yolo.cpp -o yolo_example `pkg-config --cflags --libs opencv4`
./yolo_example

Or using CUDA if available:

git clone https://github.com/doleron/yolov5-opencv-cpp-python.git
cd yolov5-opencv-cpp-python
g++ -O3 cpp/yolo.cpp -o yolo_example `pkg-config --cflags --libs opencv4`
./yolo_example cuda

PS.: Video sample from https://www.youtube.com/watch?v=NyLF8nHIquM

Which YOLO version should I use?

This repository uses YOLO V5 but it is not the only YOLO version out there. You can read this article to learn more about YOLO versions and choose the more suitable one for you.

Exporting yolo v5 models to .onnx format

Check here: ultralytics/yolov5#251

My commands were:

git clone https://github.com/ultralytics/yolov5
cd yolov5
pip install -r requirements.txt

And then to convert the model:

$ python3 export.py --weights yolov5n.pt --img 640 --include onnx
export: data=data/coco128.yaml, weights=['yolov5n.pt'], imgsz=[640], batch_size=1, device=cpu, half=False, inplace=False, train=False, optimize=False, int8=False, dynamic=False, simplify=False, opset=12, verbose=False, workspace=4, nms=False, agnostic_nms=False, topk_per_class=100, topk_all=100, iou_thres=0.45, conf_thres=0.25, include=['onnx']
YOLOv5 🚀 v6.0-192-g436ffc4 torch 1.10.1+cu102 CPU

Fusing layers... 
Model Summary: 213 layers, 1867405 parameters, 0 gradients

PyTorch: starting from yolov5n.pt (4.0 MB)

ONNX: starting export with onnx 1.10.2...
/home/user/workspace/smartcam/yolov5/models/yolo.py:57: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if self.onnx_dynamic or self.grid[i].shape[2:4] != x[i].shape[2:4]:
ONNX: export success, saved as yolov5n.onnx (7.9 MB)

Export complete (1.33s)
Results saved to /home/doleron/workspace/smartcam/yolov5
Visualize with https://netron.app
Detect with `python detect.py --weights yolov5n.onnx` or `model = torch.hub.load('ultralytics/yolov5', 'custom', 'yolov5n.onnx')
Validate with `python val.py --weights yolov5n.onnx`
$ 

throubleshooting

First time I got an error with protobuf version:

"AttributeError: module 'google.protobuf.descriptor' has no attribute '_internal_create_key"?

I fixed it by running:

pip install --upgrade protobuf

References

• ultralytics/yolov5#708

yolov5-opencv-cpp-python's Issues

Share enviroments working.

certifi 2024.8.30
charset-normalizer 3.3.2
contourpy 1.3.0
cycler 0.12.1
fonttools 4.53.1
idna 3.8
importlib_resources 6.4.4
kiwisolver 1.4.7
matplotlib 3.9.2
numpy 1.26.4
onnx 1.12.0
opencv-python 4.10.0.84
packaging 24.1
pandas 1.2.0
pillow 10.4.0
pip 24.2
protobuf 3.20.1
psutil 6.0.0
py-cpuinfo 9.0.0
pyparsing 3.1.4
python-dateutil 2.9.0.post0
pytz 2024.1
PyYAML 6.0.2
requests 2.32.3
scipy 1.13.1
seaborn 0.13.2
setuptools 72.1.0
six 1.16.0
torch 1.11.0
torchaudio 0.11.0
torchtext 0.12.0
torchvision 0.12.0
tqdm 4.66.5
typing_extensions 4.12.2
tzdata 2024.1
ultralytics 8.2.89
ultralytics-thop 2.0.6
urllib3 2.2.2
wheel 0.43.0
zipp 3.20.1

undefined symbol: _ZN2cv3dnn14dnn4_v202103013NetC1Ev

./yolo_example: symbol lookup error: ./yolo_example: undefined symbol: _ZN2cv3dnn14dnn4_v202103013NetC1Ev

Fail to run the yolo_example

OpenCV 4.5.4 was installed

@doleron I need help🤕

[Fixed] Custom Weight Error : Segmentation fault

Hi, it's me again.

I exported my custom weight file and modified yolo.cpp towards the path and classes.txt towards the labels.

the error came from your code or my custom weight? or anything else to modified?

hope to your reply😍

file too short error

Hello I keep getting this error:
./yolo_example: error while loading shared libraries: /home/dwight/intel/openvino_2022/runtime/3rdparty/tbb/lib/libtbb.so: file too short
I am following the correct steps for running this repo and I have reinstalled openvino multiple times but keep getting the same error. I have changed nothing else. Any help would be appreciated thank you.

Example for the Yolo classifier

The classifier function has been officially added to the yolo repository. Could you add a usage example similar to segmentation?

Android app implementation

Thanks for your good.. how to implement the opencv on Android app.?

Python code confidence is an array not a number

Upon running the python code I am getting:

    if confidence >= 0.4:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

And when trying condfidence.any() it won't work as well. Any advice?

error: (-215:Assertion failed) inputs.size() in function 'getMemoryShapes'

Hello! I'm trying to run the code for c++ ./yolo_example code with yolov5s.onnx, which was converted
by the python command
export.py --weights yolov5s.pt --include onnx.
I get an error

[ERROR:[email protected]] global /home/sergey/opencv/modules/dnn/src/onnx/onnx_importer.cpp (906) handleNode DNN/ONNX: ERROR during processing node with 1 inputs and 1 outputs: [Identity]:(onnx::Reshape_475) from domain='ai.onnx'
terminate called after throwing an instance of 'cv::Exception'
what(): OpenCV(4.5.5) /home/sergey/opencv/modules/dnn/src/onnx/onnx_importer.cpp:928: error: (-2:Unspecified error) in function 'handleNode'
Node [[email protected]]:(onnx::Reshape_475) parse error: OpenCV(4.5.5) /home/sergey/opencv/modules/dnn/src/dnn.cpp:6017: error: (-215:Assertion failed) inputs.size() in function 'getMemoryShapes'
Aborted (core dumped)

P.S. but the native yolov5.onnx works fine.

Error exception

Hi guys

I have this error when I start the code .cpp

I don't know where this error comes from.
I trying to follow this issue : #3
but it dosen't work for me.

Anywone has a solution ?

I use my own dataset, with 1 classe.

Thx a lot ;)

detect output nothing?

Hi author, i have ran it on cpp application, with exported yolo model, but detect nothing. I use vs2022 and opencv-4.5.4-openvino-dldt-2021.4.1-vc16-avx2, what could the problem be?

是否可添加windows版本

c++的编译样例看起来是在linux上的，是否可添加windows版本

run this yolo.cpp ERROR

follow:

8fps on 3070TI

I installed opencv with CUDA with version 4.5.4

I run the file yolo.py

I get 8 - 10 fps, which is too low. I removed the imshow and the fps stayed at the same level. I have noticed that the video card has almost no load. The load is 3 - 6%. I also tried running a smaller model, but fps did not rise above 16.

I ran this on the CPU as well, and got about 4 fps. I tried to run the second script, but it didn't work.

Traceback (most recent call last):
  File "C:\Users\test\Desktop\yolov5-opencv-cpp-python-main\yolo-tiny.py", line 19, in <module>
    input_image = format_yolov5(image) # making the image square
  File "C:\Users\test\Desktop\yolov5-opencv-cpp-python-main\yolo-tiny.py", line 12, in format_yolov5
    row, col, _ = frame.shape
AttributeError: 'NoneType' object has no attribute 'shape'

How can I improve the speed?

Error launch yolo.py or yolo-tiny.py

Hello,

When i test your program, with opencv 4.5.5, i have this error.
Do you have an idear, what is the problem ?

Command :
Traceback (most recent call last):
File "python/yolo-tiny.py", line 40, in
if confidence >= 0.4:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Thanks for your answer.
Have a nice day.

the vsion of yolov5 ?

Hi, doleron.
Could you tell me which version of yoloV5 you use?
I modified data+=7, using custom weight still error.

hope to your reply.

代码里是推理mp4视频，推理图片怎么改

代码里是推理mp4视频，推理图片怎么改

on windows 11

it works very well on windows too.
just need to change cv2 to opencv-python

would it be possible to grab a live camera instead of mp4 files?

yolov5 classification

Hi,

Is it possible to run the Yolov5 classification?

Thanks

image = cv2.imread('misc/corrida-crianças-1280x720.jpg')

There is a garbled code in the picture. Please modify it.

No detection of objects using C++

Hello, I am running c++ script with "Opencv==4.6.0" and "gcc==gcc 9.4.0" and "Ubuntu == 20.04.4 LTS" in CPU - 16gb ram syatem

I tried many ways and still could not get detections in the image or video,
Even though the code does not show any errors and at the same time no boundary boxes or confidence scores around objects

output (FPS printed on it)-->>

Let me know where I went wrong...

This is my code-->>

// Include Libraries
#include <fstream>
#include <opencv2/opencv.hpp>

// Namespaces
using namespace cv;
using namespace std;
using namespace cv::dnn;

// Text parameters
const float FONT_SCALE = 0.7;
const int FONT_FACE = FONT_HERSHEY_SIMPLEX;
const int THICKNESS = 1;

// Loading label names
std::vector<std::string> load_class_list()
{
    std::vector<std::string> class_list;
    std::ifstream ifs("/home/linus/yolov5-opencv-cpp-python/classes.txt");
    std::string line;
    while (getline(ifs, line))
    {
        class_list.push_back(line);
    }
    return class_list;
}

// Loading Onnx format weight file
void load_net(cv::dnn::Net &net, bool is_cuda)
{
    auto result = cv::dnn::readNetFromONNX("/home/linus/yolov5-opencv-cpp-python/weights/yolov5s.onnx");
    
    // Using CPU or GPU based on available system
    if (is_cuda)
    {
        std::cout << "Running on GPU\n";
        result.setPreferableBackend(cv::dnn::DNN_BACKEND_CUDA);
        result.setPreferableTarget(cv::dnn::DNN_TARGET_CUDA_FP16);
    }
    else
    {
        std::cout << "Running on CPU\n";
        result.setPreferableBackend(cv::dnn::DNN_BACKEND_OPENCV);
        result.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);
    }
    net = result;
}

// Colors
const std::vector<cv::Scalar> colors = {cv::Scalar(255, 255, 0), cv::Scalar(0, 255, 0), cv::Scalar(0, 255, 255), cv::Scalar(255, 0, 0)};

// Constants
const float INPUT_WIDTH = 640.0;
const float INPUT_HEIGHT = 640.0;
const float SCORE_THRESHOLD = 0.5;
const float NMS_THRESHOLD = 0.45;
const float CONFIDENCE_THRESHOLD = 0.45;

struct Detection
{
    int class_id;
    float confidence;
    cv::Rect box;
};

cv::Mat format_yolov5(const cv::Mat &source) {
    int col = source.cols;
    int row = source.rows;
    int _max = MAX(col, row);
    cv::Mat result = cv::Mat::zeros(_max, _max, CV_8UC3);
    source.copyTo(result(cv::Rect(0, 0, col, row)));
    return result;
}

void detect(cv::Mat &image, cv::dnn::Net &net, std::vector<Detection> &output, const std::vector<std::string> &className) {
    cv::Mat blob;

    auto input_image = format_yolov5(image);
    
    // Convert to blob
    cv::dnn::blobFromImage(input_image, blob, 1./255., cv::Size(INPUT_WIDTH, INPUT_HEIGHT), cv::Scalar(), true, false);
    net.setInput(blob);
    std::vector<cv::Mat> outputs;

    // Forward propagate
    net.forward(outputs, net.getUnconnectedOutLayersNames());

    // Resizing factor
    float x_factor = input_image.cols / INPUT_WIDTH;
    float y_factor = input_image.rows / INPUT_HEIGHT;
    
    float *data = (float *)outputs[0].data;

    const int dimensions = 85;
    const int rows = 25200;

    // Initialize vectors to hold respective outputs while unwrapping detections    
    std::vector<int> class_ids;
    std::vector<float> confidences;
    std::vector<cv::Rect> boxes;

    // Iterate through 25200 detections
    for (int i = 0; i < rows; ++i) {

        float confidence = data[4];

        // Discard bad detections and continue
        if (confidence >= CONFIDENCE_THRESHOLD) {

            float * classes_scores = data + 5;

            // Create a 1x85 Mat and store class scores of 'n' no.of classes
            cv::Mat scores(1, className.size(), CV_32FC1, classes_scores);
            cv::Point class_id;

            // Perform minMaxLoc and acquire index of best class score
            double max_class_score;
            minMaxLoc(scores, 0, &max_class_score, 0, &class_id);

            // Continue if the class score is above the threshold
            if (max_class_score > SCORE_THRESHOLD) {

                // Store class ID and confidence in the pre-defined respective vectors
                confidences.push_back(confidence);

                class_ids.push_back(class_id.x);

                // Center
                float x = data[0];
                float y = data[1];

                // Box dimension
                float w = data[2];
                float h = data[3];

                // Bounding box coordinates
                int left = int((x - 0.5 * w) * x_factor);
                int top = int((y - 0.5 * h) * y_factor);
                int width = int(w * x_factor);
                int height = int(h * y_factor);

                // Store good detections in the boxes vector
                boxes.push_back(cv::Rect(left, top, width, height));
            }
        }
        // Jumping to the next column
        data += 85;
    }

    // Perform Non Maximum Suppression and draw predictions
    std::vector<int> nms_result;
    cv::dnn::NMSBoxes(boxes, confidences, SCORE_THRESHOLD, NMS_THRESHOLD, nms_result);
    for (int i = 0; i < nms_result.size(); i++) {
        int idx = nms_result[i];
        Detection result;
        result.class_id = class_ids[idx];
        result.confidence = confidences[idx];
        result.box = boxes[idx];
        output.push_back(result);
    }
}

int main(int argc, char **argv)
{

    std::vector<std::string> class_list = load_class_list();

    // Load input video/image 
    cv::Mat frame;
    cv::VideoCapture capture("/home/linus/yolov5-opencv-cpp-python/test.mp4");
    // cv::VideoCapture capture(0);
    if (!capture.isOpened())
    {
        std::cerr << "Error opening video file\n";
        return -1;
    }

    bool is_cuda = argc > 1 && strcmp(argv[1], "cuda") == 0;
    cv::dnn::Net net;
    load_net(net, is_cuda);

    auto start = std::chrono::high_resolution_clock::now();
    int frame_count = 0;
    float fps = -1;
    int total_frames = 0;

    while (true)
    {
        capture.read(frame);
        if (frame.empty())
        {
            std::cout << "End of stream\n";
            break;
        }

        std::vector<Detection> output;
        detect(frame, net, output, class_list);

        frame_count++;
        total_frames++;

        int detections = output.size();

        for (int i = 0; i < detections; ++i)
        {

            auto detection = output[i];
            auto box = detection.box;
            auto classId = detection.class_id;
            const auto color = colors[classId % colors.size()];

            // Draw bounding box
            cv::rectangle(frame, box, color, 3);
            cv::rectangle(frame, cv::Point(box.x, box.y - 20), cv::Point(box.x + box.width, box.y), color, cv::FILLED);

            // Draw class labels
            cv::putText(frame, class_list[classId].c_str(), cv::Point(box.x, box.y - 5), cv::FONT_HERSHEY_SIMPLEX, 0.5, cv::Scalar(0, 0, 0));
        }

        if (frame_count >= 10)
        {

            auto end = std::chrono::high_resolution_clock::now();
            fps = frame_count * 1000.0 / std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count();

            frame_count = 0;
            start = std::chrono::high_resolution_clock::now();
        }

        if (fps > 0)
        {
            std::ostringstream fps_label;
            fps_label << std::fixed << std::setprecision(2);
            fps_label << "FPS: " << fps;
            std::string fps_label_str = fps_label.str();
            cv::putText(frame, fps_label_str.c_str(), cv::Point(10, 25), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(0, 0, 0), 3);
        }

        cv::imshow("output", frame);
        //Initialize video writer object
        if (cv::waitKey(1) != -1)
        {
            capture.release();
            std::cout << "Done!!\n";
            break;
        }
    }

    std::cout << "Total frames: " << total_frames << "\n";
    return 0;
}

What's the diff between the yolo.py and yolo-tiny.py under the python folder?

Aborted (core dumped)

(YOLO) han@han:~/Desktop/hxb_projects/CPP_Instance/10-30/yolov5_cpp/yolov5-opencv-cpp-python$ ./yolo_example
Aborted (core dumped)

Custom model problem

Thanks for sharing but it is not working on a custom yolov5 model