uneedcrydear / yolov5-seg-opencv-onnxruntime-cpp Goto Github PK
View Code? Open in Web Editor NEWyolov5 segmentation with onnxruntime and opencv
License: Apache License 2.0
yolov5 segmentation with onnxruntime and opencv
License: Apache License 2.0
我在yolov5里训练了一个分割模型,类别只有“box“一类,导出为onnx的时候,(className已改为只有”box“一类)输出的boxScore都是很小的值,导致无输出boxes,但是训练出来(被导出)的模型我在yolov5里面是正常运行的
Hello @UNeedCryDear
I am writing to know if your software was tested in a Ubuntu 18.04 machine without GPU.
I am testing it and the onnxruntime software is returning errors, my problem for sure, just wanted to know if it was already tested in a Ubuntu 18.04 machine withou any GPU?
Best Regards.
Paulo Mendes
Deat @UNeedCryDear ,
Thank you for your work.
I have a quite an annoying exception while reading my ONNX file :
yolov5s-seg.onnx terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(4.1.1) /home/nvidia/debbuild/opencv-4.1.1/opencv/modules/dnn/src/onnx/onnx_importer.cpp:110: error: (-215:Assertion failed) !tensor_proto.raw_data().empty() || !tensor_proto.float_data().empty() || !tensor_proto.double_data().empty() || !tensor_proto.int64_data().empty() in function 'getMatFromTensor'
This exception comes when i'm calling : net = readNetFromONNX(netPath);
Do you have any idea where this error comes from? I am using opencv-4.1.1 .
FYI, I have used your command line to export the model without the --dnn argument, since they removed it.
Thank you
这个地方你已经吧mask坐标还原到了原图,width heiht计算也是有问题:
int left = floor((net_width / seg_width * rang_x - params[2]) / params[0]);
int top = floor((net_height / seg_height * rang_y - params[3]) / params[1]);
int width = ceil(net_width / seg_width * rang_w / params[0]);
int height = ceil(net_height / seg_height * rang_h / params[1]);
这里又用目标框的左上角坐标减去mask的左上角坐标,这不会出错?
mask = mask(temp_rect - Point(left, top)) > mask_threshold;
跑的时候直接崩溃
_className 未声明定义
编译过程中提示了一些报错。我确定自己安装的onnxruntime(cpu版)是安装成功的。因为我先用Yolov5的V6.1版本调用Onnxruntime成功运行了。
但是我在编译过程中出现报错,报错内容如下:无法解析的外部符号 OrtSessionOptionsAppendExecutionProvider_CUDA
需要安装visual studio吗
怎么在vs上面使用呀?
yolov5分割任务,图像分辨率1440×1080,python的predict.py脚本推理时间一张图在30ms左右,但是在c++中推理时间一张图在200ms左右,我尝试在c++推理之前将分辨率下采样,但是时间并没有什么变化,大佬帮忙看下,main.cpp如下:
#include
#include <opencv2/opencv.hpp>
#include
#include "yolo_seg.h"
#include "yolov5_seg_onnx.h"
#include <time.h>
#include <boost/filesystem.hpp>
using namespace std;
using namespace cv;
using namespace dnn;
namespace fs = boost::filesystem;
std::set uniqueClassIDs;
string results_folder = "./test_results_time"; // 结果图像保存文件夹路径
// 函数用于将绘制后的图像保存到结果文件夹
void SaveResultImage(const Mat& img, const string& img_path) {
// 确保图像为单通道图像
Mat singleChannelImg;
cvtColor(img, singleChannelImg, COLOR_GRAY2BGR);
string result_img_path = results_folder + "/" + fs::path(img_path).filename().string();
imwrite(result_img_path, singleChannelImg);
}
// 用于生成掩码图像的函数
Mat GenerateMaskImage(const vector& result, const Mat& inputImage) {
// 创建一个与原始图像相同大小的单通道掩码图像,初始化为全零
Mat maskImage = Mat::ones(inputImage.size(), CV_8UC1) * 255;
// 遍历检测到的对象
for (const OutputSeg& obj : result) {
int classID = obj.id; // 获取类别 ID
Rect bbox = obj.box; // 获取对象的边界框
obj.boxMask.forEach<uchar>([=](uchar& pixel, const int* position) -> void {
if (pixel == uchar()) {
pixel = 255;
}
else if (pixel == 255)
pixel = static_cast<uchar>(classID);
});
// 从 boxMask 复制像素值到 inputImage
obj.boxMask.copyTo(maskImage(bbox));
}
return maskImage;
}
// 批量推理所有图片
int yolov5_seg_onnx() {
string images_folder = "./images";
string model_path = "./models/best.onnx";
YoloSegOnnx test;
if (!test.ReadModel(model_path, true, 0, true)) {
cout << "Failed to read model." << endl;
return -1;
}
// 创建颜色映射表,根据类别ID为每个类别分配颜色
vector<Scalar> colorMap;
colorMap.push_back(Scalar(255, 0, 0));
colorMap.push_back(Scalar(0, 255, 0));
// 遍历 images 文件夹下的所有图片文件
for (const auto& entry : fs::directory_iterator(images_folder)) {
if (fs::is_regular_file(entry) && entry.path().extension() == ".png") {
string img_path = entry.path().string();
cout << "Processing image: " << img_path << endl;
vector<OutputSeg> result;
Mat img = imread(img_path);
// 下采样到 480x640
Mat resized_img;
resize(img, resized_img, Size(480, 640));
// 启动计时器
auto start = chrono::high_resolution_clock::now();
if (test.OnnxDetect(resized_img, result)) {
// 在这里添加生成掩码图像的代码
Mat maskImage = GenerateMaskImage(result, resized_img);
// 将掩码图像恢复至原始分辨率
Mat final_maskImage;
resize(maskImage, final_maskImage, img.size());
// 将掩码图像保存到文件
SaveResultImage(final_maskImage, img_path);
}
else {
cout << "Failed to detect objects in image: " << img_path << endl;
}
// 结束计时器
auto end = chrono::high_resolution_clock::now();
// 计算处理时间(毫秒)
auto elapsed_ms = chrono::duration_cast<chrono::milliseconds>(end - start);
cout << "Image processed in " << elapsed_ms.count() << " milliseconds." << endl;
}
}
return 0;
}
int main() {
if (!fs::exists(results_folder)) {
fs::create_directory(results_folder);
}
yolov5_seg_onnx(); // OnnxRuntime, support dynamic!
return 0;
}
could parem the yolov7 mask with dnn?
您好,我使用您的代码通过onnx推理yolov5实例分割,加载模型后内存占用会超过4个G,在推理时最高接近7个G,请问这个内存占用是正常的吗?
Your code was running properly before the following merged PR: opencv/opencv#22593
Could you confirm that there is indeed a regression in OpenCV due to this PR?
I tested your code with yolov5s-seg.onnx under Visual Studio 17.1.2
2023-10-17 09:47:02.711437464 [E:onnxruntime:Yolov5-Seg, allocatormgr.cc:44 CreateAllocator] Received invalid value of arena_extend_strategy 12684664
Segmentation fault (core dumped)
Ubuntu 20.04运行make指令报错:
[100%] Linking CXX executable YOLOv5
/usr/bin/ld: CMakeFiles/YOLOv5.dir/yolov5_seg_onnx.cpp.o: in function YoloSegOnnx::ReadModel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, int, bool)': yolov5_seg_onnx.cpp:(.text+0x219): undefined reference to
OrtSessionOptionsAppendExecutionProvider_CUDA'
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/YOLOv5.dir/build.make:325: YOLOv5] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/YOLOv5.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
我将#define ORT_OLD_VISON 12改为#define ORT_OLD_VISON 13后还是报这个错误
我使用onnxruntime推理出来的结果,很烂
这是使用您的dnn推理出来的:
最多的区别是预处理部分和推理完后数据格式部分:
onnxruntime预处理部分:
1、转rgb格式:
cv::Mat canvas;
cv::cvtColor(mat_rs, canvas, cv::COLOR_BGR2RGB);
2、然后归一化
normalize_inplace(canvas, 0, 1/255);
void normalize_inplace(cv::Mat& mat_inplace, const float* mean, const float* scale)
{
if (mat_inplace.type() != CV_32FC3) mat_inplace.convertTo(mat_inplace, CV_32FC3);
for (unsigned int i = 0; i < mat_inplace.rows; ++i)
{
cv::Vec3f* p = mat_inplace.ptrcv::Vec3f(i);
for (unsigned int j = 0; j < mat_inplace.cols; ++j)
{
p[j][0] = (p[j][0] - mean[0]) * scale[0];
p[j][1] = (p[j][1] - mean[1]) * scale[1];
p[j][2] = (p[j][2] - mean[2]) * scale[2];
}
}
}
3、推理部分结果处理:
由于推理出来的数据是float*格式的,但是dnn推理出来的结果直接是cv::Mat格式
对于淹码数据部分:
const float* mask_buffer = outputTensors[1].GetTensorMutableData<float>();
然后,把它格式化成4维数据:
int mask_size[4] = { mask_bat, mask_channel, mask_rows, mask_cols };
cv::Mat mask_protos = cv::Mat::zeros(4, mask_size, CV_32FC1);
memcpy(mask_protos.data, mask_buffer, mask_bat * mask_channel * mask_rows * mask_cols);
其他部分,跟dnn一样,但是结果就是不一样,很奇怪
作者您好:
请问在是否onnxruntime以及是否GPU环境下,yolov5m-seg模型的运行时间大概是多少?
谢谢
您好,我正在尝试移植您的yolov5-seg_onnx.cpp到自己的程序里,我是个c++小白,在移植过程中遇到了如下问题:引发了异常: 读取访问权限冲突。 Ort::GetApi(...) 返回 nullptr。
创建部分
int yolov5_seg_onnx()
{
string img_path = "../test.jpg";
string model_path = "../best.onnx";
YoloSegOnnx test; // 这边报错了: **Ort::GetApi**(...) 返回 nullptr。
//Net net;
if (test.ReadModel(model_path, true, 0, true)) {
cout << "read net ok!" << endl;
}
else {
return -1;
}
//生成随机颜色
vector<Scalar> color;
srand(time(0));
for (int i = 0; i < 80; i++) {
int b = rand() % 256;
int g = rand() % 256;
int r = rand() % 256;
color.push_back(Scalar(b, g, r));
}
vector<OutputSeg> result;
Mat img = imread(img_path);
//clock_t t1, t2;
if (test.OnnxDetect(img, result)) {
DrawPred(img, result, test._className, color);
}
else {
cout << "Detect Failed!" << endl;
}
system("pause");
return 0;
}
在debug的时候问题定位到了yolov5-seg_onnx.h文件下的60行这里: Ort::Env _OrtEnv = Ort::Env(OrtLoggingLevel::ORT_LOGGING_LEVEL_ERROR, "Yolov5-Seg");
报错信息如下:
引发了异常: 读取访问权限冲突。
Ort::GetApi(...) 返回 nullptr。
还望您解答,若有需要可以将整个项目发给您,万分感谢
Hi, I trained a model and export it to onnx model. The onnx model takes (1,3,1376,1376) input and its outputs are (1, 116487, 38) and (1, 32, 344, 344).
I have changed these part in yolo_seg.h like the given image below.
and when I run the code I get this error
When I export the same model with input size (1,3,640,640) it runs without errors. But when I change the input sizes it throws this error. Please help!
Thank you.
在代码中有_strideSize的这个参数,想知道这个参数的大小是根据什么来定的,还是说这个_strideSize为固定的size=3?我在更换了我自己训练的数据集以后,改了 _netWidth 、_netHeight 、_segWidth = 160; _segHeight 后会报错。
net.forward(netOutputImg, outputLayerName); //获取output的输出
netOutputImg[1]为mask数据,数据大小应该为:132160*160,但是这里这个维度应该是咋样的?
正常推理处理这个mask数据应该是一维数组,怎么变成相应维度呢?
另外,进行GetMask2的时候:
for (int i = 0; i < temp_mask_proposals.size(); ++i) {
GetMask2(Mat(temp_mask_proposals[i]).t(), netOutputImg[1], params, SrcImg.size(), output[i]);
}
里面:
vectorcv::Range roi_rangs;
roi_rangs.push_back(cv::Range(0, 1));
roi_rangs.push_back(cv::Range::all());
roi_rangs.push_back(cv::Range(rang_y, rang_h + rang_y));
roi_rangs.push_back(cv::Range(rang_x, rang_w + rang_x));
//crop
cv::Mat temp_mask_protos = maskmat(roi_rangs).clone();
这个cv::Range::all()是啥意思,这个maskmat维度不对,直接会导致maskmat(roi_rangs).clone()崩溃!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.