devbox
小丑西瓜9
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

YOLOv5 实例分割 用 OPenCV DNN C++ 部署_opencv yolov5 实例分割

作者：小丑西瓜9 | 2024-03-16 09:46:12

赞

踩

opencv yolov5 实例分割

Pre:

如果之前从没接触过实例分割，建议先了解一下实例分割的输出是什么。

实例分割两个关键输出是：mask系数、mask原型

本文参考自该项目(这么优秀的代码当然要给star!)：GitHub - UNeedCryDear/yolov5-seg-opencv-onnxruntime-cpp: yolov5 segmentation with onnxruntime and opencv

目录

Pre:

一、代码总结

1、实例分割输出 与 目标检测输出 的区别？

2、如何获得目标的mask?

二、源码

yolov5_seg_utils.h

yolov5_seg_utils.cpp

yolo_seg.h

yolo_seg.cpp

main.cpp

三、效果

一、代码总结

1、实例分割输出 与 目标检测输出 的区别？

还是写一点实例分割部分的东西吧，互相交流下，嘿嘿。

yolov5n-seg.onnx  一张图片输入会获得两个输出，分别为

output0:  float32[1, 25200,117] 。25200是输出anchor的数量，117是4个anchor坐标信息、1个置信度、80个类别概率、32个mask系数。

output1:  float32[1,32,160,160]。即一张图片输出的一组mask原型 ，你可以理解为网络32张mask。在网络输入图上突出前景的mask，共32张，然后尺寸缩小到尺寸为160*160。

2、如何获得目标的mask?

这段代码读起来还挺废脑细胞的。

首先要指出的是，这个mask是相对于目标框的，不是相对于原图的。

获得目标框的mask：

  1. 检测框区域映射到mask原型上，然后在原型上裁剪下该区域。
  2. mask系数与目标框区域原型作矩阵乘法，其结果然后进行sigmod运算，阈值化（01二值），便获得突出目标的mask
  3. 目标mask映射到原图上

对应源码：

  1. void GetMask2(const Mat& maskProposals, const Mat& mask_protos, OutputSeg& output, const MaskParams& maskParams) {
  2.     int seg_channels = maskParams.segChannels;
  3.     int net_width = maskParams.netWidth;
  4.     int seg_width = maskParams.segWidth;
  5.     int net_height = maskParams.netHeight;
  6.     int seg_height = maskParams.segHeight;
  7.     float mask_threshold = maskParams.maskThreshold;
  8.     Vec4f params = maskParams.params;
  9.     Size src_img_shape = maskParams.srcImgShape;
  10.  
  11.     Rect temp_rect = output.box;
  12.     // 把已经到原图的检测框坐标信息  映射到  获得mask原型分支的输入尺寸上【160, 160】
  13.     int rang_x = floor((temp_rect.x * params[0] + params[2]) / net_width * seg_width);
  14.     int rang_y = floor((temp_rect.y * params[1] + params[3]) / net_height * seg_height);
  15.     int rang_w = ceil(((temp_rect.x + temp_rect.width) * params[0] + params[2]) / net_width * seg_width) - rang_x;
  16.     int rang_h = ceil(((temp_rect.y  + temp_rect.height) * params[0] + params[3]) / net_width * seg_height) - rang_y;
  17.  
  18.     //
  19.     rang_w = MAX(rang_w, 1);
  20.     rang_h = MAX(rang_h, 1);
  21.     if (rang_x + rang_w > seg_width){
  22.         if (seg_width - rang_x > 0)
  23.             rang_w =seg_width -rang_x;
  24.         else
  25.             rang_x -= 1;
  26.     }
  27.     if (rang_y + rang_h > seg_height) {
  28.         if (seg_height - rang_y > 0)
  29.             rang_h = seg_height - rang_y;
  30.         else
  31.             rang_y -= 1;
  32.     }
  33.  
  34.     vector<Range> roi_ranges;
  35.     roi_ranges.push_back(Range(0,1));
  36.     roi_ranges.push_back(Range::all());
  37.     roi_ranges.push_back(Range(rang_y, rang_h+rang_y));
  38.     roi_ranges.push_back(Range(rang_x, rang_w+rang_x));
  39.  
  40.     // 裁剪mask原型
  41.     Mat temp_mask_protos = mask_protos(roi_ranges).clone(); // 剪裁原型，保存检测框内部的原型，其余位置清零,  以此来获得感兴趣区域(roi)
  42.     Mat protos = temp_mask_protos.reshape(0, { seg_channels, rang_w*rang_h});// 检测至检测框大小？
  43.  
  44.     // mask系数与mask原型做矩阵乘法
  45.     Mat matmul_res = (maskProposals * protos).t(); // mask系数【1,32】 与 mask原型【32, h*w】进行矩阵相称
  46.     Mat masks_feature = matmul_res.reshape(1,{rang_h, rang_w}); //【1,h,w】
  47.     Mat dest, mask;
  48.  
  49.     // sigmod
  50.     cv::exp(-masks_feature, dest);
  51.     dest = 1.0 / (1.0 + dest);
  52.  
  53.     // 检测框坐标 映射到 原图尺寸
  54.     int left = floor((net_width / seg_width * rang_x - params[2]) / params[0]);
  55.     int top = floor((net_width / seg_height * rang_y - params[3]) / params[1]);
  56.     int width = ceil(net_width / seg_height * rang_w / params[0]);
  57.     int height = ceil(net_height / seg_height * rang_h / params[1]);
  58.  
  59.     // 检测框mask缩放到原图尺寸
  60.     resize(dest, mask, Size(width, height), INTER_NEAREST);
  61.  
  62.     // 阈值化
  63.     mask = mask(temp_rect - Point(left, top)) > mask_threshold;
  64.     output.boxMask = mask;
  65. }

二、源码

yolov5_seg_utils.h

  1. #pragma once
  2. #include<iostream>
  3. #include <numeric>
  4. #include<opencv2/opencv.hpp>
  5.  
  6. #define YOLO_P6 false //是否使用P6模型
  7. #define ORT_OLD_VISON 12  //ort1.12.0 之前的版本为旧版本API
  8.  
  9. struct OutputSeg {
  10.     int id;             //结果类别id
  11.     float confidence;   //结果置信度
  12.     cv::Rect box;       //矩形框
  13.     cv::Mat boxMask;       //矩形框内mask，节省内存空间和加快速度
  14. };
  15. struct MaskParams {
  16.     int segChannels = 32;
  17.     int segWidth = 160;
  18.     int segHeight = 160;
  19.     int netWidth = 640;
  20.     int netHeight = 640;
  21.     float maskThreshold = 0.5;
  22.     cv::Size srcImgShape;
  23.     cv::Vec4d params;
  24.  
  25. };
  26. bool CheckParams(int netHeight, int netWidth, const int* netStride, int strideSize);
  27. void DrawPred(cv::Mat& img, std::vector<OutputSeg> result, std::vector<std::string> classNames, std::vector<cv::Scalar> color);
  28. void LetterBox(const cv::Mat& image, cv::Mat& outImage,
  29.     cv::Vec4d& params, //[ratio_x,ratio_y,dw,dh]
  30.     const cv::Size& newShape = cv::Size(640, 640),
  31.     bool autoShape = false,
  32.     bool scaleFill = false,
  33.     bool scaleUp = true,
  34.     int stride = 32,
  35.     const cv::Scalar& color = cv::Scalar(114, 114, 114));
  36. void GetMask(const cv::Mat& maskProposals, const cv::Mat& maskProtos, std::vector<OutputSeg>& output, const MaskParams& maskParams);
  37. void GetMask2(const cv::Mat& maskProposals, const cv::Mat& maskProtos, OutputSeg& output, const MaskParams& maskParams);

yolov5_seg_utils.cpp

  1. #pragma once
  2. #include "yolov5_seg_utils.h"
  3. using namespace cv;
  4. using namespace std;
  5. bool CheckParams(int netHeight, int netWidth, const int* netStride, int strideSize) {
  6.     if (netHeight % netStride[strideSize - 1] != 0 || netWidth % netStride[strideSize - 1] != 0)
  7.     {
  8.         cout << "Error:_netHeight and _netWidth must be multiple of max stride " << netStride[strideSize - 1] << "!" << endl;
  9.         return false;
  10.     }
  11.     return true;
  12. }
  13.  
  14. void LetterBox(const cv::Mat& image, cv::Mat& outImage, cv::Vec4d& params, const cv::Size& newShape,
  15.     bool autoShape, bool scaleFill, bool scaleUp, int stride, const cv::Scalar& color)
  16. {
  17.     if (false) {
  18.         int maxLen = MAX(image.rows, image.cols);
  19.         outImage = Mat::zeros(Size(maxLen, maxLen), CV_8UC3);
  20.         image.copyTo(outImage(Rect(0, 0, image.cols, image.rows)));
  21.         params[0] = 1;
  22.         params[1] = 1;
  23.         params[3] = 0;
  24.         params[2] = 0;
  25.     }
  26.  
  27.     cv::Size shape = image.size();
  28.     float r = std::min((float)newShape.height / (float)shape.height,
  29.         (float)newShape.width / (float)shape.width);
  30.     if (!scaleUp)
  31.         r = std::min(r, 1.0f);
  32.  
  33.     float ratio[2]{ r, r };
  34.     int new_un_pad[2] = { (int)std::round((float)shape.width * r),(int)std::round((float)shape.height * r) };
  35.  
  36.     auto dw = (float)(newShape.width - new_un_pad[0]);
  37.     auto dh = (float)(newShape.height - new_un_pad[1]);
  38.  
  39.     if (autoShape)
  40.     {
  41.         dw = (float)((int)dw % stride);
  42.         dh = (float)((int)dh % stride);
  43.     }
  44.     else if (scaleFill)
  45.     {
  46.         dw = 0.0f;
  47.         dh = 0.0f;
  48.         new_un_pad[0] = newShape.width;
  49.         new_un_pad[1] = newShape.height;
  50.         ratio[0] = (float)newShape.width / (float)shape.width;
  51.         ratio[1] = (float)newShape.height / (float)shape.height;
  52.     }
  53.  
  54.     dw /= 2.0f;
  55.     dh /= 2.0f;
  56.  
  57.     if (shape.width != new_un_pad[0] && shape.height != new_un_pad[1])
  58.     {
  59.         cv::resize(image, outImage, cv::Size(new_un_pad[0], new_un_pad[1]));
  60.     }
  61.     else {
  62.         outImage = image.clone();
  63.     }
  64.  
  65.     int top = int(std::round(dh - 0.1f));
  66.     int bottom = int(std::round(dh + 0.1f));
  67.     int left = int(std::round(dw - 0.1f));
  68.     int right = int(std::round(dw + 0.1f));
  69.     params[0] = ratio[0];
  70.     params[1] = ratio[1];
  71.     params[2] = left;
  72.     params[3] = top;
  73.     cv::copyMakeBorder(outImage, outImage, top, bottom, left, right, cv::BORDER_CONSTANT, color);
  74. }
  75.  
  76. void GetMask(const cv::Mat& maskProposals, const cv::Mat& maskProtos, std::vector<OutputSeg>& output, const MaskParams& maskParams) {
  77.     //cout << maskProtos.size << endl;
  78.  
  79.     int seg_channels = maskParams.segChannels;
  80.     int net_width = maskParams.netWidth;
  81.     int seg_width = maskParams.segWidth;
  82.     int net_height = maskParams.netHeight;
  83.     int seg_height = maskParams.segHeight;
  84.     float mask_threshold = maskParams.maskThreshold;
  85.     Vec4f params = maskParams.params;
  86.     Size src_img_shape = maskParams.srcImgShape;
  87.  
  88.     Mat protos = maskProtos.reshape(0, { seg_channels,seg_width * seg_height });
  89.  
  90.     Mat matmul_res = (maskProposals * protos).t();
  91.     Mat masks = matmul_res.reshape(output.size(), { seg_width,seg_height });
  92.     vector<Mat> maskChannels;
  93.     split(masks, maskChannels);
  94.     for (int i = 0; i < output.size(); ++i) {
  95.         Mat dest, mask;
  96.         //sigmoid
  97.         cv::exp(-maskChannels[i], dest);
  98.         dest = 1.0 / (1.0 + dest);
  99.  
  100.         Rect roi(int(params[2] / net_width * seg_width), int(params[3] / net_height * seg_height), int(seg_width - params[2] / 2), int(seg_height - params[3] / 2));
  101.         dest = dest(roi);
  102.         resize(dest, mask, src_img_shape, INTER_NEAREST);
  103.  
  104.         //crop
  105.         Rect temp_rect = output[i].box;
  106.         mask = mask(temp_rect) > mask_threshold;
  107.         output[i].boxMask = mask;
  108.     }
  109. }
  110. void GetMask2(const Mat& maskProposals, const Mat& mask_protos, OutputSeg& output, const MaskParams& maskParams) {
  111.     int seg_channels = maskParams.segChannels;
  112.     int net_width = maskParams.netWidth;
  113.     int seg_width = maskParams.segWidth;
  114.     int net_height = maskParams.netHeight;
  115.     int seg_height = maskParams.segHeight;
  116.     float mask_threshold = maskParams.maskThreshold;
  117.     Vec4f params = maskParams.params;
  118.     Size src_img_shape = maskParams.srcImgShape;
  119.  
  120.     Rect temp_rect = output.box;
  121.     // 把已经到原图的检测框坐标信息  映射到  获得mask原型分支的输入尺寸上【160, 160】
  122.     int rang_x = floor((temp_rect.x * params[0] + params[2]) / net_width * seg_width);
  123.     int rang_y = floor((temp_rect.y * params[1] + params[3]) / net_height * seg_height);
  124.     int rang_w = ceil(((temp_rect.x + temp_rect.width) * params[0] + params[2]) / net_width * seg_width) - rang_x;
  125.     int rang_h = ceil(((temp_rect.y  + temp_rect.height) * params[0] + params[3]) / net_width * seg_height) - rang_y;
  126.  
  127.     //
  128.     rang_w = MAX(rang_w, 1);
  129.     rang_h = MAX(rang_h, 1);
  130.     if (rang_x + rang_w > seg_width){
  131.         if (seg_width - rang_x > 0)
  132.             rang_w =seg_width -rang_x;
  133.         else
  134.             rang_x -= 1;
  135.     }
  136.     if (rang_y + rang_h > seg_height) {
  137.         if (seg_height - rang_y > 0)
  138.             rang_h = seg_height - rang_y;
  139.         else
  140.             rang_y -= 1;
  141.     }
  142.  
  143.     vector<Range> roi_ranges;
  144.     roi_ranges.push_back(Range(0,1));
  145.     roi_ranges.push_back(Range::all());
  146.     roi_ranges.push_back(Range(rang_y, rang_h+rang_y));
  147.     roi_ranges.push_back(Range(rang_x, rang_w+rang_x));
  148.  
  149.     // 裁剪mask原型
  150.     Mat temp_mask_protos = mask_protos(roi_ranges).clone(); // 剪裁原型，保存检测框内部的原型，其余位置清零,  以此来获得感兴趣区域(roi)
  151.     Mat protos = temp_mask_protos.reshape(0, { seg_channels, rang_w*rang_h});// 检测至检测框大小？
  152.  
  153.     // mask系数与mask原型做矩阵乘法
  154.     Mat matmul_res = (maskProposals * protos).t(); // mask系数【1,32】 与 mask原型【32, h*w】进行矩阵相称
  155.     Mat masks_feature = matmul_res.reshape(1,{rang_h, rang_w}); //【1,h,w】
  156.     Mat dest, mask;
  157.  
  158.     // sigmod
  159.     cv::exp(-masks_feature, dest);
  160.     dest = 1.0 / (1.0 + dest);
  161.  
  162.     // 检测框坐标 映射到 原图尺寸
  163.     int left = floor((net_width / seg_width * rang_x - params[2]) / params[0]);
  164.     int top = floor((net_width / seg_height * rang_y - params[3]) / params[1]);
  165.     int width = ceil(net_width / seg_height * rang_w / params[0]);
  166.     int height = ceil(net_height / seg_height * rang_h / params[1]);
  167.  
  168.     // 检测框mask缩放到原图尺寸
  169.     resize(dest, mask, Size(width, height), INTER_NEAREST);
  170.  
  171.     // 阈值化
  172.     mask = mask(temp_rect - Point(left, top)) > mask_threshold;
  173.     output.boxMask = mask;
  174. }
  175.  
  176. void DrawPred(Mat& img, vector<OutputSeg> result, std::vector<std::string> classNames, vector<Scalar> color) {
  177.     Mat mask = img.clone();
  178.     for (int i=0; i< result.size(); i++){
  179.         int left, top;
  180.         left = result[i].box.x;
  181.         top = result[i].box.y;
  182.         int color_num =i;
  183.  
  184.         // 目标画框
  185.         rectangle(img, result[i].box, color[result[i].id], 2, 8);
  186.  
  187.         // 目标mask，这里非目标像素值为0
  188.         mask(result[i].box).setTo(color[result[i].id], result[i].boxMask);
  189.         string label = classNames[result[i].id] + ":" + to_string(result[i].confidence);
  190.  
  191.         // 框左上角打印信息
  192.         int baseLine;
  193.         Size labelSize = getTextSize(label, FONT_HERSHEY_SIMPLEX, 0.5, 1, &baseLine);
  194.         top = max(top, labelSize.height);
  195.         putText(img, label, Point(left, top), FONT_HERSHEY_SIMPLEX, 0.5, color[result[i].id], 2);
  196.     }
  197.  
  198.     addWeighted(img, 0.5, mask, 0.5, 0, img);
  199. }

yolo_seg.h

  1. #pragma once
  2. #include<iostream>
  3. #include<opencv2/opencv.hpp>
  4. #include "yolov5_seg_utils.h"
  5.  
  6. class YoloSeg {
  7. public:
  8.     YoloSeg() {
  9.     }
  10.     ~YoloSeg() {}
  11.     /** \brief Read onnx-model
  12.     * \param[out] read onnx file into cv::dnn::Net
  13.     * \param[in] modelPath:onnx-model path
  14.     * \param[in] isCuda:if true and opencv built with CUDA(cmake),use OpenCV-GPU,else run it on cpu.
  15.     */
  16.     bool ReadModel(cv::dnn::Net& net, std::string& netPath, bool isCuda);
  17.     /** \brief  detect.
  18.     * \param[in] srcImg:a 3-channels image.
  19.     * \param[out] output:detection results of input image.
  20.     */
  21.     bool Detect(cv::Mat& srcImg, cv::dnn::Net& net, std::vector<OutputSeg>& output);
  22.  
  23. #if(defined YOLO_P6 && YOLO_P6==true)
  24.  
  25.     const int _netWidth = 1280;  //ONNX图片输入宽度
  26.     const int _netHeight = 1280; //ONNX图片输入高度
  27.     const int _segWidth = 320;  //_segWidth=_netWidth/mask_ratio
  28.     const int _segHeight = 320;
  29.     const int _segChannels = 32;
  30. #else
  31.  
  32.     const int _netWidth = 640;   //ONNX图片输入宽度
  33.     const int _netHeight = 640;  //ONNX图片输入高度
  34.     const int _segWidth = 160;    //_segWidth=_netWidth/mask_ratio
  35.     const int _segHeight = 160;
  36.     const int _segChannels = 32;
  37.  
  38. #endif // YOLO_P6
  39.  
  40.     float _classThreshold = 0.25;
  41.     float _nmsThreshold = 0.45;
  42.     float _maskThreshold = 0.5;
  43.  
  44. public:
  45.     std::vector<std::string> _className = { "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", "truck", "boat", "traffic light",
  46.         "fire hydrant", "stop sign", "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow",
  47.         "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", "tie", "suitcase", "frisbee",
  48.         "skis", "snowboard", "sports ball", "kite", "baseball bat", "baseball glove", "skateboard", "surfboard",
  49.         "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", "bowl", "banana", "apple",
  50.         "sandwich", "orange", "broccoli", "carrot", "hot dog", "pizza", "donut", "cake", "chair", "couch",
  51.         "potted plant", "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", "keyboard", "cell phone",
  52.         "microwave", "oven", "toaster", "sink", "refrigerator", "book", "clock", "vase", "scissors", "teddy bear",
  53.         "hair drier", "toothbrush" };//类别名，换成自己的模型需要修改此项
  54. };

yolo_seg.cpp

  1. #include"yolo_seg.h"
  2. using namespace std;
  3. using namespace cv;
  4. using namespace cv::dnn;
  5.  
  6. bool YoloSeg::ReadModel(Net& net, string& netPath, bool isCuda = false) {
  7.     try {
  8.         net = readNet(netPath);
  9. #if CV_VERSION_MAJOR==4 &&CV_VERSION_MINOR==7&&CV_VERSION_REVISION==0
  10.         net.enableWinograd(false);  //bug of opencv4.7.x in AVX only platform ,https://github.com/opencv/opencv/pull/23112 and https://github.com/opencv/opencv/issues/23080
  11.         //net.enableWinograd(true);		//If your CPU supports AVX2, you can set it true to speed up
  12. #endif
  13.     }
  14.     catch (const std::exception&) {
  15.         return false;
  16.     }
  17.     if (isCuda) {
  18.         //cuda
  19.         net.setPreferableBackend(cv::dnn::DNN_BACKEND_CUDA);
  20.         net.setPreferableTarget(cv::dnn::DNN_TARGET_CUDA); //or DNN_TARGET_CUDA_FP16
  21.     }
  22.     else {
  23.         //cpu
  24.         cout << "Inference device: CPU" << endl;
  25.         net.setPreferableBackend(cv::dnn::DNN_BACKEND_DEFAULT);
  26.         net.setPreferableTarget(cv::dnn::DNN_TARGET_CPU);
  27.     }
  28.     return true;
  29. }
  30.  
  31.  
  32. bool YoloSeg::Detect(Mat& srcImg, Net& net, vector<OutputSeg>& output) {
  33.     Mat blob;
  34.     output.clear();
  35.     int col = srcImg.cols;
  36.     int row = srcImg.rows;
  37.     Mat netInputImg;
  38.     Vec4d params;
  39.     LetterBox(srcImg, netInputImg, params, cv::Size(_netWidth, _netHeight));
  40.     blobFromImage(netInputImg, blob, 1 / 255.0, cv::Size(_netWidth, _netHeight), cv::Scalar(0, 0, 0), true, false);
  41.     //**************************************************************************************************************************************************/
  42.     //如果在其他设置没有问题的情况下但是结果偏差很大，可以尝试下用下面两句语句
  43.     // If there is no problem with other settings, but results are a lot different from  Python-onnx , you can try to use the following two sentences
  44.     //
  45.     //$ blobFromImage(netInputImg, blob, 1 / 255.0, cv::Size(_netWidth, _netHeight), cv::Scalar(104, 117, 123), true, false);
  46.     //$ blobFromImage(netInputImg, blob, 1 / 255.0, cv::Size(_netWidth, _netHeight), cv::Scalar(114, 114,114), true, false);
  47.     //****************************************************************************************************************************************************/
  48.     net.setInput(blob);
  49.     std::vector<cv::Mat> net_output_img;
  50.     //*********************************************************************************************************************************
  51.     //net.forward(net_output_img, net.getUnconnectedOutLayersNames());
  52.     //opencv4.5.x和4.6.x这里输出不一致，推荐使用下面的固定名称输出
  53.     // 如果使用net.forward(net_output_img, net.getUnconnectedOutLayersNames())，需要确认下net.getUnconnectedOutLayersNames()返回值中output0在前，output1在后，否者出错
  54.     //
  55.     // The outputs of opencv4.5.x and 4.6.x are inconsistent.Please make sure "output0" is in front of "output1" if you use net.forward(net_output_img, net.getUnconnectedOutLayersNames())
  56.     //*********************************************************************************************************************************
  57.     vector<string> output_layer_names{ "output0","output1" };
  58.     net.forward(net_output_img, output_layer_names); //获取output的输出
  59.  
  60.     std::vector<int> class_ids;//结果id数组
  61.     std::vector<float> confidences;//结果每个id对应置信度数组
  62.     std::vector<cv::Rect> boxes;//每个id矩形框
  63.     std::vector<vector<float>> picked_proposals;  //output0[:,:, 5 + _className.size():net_width]===> for mask
  64.     int net_width = _className.size() + 5 + _segChannels;// 80 + 5 + 32 = 117
  65.     int out0_width= net_output_img[0].size[2];
  66.  
  67. //    assert(net_width == out0_width, "Error Wrong number of _className or _segChannels");  //模型类别数目不对或者_segChannels设置错误
  68.     int net_height = net_output_img[0].size[1];// 25200
  69.     float* pdata = (float*)net_output_img[0].data;
  70.     for (int r = 0; r < net_height; r++) {    //lines
  71.         float box_score = pdata[4];
  72.         if (box_score >= _classThreshold) {
  73.             cv::Mat scores(1, _className.size(), CV_32FC1, pdata + 5); //  可是 后面不只是有80个类别的概率；
  74.             Point classIdPoint;
  75.             double max_class_socre;
  76.             minMaxLoc(scores, 0, &max_class_socre, 0, &classIdPoint);
  77.             max_class_socre = (float)max_class_socre;
  78.             if (max_class_socre >= _classThreshold) {
  79.  
  80.                 vector<float> temp_proto(pdata + 5 + _className.size(), pdata + net_width); // Mask Coeffcients,mask的掩码系数
  81.                 picked_proposals.push_back(temp_proto);
  82.                 //rect [x,y,w,h]
  83.                 float x = (pdata[0] - params[2]) / params[0];  //x
  84.                 float y = (pdata[1] - params[3]) / params[1];  //y
  85.                 float w = pdata[2] / params[0];  //w
  86.                 float h = pdata[3] / params[1];  //h
  87.                 int left = MAX(int(x - 0.5 * w + 0.5), 0);
  88.                 int top = MAX(int(y - 0.5 * h + 0.5), 0);
  89.                 class_ids.push_back(classIdPoint.x);
  90.                 confidences.push_back(max_class_socre * box_score);
  91.                 boxes.push_back(Rect(left, top, int(w + 0.5), int(h + 0.5)));
  92.             }
  93.         }
  94.         pdata += net_width;//下一行
  95.  
  96.     }
  97.  
  98.     //NMS
  99.     vector<int> nms_result;
  100.     cv::dnn::NMSBoxes(boxes, confidences, _classThreshold, _nmsThreshold, nms_result);
  101.     std::vector<vector<float>> temp_mask_proposals;
  102.     Rect holeImgRect(0, 0, srcImg.cols, srcImg.rows);
  103.     for (int i = 0; i < nms_result.size(); ++i) {
  104.  
  105.         int idx = nms_result[i];
  106.         OutputSeg result;
  107.         result.id = class_ids[idx];
  108.         result.confidence = confidences[idx];
  109.         result.box = boxes[idx] & holeImgRect;
  110.         temp_mask_proposals.push_back(picked_proposals[idx]);
  111.         output.push_back(result);
  112.     }
  113.  
  114.     MaskParams mask_params;
  115.     mask_params.params = params;
  116.     mask_params.srcImgShape = srcImg.size();
  117.     for (int i = 0; i < temp_mask_proposals.size(); ++i) {
  118.         GetMask2(Mat(temp_mask_proposals[i]).t(), net_output_img[1], output[i], mask_params); // 注意这里是net_output_img[1]，为原型mask
  119.     }
  120.  
  121.  
  122.     //******************** ****************
  123.     // 老版本的方案，如果上面GetMask2出错，建议使用这个。
  124.     // If the GetMask2() still reports errors , it is recommended to use GetMask().
  125.     // Mat mask_proposals;
  126.     //for (int i = 0; i < temp_mask_proposals.size(); ++i)
  127.     //	mask_proposals.push_back(Mat(temp_mask_proposals[i]).t());
  128.     //GetMask(mask_proposals, net_output_img[1], output, mask_params);
  129.     //*****************************************************/
  130.  
  131.  
  132.     if (output.size())
  133.         return true;
  134.     else
  135.         return false;
  136. }

main.cpp

  1. #include <iostream>
  2. #include<opencv2/opencv.hpp>
  3. #include "yolo_seg.h"
  4.  
  5. #include<sys/time.h>
  6.  
  7. using namespace std;
  8. using namespace cv;
  9. using namespace dnn;
  10.  
  11.  
  12.  
  13.  
  14. int main() {
  15.     //yolov5(); //https://github.com/UNeedCryDear/yolov5-opencv-dnn-cpp
  16.  
  17.     string model_path = "/home/jason/PycharmProjects/pytorch_learn/yolo/yolov5-7.0/yolov5n-seg.onnx";
  18.     YoloSeg test;
  19.     Net net;
  20.     if (test.ReadModel(net, model_path, true)) {
  21.         cout << "read net ok!" << endl;
  22.     }
  23.     else {
  24.         return -1;
  25.     }
  26.     //生成随机颜色
  27.     vector<Scalar> color;
  28.     srand(time(0));
  29.     for (int i = 0; i < 80; i++) {
  30.         int b = rand() % 256;
  31.         int g = rand() % 256;
  32.         int r = rand() % 256;
  33.         color.push_back(Scalar(b, g, r));
  34.     }
  35.  
  36.  
  37.     VideoCapture capture(0);
  38.  
  39.     struct timeval t1, t2;
  40.     double timeuse;
  41.  
  42.  
  43.  
  44.     Mat img;
  45.     while (1) {
  46.         capture >> img;
  47.        vector<OutputSeg> result;
  48.  
  49.  
  50.        gettimeofday(&t1, NULL);
  51.        bool find = test.Detect(img, net, result);
  52.        gettimeofday(&t2, NULL);
  53.        if (find) {
  54.             DrawPred(img, result, test._className, color);
  55.         }
  56.         else {
  57.             cout << "Detect Failed!"<<endl;
  58.         }
  59.  
  60.        timeuse = (t2.tv_sec - t1.tv_sec) + (double)(t2.tv_usec - t1.tv_usec)/1000000; //s
  61.        string label = "duration:" + to_string(timeuse*1000); //ms
  62.        putText(img, label, Point(30,30), FONT_HERSHEY_SIMPLEX,0.5, Scalar(0,0,255), 2, 8);
  63.  
  64.  
  65.         imshow("result", img);
  66.         if(waitKey(1)=='q') break;
  67.  
  68.     }
  69.     return 0;
  70. }

三、效果

图片输入：

摄像头输入：

 Yolov5n-seg用OpenCV DNN 进行C++部署，在CPU上居然200多ms，我承认与yolov5 检测相比输出变大，运算量也会增加。但是Yolov5n-seg在Python推理才100ms左右，可能是python的矩阵运算确实很快？？有大佬说下原因吗？

觉得有帮助点个赞哦！

 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/小丑西瓜9/article/detail/248660
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号