当前位置:   article > 正文

【OpenCV】C++红绿灯轮廓识别+ROS话题实现_ros中使用opencv检测红绿灯

ros中使用opencv检测红绿灯

目录

前言

一、背景知识

Opencv轮廓检测

ROS相关知识

二、环境依赖

三、具体实现

Step1:初始化ROS,订阅话题

Step2:接收话题,进入回调

1. 帧处理 

2. 膨胀腐蚀处理

Step3:红绿特征处理

1. 提取绘制轮廓

2. 转换矩形、排序

3. 显示检测结果

四、完整代码

五、使用方法

CMakeLists.txt

 package.xml

detect.launch

六、后续改进思路 


前言

根据需求需要使用Opencv实现红绿灯检测的功能,于是在猿力猪大佬的【OpenCV】红绿灯识别 轮廓识别 C++ OpenCV 案例实现 文章的基础上,将Opencv 3中的写法改成了Opencv 4,在具体图片处理的部分也按照我自己的逻辑进行了一些改动,并写成ROS工作空间包含了完整的话题读取,图片处理及检测结果显示。

一、背景知识

Opencv轮廓检测

这个部分主要用到Opencv中的findContours函数,具体介绍可以参考:findContours函数参数详解,关于轮廓检测的基本概念参考上面提到的猿力猪大佬的博文即可。

ROS相关知识

ROS编译方式:[详细教程]使用ros编译运行自己写的程序

ROS多节点运行:ROS中的roslaunch命令和launch文件(ROS入门学习笔记四)

ROS话题订阅:ROS消息发布(publish)与订阅(subscribe)(C++代码详解)

二、环境依赖

  • OpenCV 4
  • cv_bridge

三、具体实现

Step1:初始化ROS,订阅话题

  1. int main(int argc, char **argv)
  2. {
  3. ros::init(argc, argv, "tld_cv_node");
  4. ros::NodeHandle nh;
  5. std::string image_topic;
  6. nh.param<std::string>("sub_topic", image_topic, "/src_rgb/compressed");
  7. std::cout << "image_topic: " << image_topic << std::endl;
  8. ros::Subscriber camera_sub =
  9. nh.subscribe(image_topic, 2, camera_callback);
  10. ros::spin();
  11. ros::waitForShutdown();
  12. return 0;
  13. }

Step2:接收话题,进入回调

1. 帧处理 

  • 从图像话题中读取图像并转换为BGR格式,调整亮度,而后转为YCrCb格式,提取ROI,根据红绿阈值拆分红色和绿色分量
  1. cv_bridge::CvImagePtr cv_ptr =
  2. cv_bridge::toCvCopy(msg_pic, sensor_msgs::image_encodings::BGR8);
  3. if (rotated)
  4. {
  5. cv::flip(cv_ptr->image, src_image, -1);
  6. }
  7. else
  8. {
  9. src_image = cv_ptr->image;
  10. }
  11. // std::cout << "src_image" << src_image.size() << std::endl;
  12. // 亮度参数
  13. double a = 0.3;
  14. double b = (1 - a) * 125;
  15. // 统计检测用时
  16. clock_t start, end;
  17. start = clock();
  18. src_image.copyTo(frame);
  19. // 调整亮度
  20. src_image.convertTo(img, img.type(), a, b);
  21. // cv::imshow("img",img);
  22. // 使用ROI(感兴趣区域)方式截取图像
  23. cv::Rect roi(0, 0, 2048, 768); // 定义roi,图片尺寸2048*1536
  24. // std::cout << "img size:" << img.size() << std::endl;
  25. cv::Mat roi_image = img(roi);
  26. // 转换为YCrCb颜色空间
  27. cvtColor(roi_image, imgYCrCb, cv::COLOR_BGR2YCrCb);
  28. // cvtColor(img, imgYCrCb, cv::COLOR_BGR2YCrCb);
  29. imgRed.create(imgYCrCb.rows, imgYCrCb.cols, CV_8UC1);
  30. imgGreen.create(imgYCrCb.rows, imgYCrCb.cols, CV_8UC1);
  31. // 分解YCrCb的三个成分
  32. std::vector<cv::Mat> planes;
  33. split(imgYCrCb, planes);
  34. // 遍历以根据Cr分量拆分红色和绿色
  35. cv::MatIterator_<uchar> it_Cr = planes[1].begin<uchar>(),
  36. it_Cr_end = planes[1].end<uchar>();
  37. cv::MatIterator_<uchar> it_Red = imgRed.begin<uchar>();
  38. cv::MatIterator_<uchar> it_Green = imgGreen.begin<uchar>();
  39. for (; it_Cr != it_Cr_end; ++it_Cr, ++it_Red, ++it_Green)
  40. {
  41. // RED, 145<Cr<470 红色
  42. // if (*it_Cr > 145 && *it_Cr < 470)
  43. if (*it_Cr > 140 && *it_Cr < 470)
  44. *it_Red = 255;
  45. else
  46. *it_Red = 0;
  47. // GREEN 95<Cr<110 绿色
  48. if (*it_Cr > 95 && *it_Cr < 110)
  49. *it_Green = 255;
  50. else
  51. *it_Green = 0;
  52. // YELLOW 黄色
  53. }

PS:ROI选取这里只是随意截取了图片的上半部分。

2. 膨胀腐蚀处理

  • 膨胀的第三个参数:膨胀操作的内核,我根据实际场景的检测效果进行了修改
  1. // 膨胀和腐蚀
  2. dilate(imgRed, imgRed, cv::Mat(8, 8, CV_8UC1), cv::Point(-1, -1));
  3. erode(imgRed, imgRed, cv::Mat(1, 1, CV_8UC1), cv::Point(-1, -1));
  4. dilate(imgGreen, imgGreen, cv::Mat(12, 12, CV_8UC1), cv::Point(-1, -1));
  5. erode(imgGreen, imgGreen, cv::Mat(1, 1, CV_8UC1), cv::Point(-1, -1));

Step3:红绿特征处理

  • 这是我改动最大的一个函数,只保留了原作者提取轮廓转换为矩形的思路。先提取、绘制轮廓、显示检测结果,然后对得到的矩形进行位置排序,再对轮廓依次进行显示。

1. 提取绘制轮廓

  1. // 提取轮廓
  2. findContours(tmp_Red, contours_Red, hierarchy_Red, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
  3. findContours(tmp_Green, contours_Green, hierarchy_Green, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
  4. // 绘制轮廓
  5. drawContours(frame, contours_Red, -1, cv::Scalar(0, 0, 255), cv::FILLED); // Red
  6. std::cout << "提取到的红色轮廓数量:" << contours_Red.size() << std::endl;
  7. drawContours(frame, contours_Green, -1, cv::Scalar(0, 255, 0), cv::FILLED); // Green
  8. std::cout << "提取到的绿色轮廓数量:" << contours_Green.size() << std::endl;
  9. // 显示轮廓
  10. // imshow("contours", frame);
  11. trackBox_Red = new cv::Rect[contours_Red.size()];
  12. trackBox_Green = new cv::Rect[contours_Green.size()];

2. 转换矩形、排序

  1. // 确定要跟踪的区域
  2. for (int i = 0; i < contours_Red.size(); i++)
  3. {
  4. // Yi opencv4 不支持 CvSeq
  5. trackBox_Red[i] = cv::boundingRect(contours_Red[i]);
  6. }
  7. for (int i = 0; i < contours_Green.size(); i++)
  8. {
  9. // Yi opencv4 不支持 CvSeq
  10. trackBox_Green[i] = cv::boundingRect(contours_Green[i]);
  11. }
  12. // imshow("contours", tmp);
  13. // Rect.tl() 返回矩形左上顶点的坐标
  14. for (int i = 0; i < contours_Red.size(); i++)
  15. {
  16. Store_x_color a;
  17. a.x = trackBox_Red[i].tl().x;
  18. a.y = trackBox_Red[i].tl().y;
  19. a.color = 0;
  20. x_color.push_back(a);
  21. }
  22. for (int i = 0; i < contours_Green.size(); i++)
  23. {
  24. Store_x_color a;
  25. a.x = trackBox_Green[i].tl().x;
  26. a.y = trackBox_Green[i].tl().y;
  27. a.color = 1;
  28. x_color.push_back(a);
  29. }
  30. // 清空指针
  31. delete[] trackBox_Red;
  32. delete[] trackBox_Green;
  33. // 对左上顶点横坐标进行排序
  34. sort(x_color.begin(), x_color.end(), CompareByX);

3. 显示检测结果

  1. // 显示结果
  2. for (int i = 0; i < x_color.size(); i++)
  3. {
  4. if (0 == x_color[i].color)
  5. cv::putText(frame, "Red", cv::Point(x_color[i].x, x_color[i].y - 25), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(0, 0, 255), 2, 8, 0);
  6. else if (1 == x_color[i].color)
  7. cv::putText(frame, "Green", cv::Point(x_color[i].x, x_color[i].y - 25), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(0, 255, 0), 2, 8, 0);
  8. else if (2 == x_color[i].color)
  9. cv::putText(frame, "Yellow", cv::Point(x_color[i].x, x_color[i].y - 25), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(0, 255, 255), 2, 8, 0);
  10. else
  11. cv::putText(frame, "Lights off", cv::Point(x_color[i].x, x_color[i].y - 25), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(255, 255, 255), 2, 8, 0);
  12. }
  13. cv::namedWindow("tld_result", 0);
  14. cv::resizeWindow("tld_result", 1920, 1080);
  15. cv::imshow("tld_result", frame);
  16. cv::waitKey(1);

实际检测结果如下图所示: 

  

四、完整代码

  1. /*
  2. * @CopyRight: All Rights Reserved by Plusgo
  3. * @Author: Yi
  4. * @E-mail: waterwinsor@gmail.com
  5. * @Date: 2023年 05月 06日 星期六 15:44:35
  6. * @LastEditTime: 2023年 05月 08日 星期一 10:07:30
  7. */
  8. // requirements: opencv 4
  9. #include <iostream>
  10. #include <fstream>
  11. #include <time.h>
  12. #include <algorithm>
  13. #include <cv_bridge/cv_bridge.h>
  14. #include <image_transport/image_transport.h>
  15. #include <ros/ros.h>
  16. #include <sensor_msgs/Image.h>
  17. #include <sensor_msgs/PointCloud2.h>
  18. #include <opencv2/opencv.hpp>
  19. #include "opencv2/imgproc.hpp"
  20. #include <opencv2/imgproc/types_c.h>
  21. struct Store_x_color
  22. {
  23. int x; // 存储左上顶点横坐标
  24. int y; // 存储左上顶点纵坐标
  25. int color; // 存储当前点颜色
  26. };
  27. // Function headers
  28. void processImg(cv::Mat, cv::Mat); // 前红后绿
  29. bool CompareByX(const Store_x_color &, const Store_x_color &);
  30. // Global variables
  31. cv::Mat src_image;
  32. bool rotated = true; // rotate 180
  33. cv::Mat frame;
  34. cv::Mat img;
  35. cv::Mat imgYCrCb;
  36. cv::Mat imgGreen;
  37. cv::Mat imgRed;
  38. cv::Mat imgYellow;
  39. std::vector<Store_x_color> x_color;
  40. void camera_callback(const sensor_msgs::CompressedImageConstPtr &msg_pic)
  41. {
  42. try
  43. {
  44. cv_bridge::CvImagePtr cv_ptr =
  45. cv_bridge::toCvCopy(msg_pic, sensor_msgs::image_encodings::BGR8);
  46. if (rotated)
  47. {
  48. cv::flip(cv_ptr->image, src_image, -1);
  49. }
  50. else
  51. {
  52. src_image = cv_ptr->image;
  53. }
  54. // std::cout << "src_image" << src_image.size() << std::endl;
  55. // 亮度参数
  56. double a = 0.3;
  57. double b = (1 - a) * 125;
  58. // 统计检测用时
  59. clock_t start, end;
  60. start = clock();
  61. src_image.copyTo(frame);
  62. // 调整亮度
  63. src_image.convertTo(img, img.type(), a, b);
  64. // cv::imshow("img",img);
  65. // 使用ROI(感兴趣区域)方式截取图像
  66. cv::Rect roi(0, 0, 2048, 768); // 定义roi,图片尺寸2048*1536
  67. // std::cout << "img size:" << img.size() << std::endl;
  68. cv::Mat roi_image = img(roi);
  69. // 转换为YCrCb颜色空间
  70. cvtColor(roi_image, imgYCrCb, cv::COLOR_BGR2YCrCb);
  71. // cvtColor(img, imgYCrCb, cv::COLOR_BGR2YCrCb);
  72. imgRed.create(imgYCrCb.rows, imgYCrCb.cols, CV_8UC1);
  73. imgGreen.create(imgYCrCb.rows, imgYCrCb.cols, CV_8UC1);
  74. // 分解YCrCb的三个成分
  75. std::vector<cv::Mat> planes;
  76. split(imgYCrCb, planes);
  77. // 遍历以根据Cr分量拆分红色和绿色
  78. cv::MatIterator_<uchar> it_Cr = planes[1].begin<uchar>(),
  79. it_Cr_end = planes[1].end<uchar>();
  80. cv::MatIterator_<uchar> it_Red = imgRed.begin<uchar>();
  81. cv::MatIterator_<uchar> it_Green = imgGreen.begin<uchar>();
  82. for (; it_Cr != it_Cr_end; ++it_Cr, ++it_Red, ++it_Green)
  83. {
  84. // RED, 145<Cr<470 红色
  85. // if (*it_Cr > 145 && *it_Cr < 470)
  86. if (*it_Cr > 140 && *it_Cr < 470)
  87. *it_Red = 255;
  88. else
  89. *it_Red = 0;
  90. // GREEN 95<Cr<110 绿色
  91. if (*it_Cr > 95 && *it_Cr < 110)
  92. *it_Green = 255;
  93. else
  94. *it_Green = 0;
  95. // YELLOW 黄色
  96. }
  97. // 膨胀和腐蚀
  98. dilate(imgRed, imgRed, cv::Mat(8, 8, CV_8UC1), cv::Point(-1, -1));
  99. erode(imgRed, imgRed, cv::Mat(1, 1, CV_8UC1), cv::Point(-1, -1));
  100. dilate(imgGreen, imgGreen, cv::Mat(12, 12, CV_8UC1), cv::Point(-1, -1));
  101. erode(imgGreen, imgGreen, cv::Mat(1, 1, CV_8UC1), cv::Point(-1, -1));
  102. // 检测和显示
  103. processImg(imgRed, imgGreen);
  104. // 清空x_color
  105. x_color.clear();
  106. end = clock();
  107. std::cout << "检测时间:" << (double)(end - start) / CLOCKS_PER_SEC << std::endl; // 打印检测时间
  108. }
  109. catch (cv_bridge::Exception e)
  110. {
  111. ROS_ERROR_STREAM("cant't get image");
  112. }
  113. }
  114. int main(int argc, char **argv)
  115. {
  116. ros::init(argc, argv, "tld_cv_node");
  117. ros::NodeHandle nh;
  118. std::string image_topic;
  119. nh.param<std::string>("sub_topic", image_topic, "/src_rgb/compressed");
  120. std::cout << "image_topic: " << image_topic << std::endl;
  121. ros::Subscriber camera_sub =
  122. nh.subscribe(image_topic, 2, camera_callback);
  123. ros::spin();
  124. ros::waitForShutdown();
  125. return 0;
  126. }
  127. void processImg(cv::Mat src_Red, cv::Mat src_Green)
  128. {
  129. cv::Mat tmp_Red;
  130. cv::Mat tmp_Green;
  131. std::vector<std::vector<cv::Point>> contours_Red;
  132. std::vector<std::vector<cv::Point>> contours_Green;
  133. std::vector<cv::Vec4i> hierarchy_Red;
  134. std::vector<cv::Vec4i> hierarchy_Green;
  135. cv::Rect *trackBox_Red;
  136. cv::Rect *trackBox_Green;
  137. src_Red.copyTo(tmp_Red);
  138. src_Green.copyTo(tmp_Green);
  139. // 提取轮廓
  140. findContours(tmp_Red, contours_Red, hierarchy_Red, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
  141. findContours(tmp_Green, contours_Green, hierarchy_Green, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
  142. // 绘制轮廓
  143. drawContours(frame, contours_Red, -1, cv::Scalar(0, 0, 255), cv::FILLED); // Red
  144. std::cout << "提取到的红色轮廓数量:" << contours_Red.size() << std::endl;
  145. drawContours(frame, contours_Green, -1, cv::Scalar(0, 255, 0), cv::FILLED); // Green
  146. std::cout << "提取到的绿色轮廓数量:" << contours_Green.size() << std::endl;
  147. // 显示轮廓
  148. // imshow("contours", frame);
  149. trackBox_Red = new cv::Rect[contours_Red.size()];
  150. trackBox_Green = new cv::Rect[contours_Green.size()];
  151. // 确定要跟踪的区域
  152. for (int i = 0; i < contours_Red.size(); i++)
  153. {
  154. // Yi opencv4 不支持 CvSeq
  155. trackBox_Red[i] = cv::boundingRect(contours_Red[i]);
  156. }
  157. for (int i = 0; i < contours_Green.size(); i++)
  158. {
  159. // Yi opencv4 不支持 CvSeq
  160. trackBox_Green[i] = cv::boundingRect(contours_Green[i]);
  161. }
  162. // imshow("contours", tmp);
  163. // Rect.tl() 返回矩形左上顶点的坐标
  164. for (int i = 0; i < contours_Red.size(); i++)
  165. {
  166. Store_x_color a;
  167. a.x = trackBox_Red[i].tl().x;
  168. a.y = trackBox_Red[i].tl().y;
  169. a.color = 0;
  170. x_color.push_back(a);
  171. }
  172. for (int i = 0; i < contours_Green.size(); i++)
  173. {
  174. Store_x_color a;
  175. a.x = trackBox_Green[i].tl().x;
  176. a.y = trackBox_Green[i].tl().y;
  177. a.color = 1;
  178. x_color.push_back(a);
  179. }
  180. // 清空指针
  181. delete[] trackBox_Red;
  182. delete[] trackBox_Green;
  183. // 对左上顶点横坐标进行排序
  184. sort(x_color.begin(), x_color.end(), CompareByX);
  185. // 显示结果
  186. for (int i = 0; i < x_color.size(); i++)
  187. {
  188. if (0 == x_color[i].color)
  189. cv::putText(frame, "Red", cv::Point(x_color[i].x, x_color[i].y - 25), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(0, 0, 255), 2, 8, 0);
  190. else if (1 == x_color[i].color)
  191. cv::putText(frame, "Green", cv::Point(x_color[i].x, x_color[i].y - 25), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(0, 255, 0), 2, 8, 0);
  192. else if (2 == x_color[i].color)
  193. cv::putText(frame, "Yellow", cv::Point(x_color[i].x, x_color[i].y - 25), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(0, 255, 255), 2, 8, 0);
  194. else
  195. cv::putText(frame, "Lights off", cv::Point(x_color[i].x, x_color[i].y - 25), cv::FONT_HERSHEY_SIMPLEX, 1, cv::Scalar(255, 255, 255), 2, 8, 0);
  196. }
  197. cv::namedWindow("tld_result", 0);
  198. cv::resizeWindow("tld_result", 1920, 1080);
  199. cv::imshow("tld_result", frame);
  200. cv::waitKey(1);
  201. return;
  202. }
  203. bool CompareByX(const Store_x_color &a, const Store_x_color &b)
  204. {
  205. return a.x < b.x;
  206. }

五、使用方法

编译所需的CMakeLists.txt、package.xml和运行所需roslaunch文件如下

  • CMakeLists.txt

  1. cmake_minimum_required(VERSION 2.8.3)
  2. project(tld_cv)
  3. set(CMAKE_INCLUDE_CURRENT_DIR ON)
  4. set(CMAKE_BUILD_TYPE "Release") # Debug Release RelWithDebInfo
  5. add_definitions(-O2 -pthread)
  6. add_compile_options(-std=c++11)
  7. find_package(OpenCV REQUIRED)
  8. find_package(catkin REQUIRED COMPONENTS
  9. roscpp
  10. std_msgs
  11. sensor_msgs
  12. cv_bridge
  13. image_transport
  14. )
  15. catkin_package(
  16. CATKIN_DEPENDS
  17. roscpp
  18. std_msgs
  19. sensor_msgs
  20. cv_bridge
  21. image_transport
  22. )
  23. include_directories(
  24. # include
  25. ${catkin_INCLUDE_DIRS}
  26. ${OpenCV_INCLUDE_DIRS}
  27. )
  28. add_executable(tld_cv src/main.cpp)
  29. target_link_libraries(tld_cv
  30. ${catkin_LIBRARIES}
  31. ${OpenCV_LIBRARIES}
  32. )
  •  package.xml

  1. <?xml version="1.0"?>
  2. <package format="2">
  3. <name>tld_cv</name>
  4. <version>0.0.0</version>
  5. <description>The tld_cv package</description>
  6. <maintainer email="royry@foxmail.com">Ru1yi</maintainer>
  7. <license>TODO</license>
  8. <buildtool_depend>catkin</buildtool_depend>
  9. <build_depend>cv_bridge</build_depend>
  10. <build_depend>image_transport</build_depend>
  11. <build_depend>roscpp</build_depend>
  12. <build_depend>sensor_msgs</build_depend>
  13. <build_depend>std_msgs</build_depend>
  14. <build_export_depend>cv_bridge</build_export_depend>
  15. <build_export_depend>image_transport</build_export_depend>
  16. <build_export_depend>roscpp</build_export_depend>
  17. <build_export_depend>sensor_msgs</build_export_depend>
  18. <build_export_depend>std_msgs</build_export_depend>
  19. <exec_depend>cv_bridge</exec_depend>
  20. <exec_depend>image_transport</exec_depend>
  21. <exec_depend>roscpp</exec_depend>
  22. <exec_depend>sensor_msgs</exec_depend>
  23. <exec_depend>std_msgs</exec_depend>
  24. <!-- The export tag contains other, unspecified, tags -->
  25. <export>
  26. <!-- Other tools can request additional information be placed here -->
  27. </export>
  28. </package>
  • detect.launch

  1. <launch>
  2. <arg name="sub_image_topic" value="/camera/image_color/compressed"/>
  3. <param name="sub_topic" value="$(arg sub_image_topic)"/>
  4. <node pkg="tld_cv" type="tld_cv" name="tld_cv" output="screen" />
  5. </launch>

六、后续改进思路 

改进可有如下几个方向:

  • ROI

根据具体自动驾驶场景,可以通过红绿灯位置、高度、相机安装方式、车辆定位和IMU信息实时计算出一个更为精确的ROI,检测再进行排序即可确定红绿灯的个数和顺序,方便编写后处理中的逻辑。

  • 筛选面积

根据检测后的结果筛选较大的几个轮廓,可以排除掉某些较小物体的误检干扰 

本人接触OpenCV时间尚短、经验尚浅,如果有任何疏漏、错误还请大家指出~

声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/从前慢现在也慢/article/detail/599231
推荐阅读
相关标签
  

闽ICP备14008679号