Ubuntu20.04LTS下安装Intel Realsense D435i驱动与ROS包_rgbd_launch下载

目标

在Ubuntu20.04LTS系统下安装D435i的驱动SDK2和ROS包，实现在ROS环境中使用D435i。

一、D435i简介

D435i是Inter公司RealSense系列摄像头的一种，采用英特尔最新的深度感应硬件和软件，封装成易于集成的产品。相比D435，D435i添加了IMU（惯性测量单元），不仅为 SLAM 和跟踪应用打开了大门，还使手持扫描系统的注册和校准更容易，在虚拟/增强现实和无人机等领域很重要。
D435i是一个RGB-D相机，也就是我们常说的深度相机，目前深度相机的实现原理主要分为三种：分别是结构光，Tof，双目成像。REALSENSE使用的是结构光的方案。正面的四个摄像头，从左向右以次是左红外相机，红外点阵投射仪，右红外相机，和RGB相机。

尽管具有红外投射器，但并不是采用红外反射测距。它的作用仅仅是投射不可见的固定的红外纹理样式，提高在纹理不明显的环境中（例如白墙）的深度计算精度，辅助双目视觉测距。左右两个相机将图像数据送入内置的深度处理器，在其中基于双目测距的原理计算每个像素的深度值。
最高 1280×720 双目深度分辨率;
最高 1920×1080 RGB 分辨率;
最高 90 FPS 深度视频流。
深度流与普通 RGB 视频流类似，只不过每个像素点的值不再是 RGB 或灰度值，而是物体相对于相机的距离。
只能对相同帧率的 RGB 与深度视频流做同步设置。
双目 baseline 为 50 mm;
深度探测范围 0.2 m ~ 10 m;
深度坐标系以左侧相机为中心（下图中 centerline of 1/4-20 是指三脚架螺丝空的中心）

二、环境配置

系统：Ubuntu20.04LTS
ROS：1.15.14
摄像机：D435i

三、RealSense的SDK2.0安装

方式一：(优先)
1.注册服务器的公钥

sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE || sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-key F6E65AC044F831AC80A06380C8B3A55A6F3EFCDE
1
2

2.将服务器添加到存储库列表中

sudo add-apt-repository "deb https://librealsense.intel.com/Debian/apt-repo $(lsb_release -cs) main" -u
1

3.安装SDK2
方式一（apt）：
基础安装

sudo apt-get install librealsense2-dkms
sudo apt-get install librealsense2-utils
1
2

选装

sudo apt-get install librealsense2-dev
sudo apt-get install librealsense2-dbg
1
2

Tip:安装dev包后，可使用g++ -std=c++11 filename.cpp -lrealsense2 或 别的IDE来编译具有librealsense的应用。
测试安装结果

realsense-viewer 
1

方式二（源代码）：
下载librealsense，然后进入该目录，运行下列指令安装和编译依赖项：

sudo apt-get install libudev-dev pkg-config libgtk-3-dev
sudo apt-get install libusb-1.0-0-dev pkg-config
sudo apt-get install libglfw3-dev
sudo apt-get install libssl-dev

sudo cp config/99-realsense-libusb.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules && udevadm trigger 
mkdir build
cd build
cmake ../ -DBUILD_EXAMPLES=true
make
sudo make install
1
2
3
4
5
6
7
8
9
10
11
12

测试安装结果

realsense-viewer 
1

四、ROS包安装

方式一（apt）：
安装ROS版本的realsense2_camera

sudo apt-get install ros-$ROS_DISTRO-realsense2-camera
sudo apt-get install ros-$ROS_DISTRO-realsense2-description
1
2

方式二（源代码）：
进入catkin_ws工作空间的src目录，下载realsense和ddynamic_reconfigure，然后用ROS的catkin_make指令编译。

cd ~/catkin_ws/src
git clone https://github.com/IntelRealSense/realsense-ros.git
git clone https://github.com/pal-robotics/ddynamic_reconfigure.git
cd ~/catkin_ws && catkin_make
1
2
3
4

安装rgbd-launch
rgbd_launch是一组打开RGBD设备，并load 所有nodelets转化 raw depth/RGB/IR 流到深度图(depth image), 视差图(disparity image)和点云(point clouds)的launch文件集。

sudo apt-get install ros-noetic-rgbd-launch
1

测试编译结果：

roslaunch realsense2_camera demo_pointcloud.launch 

1
2

五、摄像机CV的ROS包节点

在catkin_ws工作空间中创建一个usb_camera包，含有一个图像消息发布节点img_publisher与图像消息接收显示节点img_viewer。
package.xml：

<buildtool_depend>catkin</buildtool_depend>
  <build_depend>cv_bridge</build_depend>
  <build_depend>image_transport</build_depend>
  <build_depend>roscpp</build_depend>
  <build_depend>sensor_msgs</build_depend>
  <build_depend>std_msgs</build_depend>
  <build_depend>opencv2</build_depend>

  <build_export_depend>cv_bridge</build_export_depend>
  <build_export_depend>image_transport</build_export_depend>
  <build_export_depend>roscpp</build_export_depend>
  <build_export_depend>sensor_msgs</build_export_depend>
  <build_export_depend>std_msgs</build_export_depend>
  <build_export_depend>opencv2</build_export_depend>

  <exec_depend>cv_bridge</exec_depend>
  <exec_depend>image_transport</exec_depend>
  <exec_depend>roscpp</exec_depend>
  <exec_depend>sensor_msgs</exec_depend>
  <exec_depend>std_msgs</exec_depend>
 <exec_depend>opencv2</exec_depend>

  <!-- The export tag contains other, unspecified, tags -->
  <export>
    <!-- Other tools can request additional information be placed here -->

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

img_publisher.cpp节点：

#include <ros/ros.h>
#include <image_transport/image_transport.h>
#include <opencv2/highgui.hpp>
#include <opencv2/calib3d.hpp>
#include <cv_bridge/cv_bridge.h>
#include <iostream>
 
int main(int argc, char** argv)
{
  ros::init(argc, argv, "img_publisher");
  ros::NodeHandle nh;
  image_transport::ImageTransport it(nh);
  image_transport::Publisher pub = it.advertise("camera/image", 1);
 
  cv::VideoCapture cap;
  cv::Mat frame;
    
  int deviceID=0;
  
  if(argc>1)
	deviceID=argv[1][0]-'0';
  int apiID=cv::CAP_ANY;
  cap.open(deviceID+apiID);
  
  if(!cap.isOpened()){
	std::cerr<<"ERROR! Unable to open camera"<<std::endl;
	return -1;
  }
 
  ros::Rate loop_rate(30);
  while (nh.ok()) {
	cap.read(frame);
	if(!frame.empty()){
		sensor_msgs::ImagePtr msg = cv_bridge::CvImage(std_msgs::Header(), "bgr8", frame).toImageMsg();
		pub.publish(msg);
	}
    	ros::spinOnce();
    	loop_rate.sleep();
  }
  return 0;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

img_viewer：

//接收并查看图像消息的节点img_viewer.cpp
#include <ros/ros.h>
#include <image_transport/image_transport.h>
#include <opencv2/highgui/highgui.hpp>
#include <cv_bridge/cv_bridge.h>
 
 //依赖包有std_msgs（消息传递），roscpp（c++），cv_bridge（ros和opencv图像消息转换），sensor_msgs（传感器消息），image_transport（图像编码传输）
void imageCallback(const sensor_msgs::ImageConstPtr& msg)
{
  try
  {
    cv::imshow("view", cv_bridge::toCvShare(msg, "bgr8")->image);
  }
  catch (cv_bridge::Exception& e)
  {
    ROS_ERROR("Could not convert from '%s' to 'bgr8'.", msg->encoding.c_str());
  }
}
 
int main(int argc, char **argv)
{
  ros::init(argc, argv, "img_viewer");
  ros::NodeHandle nh;
  cv::namedWindow("view");
  cv::startWindowThread();
  image_transport::ImageTransport it(nh);
  image_transport::Subscriber sub = it.subscribe("camera/image", 1, imageCallback);
  ros::spin();
  cv::destroyWindow("view");
  return 0;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

CMakeLists.txt：

cmake_minimum_required(VERSION 3.0.2)
project(usb_camera)

## Compile as C++11, supported in ROS Kinetic and newer
# add_compile_options(-std=c++11)

## Find catkin macros and libraries
## if COMPONENTS list like find_package(catkin REQUIRED COMPONENTS xyz)
## is used, also find other catkin packages

find_package(catkin REQUIRED COMPONENTS
  cv_bridge
  image_transport
  roscpp
  sensor_msgs
  std_msgs
)

## System dependencies are found with CMake's conventions
# find_package(Boost REQUIRED COMPONENTS system)

find_package(OpenCV REQUIRED)
## Uncomment this if the package has a setup.py. This macro ensures
## modules and global scripts declared therein get installed
## See http://ros.org/doc/api/catkin/html/user_guide/setup_dot_py.html
# catkin_python_setup()

################################################
## Declare ROS messages, services and actions ##
################################################

## To declare and build messages, services or actions from within this
## package, follow these steps:
## * Let MSG_DEP_SET be the set of packages whose message types you use in
##   your messages/services/actions (e.g. std_msgs, actionlib_msgs, ...).
## * In the file package.xml:
##   * add a build_depend tag for "message_generation"
##   * add a build_depend and a exec_depend tag for each package in MSG_DEP_SET
##   * If MSG_DEP_SET isn't empty the following dependency has been pulled in
##     but can be declared for certainty nonetheless:
##     * add a exec_depend tag for "message_runtime"
## * In this file (CMakeLists.txt):
##   * add "message_generation" and every package in MSG_DEP_SET to
##     find_package(catkin REQUIRED COMPONENTS ...)
##   * add "message_runtime" and every package in MSG_DEP_SET to
##     catkin_package(CATKIN_DEPENDS ...)
##   * uncomment the add_*_files sections below as needed
##     and list every .msg/.srv/.action file to be processed
##   * uncomment the generate_messages entry below
##   * add every package in MSG_DEP_SET to generate_messages(DEPENDENCIES ...)

## Generate messages in the 'msg' folder
# add_message_files(
#   FILES
#   Message1.msg
#   Message2.msg
# )

## Generate services in the 'srv' folder
# add_service_files(
#   FILES
#   Service1.srv
#   Service2.srv
# )

## Generate actions in the 'action' folder
# add_action_files(
#   FILES
#   Action1.action
#   Action2.action
# )

## Generate added messages and services with any dependencies listed here
# generate_messages(
#   DEPENDENCIES
#   sensor_msgs#   std_msgs
# )

################################################
## Declare ROS dynamic reconfigure parameters ##
################################################

## To declare and build dynamic reconfigure parameters within this
## package, follow these steps:
## * In the file package.xml:
##   * add a build_depend and a exec_depend tag for "dynamic_reconfigure"
## * In this file (CMakeLists.txt):
##   * add "dynamic_reconfigure" to
##     find_package(catkin REQUIRED COMPONENTS ...)
##   * uncomment the "generate_dynamic_reconfigure_options" section below
##     and list every .cfg file to be processed

## Generate dynamic reconfigure parameters in the 'cfg' folder
# generate_dynamic_reconfigure_options(
#   cfg/DynReconf1.cfg
#   cfg/DynReconf2.cfg
# )

###################################
## catkin specific configuration ##
###################################
## The catkin_package macro generates cmake config files for your package
## Declare things to be passed to dependent projects
## INCLUDE_DIRS: uncomment this if your package contains header files
## LIBRARIES: libraries you create in this project that dependent projects also need
## CATKIN_DEPENDS: catkin_packages dependent projects also need
## DEPENDS: system dependencies of this project that dependent projects also need
catkin_package(
#  INCLUDE_DIRS include
#  LIBRARIES usb_camera
#  CATKIN_DEPENDS cv_bridge image_transport roscpp sensor_msgs std_msgs
#  DEPENDS system_lib
)

###########
## Build ##
###########

## Specify additional locations of header files
## Your package locations should be listed before other locations
include_directories(
# include
  ${catkin_INCLUDE_DIRS}
  ${OpenCV_INCLUDE_DIRS}
)

## Declare a C++ library
# add_library(${PROJECT_NAME}
#   src/${PROJECT_NAME}/usb_camera.cpp
# )

## Add cmake target dependencies of the library
## as an example, code may need to be generated before libraries
## either from message generation or dynamic reconfigure
# add_dependencies(${PROJECT_NAME} ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS})

## Declare a C++ executable
## With catkin_make all packages are built within a single CMake context
## The recommended prefix ensures that target names across packages don't collide
# add_executable(${PROJECT_NAME}_node src/usb_camera_node.cpp)

## Rename C++ executable without prefix
## The above recommended prefix causes long target names, the following renames the
## target back to the shorter version for ease of user use
## e.g. "rosrun someones_pkg node" instead of "rosrun someones_pkg someones_pkg_node"
# set_target_properties(${PROJECT_NAME}_node PROPERTIES OUTPUT_NAME node PREFIX "")

## Add cmake target dependencies of the executable
## same as for the library above
# add_dependencies(${PROJECT_NAME}_node ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS})

## Specify libraries to link a library or executable target against
# target_link_libraries(${PROJECT_NAME}_node
#   ${catkin_LIBRARIES}
# )
add_executable(img_publisher src/img_publisher.cpp)
target_link_libraries(img_publisher ${catkin_LIBRARIES} ${OpenCV_LIBS})

add_executable(img_viewer src/img_viewer.cpp)
target_link_libraries(img_viewer ${catkin_LIBRARIES} ${OpenCV_LIBS})

#############
## Install ##
#############

# all install targets should use catkin DESTINATION variables
# See http://ros.org/doc/api/catkin/html/adv_user_guide/variables.html

## Mark executable scripts (Python etc.) for installation
## in contrast to setup.py, you can choose the destination
# catkin_install_python(PROGRAMS
#   scripts/my_python_script
#   DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
# )

## Mark executables for installation
## See http://docs.ros.org/melodic/api/catkin/html/howto/format1/building_executables.html
# install(TARGETS ${PROJECT_NAME}_node
#   RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION}
# )

## Mark libraries for installation
## See http://docs.ros.org/melodic/api/catkin/html/howto/format1/building_libraries.html
# install(TARGETS ${PROJECT_NAME}
#   ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
#   LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION}
#   RUNTIME DESTINATION ${CATKIN_GLOBAL_BIN_DESTINATION}
# )

## Mark cpp header files for installation
# install(DIRECTORY include/${PROJECT_NAME}/
#   DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION}
#   FILES_MATCHING PATTERN "*.h"
#   PATTERN ".svn" EXCLUDE
# )

## Mark other files for installation (e.g. launch and bag files, etc.)
# install(FILES
#   # myfile1
#   # myfile2
#   DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}
# )

#############
## Testing ##
#############

## Add gtest based cpp test target and link libraries
# catkin_add_gtest(${PROJECT_NAME}-test test/test_usb_camera.cpp)
# if(TARGET ${PROJECT_NAME}-test)
#   target_link_libraries(${PROJECT_NAME}-test ${PROJECT_NAME})
# endif()

## Add folders to be run by python nosetests
# catkin_add_nosetests(test)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215

六、问题排查

1. 查找usb摄像头
在最初的节点运行时，提示找不到usb摄像头，改变摄像机的索引号也不行。查摄像头占用情况：

ls  /dev/video*
/dev/video0  /dev/video1  /dev/video2  /dev/video3  /dev/video4  /dev/video5
1
2

摄像机索引没问题。
根据输出信息[ WARN:0] OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Cannot identify device '/dev/video0'. [ WARN:0] OpenCV | GStreamer warning: unable to start pipeline [ WARN:0] OpenCV | GStreamer warning: GStreamer: pipeline have not been created
查找相关文献，试探性的调试如下：

find /lib/modules/ -name "*v4l2*.ko"

1
2

显示如下信息：

/lib/modules/5.13.0-30-generic/kernel/drivers/media/v4l2-core/v4l2-flash-led-class.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/v4l2-core/v4l2-dv-timings.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/v4l2-core/v4l2-fwnode.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/v4l2-core/v4l2-mem2mem.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/i2c/adv7511-v4l2.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/common/videobuf2/videobuf2-v4l2.ko
/lib/modules/5.13.0-30-generic/kernel/drivers/media/common/v4l2-tpg/v4l2-tpg.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/v4l2-core/v4l2-flash-led-class.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/v4l2-core/v4l2-dv-timings.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/v4l2-core/v4l2-fwnode.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/v4l2-core/v4l2-mem2mem.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/i2c/adv7511-v4l2.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/common/videobuf2/videobuf2-v4l2.ko
/lib/modules/5.13.0-27-generic/kernel/drivers/media/common/v4l2-tpg/v4l2-tpg.ko
1
2
3
4
5
6
7
8
9
10
11
12
13
14

发现相关的ko，其中，adv7511-v4l2.ko这个是与usb摄像机相关的文件。运行：

sudo modprobe adv7511-v4l2
1

然后就可以打开USB摄像机了。modprobe命令用于自动处理可载入模块，设置摄像头设备后相当于默认使用该设备。
另外，用v4l2-ctl --list-devices 命令可以查看摄像头对应的设备号，如果没有v4l2-ctl，可先安装：

sudo apt install v4l-utils
1

v4l2-ctl --list-devices
Video Capture 9 (usb-0000:00:14.0-3):
	/dev/video0
	/dev/video1
	/dev/video2
	/dev/video3
	/dev/video4
	/dev/video5
	/dev/media0
	/dev/media1
1
2
3
4
5
6
7
8
9
10

经过上述操作后并不一定完全解决问题。
2. 了解Gstreamer框架
gstreamer是一个开发流式多媒体应用的开源框架，是为更好的处理音视频而设计开发的，但又不限于处理音视频，只要是有明显的数据流特征的应用。
gstreamer的功能核心是pipeline框架以及用于扩展功能的插件。通过gstreamer多媒体数据协商机制和同步机制，pipeline能够很好的地处理流式数据。开发者可以灵活利用已有插件，而且还可以自定义特定功能插件。

[1] Ubantu20.04安装libRealsense SDK 2.0 和ROS Wrapper for Intel RealSense Devices
[2] Ubuntu20.04+RealSense D435i
[3] ubuntu opencv 打开摄像头失败
[4] Gstreamer基础知识介绍