赞
踩
PS:要转载请注明出处,本人版权所有。
PS: 这个只是基于《我自己》的理解,
如果和你的原则及想法相冲突,请谅解,勿喷。
BlogID=103
由于我们小组的产品落地越来越多,以前(2018年)我搭建的老旧产品框架已经有点日落西山的感觉了。倒不是说产品业务不能支撑了,只是随着使用的时间增多,逐渐的感觉框架比较’LOW’,全靠使用者调教的怎么样,缺少了很多公共的组件。我倒是觉得这是必然的,因为当时我们小组的产品功能还比较的单一,随着产品功能多样化,产品迭代的频率越来越高,使得我们现有的框架在对付当前的产品的时候,有点落后了。最主要的就是我们的框架代码复用率有点低,导致快速开发的时候,那叫一个不爽。在我们都是打工仔的前提下,这一致命缺陷让我很难受,项目Code&Debug的时间成本越来越高。还有一个原因就是当时我设计这个框架的时候,自己的想法很单纯,也很简单。
下图是当时我设计的框架的一个简单示意图:
因此,我们小组内部决定要预研和引入一套开源的比较好使的框架,在后面如果上大项目的时候,可能就要上这个预研的新框架,经过了一圈调研,可能觉得Mediapipe和我们的业务很耦合,应该是可以使用的。于是乎,就有了我来踩一踩这个大坑。
好的多说无益,直接看运行Demo效果(包含FaceDetectDemo 和 HolisticDemo),截图来自于我录制的4个视频。
由于Mediapipe使用的是一个名为bazel的编译管理工具,可以说是大大增加了我们解决问题的难度。
注意,Mediapipe使用的是bazel来编译的,如果以前没有接触过,可能使用起来不是那么的友好。而且Mediapipe基本是和特定版本的bazel是绑定的,所以在安装bazel之前,建议大家去看看你要使用的mediapipe版本所依赖的bazel版本是多少。你可以去看 ‘gay-hub’ 上面Mediapipe 的关于bazel的说明。如下图类似的说明:
一般来说,Mediapipe依赖的bazel比较新,可能常见的发行版里面自带的bazel版本比较低,所以一般都需要从源码开始从零编译bazel,如果系统已有bazel等,可以用其他方式编译,详见后文链接的文档内容。下面的内容一般都是copy来自于 https://docs.bazel.build/versions/4.0.0/install-compile-source.html 。
安装依赖
编译bazel
编译生成的bazel二进制文件在bazel-src-dir/output/ 目录。
这时我们可以选择将bazel安装到/usr/bin or /usr/local/bin或者把bazel-src-dir/output/ 添加到PATH环境变量。
然后运行可以得到如下图的内容:
至此,bazel安装完毕,这里我就不详细介绍bazel了,网上有许多的介绍资料。这里可以简单的记住这两点就够了:bazel build target 和 bazel run target。bazel build target是编译目标。bazel run target 是编译并运行目标。
首先我们要下载好源码,并切换到对应的版本:
然后,根据网页 https://google.github.io/mediapipe/solutions/face_detection.html 的Example Apps-> Desktop 小节里面,我们可以看到如下图的内容:
图中给出了关于人脸检测的demo的target 和 对应的graph-cfg文件。类似的说明,在每个solution下面都有。我推荐大家优先从hello_world开始编译,因为这个target简单,出错也好排查。此外,一些可以复用的文件,只要你不修改,就只会编译一次。
此外,我们编译的平台是aarch64+ubuntu18.04,而官方的编译结构带的是x86-64 + ubuntu 18.04,所以在链接一些第三方库的时候,例如:opencv,我们需要改一下路径才行,如下图:
可能在编译的过程中还会遇到其他的错误,大家就按着修改就好了。后面我会列出一些可能的错误及参考解决方法。
进入mediapipe的源码根目录开始编译运行CPU版本。
> bazel build --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hello_world:hello_world --local_cpu_resources=1
> bazel run --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hello_world:hello_world
进入mediapipe的源码根目录开始编译运行GPU版本。(注意,这里的helloworld并没有gpu加速,这里这是演示怎么编译一些公共的包含gpu的代码)
> bazel build --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hello_world:hello_world --local_cpu_resources=1
> bazel run --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/hello_world:hello_world
–local_cpu_resources 是为了指定线程个数,一般来说,编译这个东西比较耗费内存,建议大家合力设定。如果不指定这个参数,bazel 按照 nproc 开启多线程编译。这样可能在某些情况下要爆内存。
在bazel build 之后,第一次编译可能要耗费大量的时间,我建议去干干别的,休息一会儿。
在bazel run 之后,会打印一串helloworld。
进入mediapipe的源码根目录开始编译运行CPU版本。
> bazel build --define MEDIAPIPE_DISABLE_GPU=1 --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 mediapipe/examples/desktop/face_detection:face_detection_cpu --local_ram_resources=1500 --local_cpu_resources=1
> ./bazel-bin/mediapipe/examples/desktop/face_detection/face_detection_cpu -calculator_graph_config_file=./mediapipe/graphs/face_detection/face_detection_desktop_live.pbtxt -input_video_path=./TestVideos/out.mp4
进入mediapipe的源码根目录开始编译运行GPU版本。
> bazel build --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 mediapipe/examples/desktop/face_detection:face_detection_gpu --local_ram_resources=1500 --local_cpu_resources=1
> ./bazel-bin/mediapipe/examples/desktop/face_detection/face_detection_gpu -calculator_graph_config_file=./mediapipe/graphs/face_detection/face_detection_mobile_gpu.pbtxt -input_video_path=./TestVideos/out.mp4
运行之后,就会弹一个框开始检测了。
进入mediapipe的源码根目录开始编译运行CPU版本。
bazel build --define MEDIAPIPE_DISABLE_GPU=1 --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 mediapipe/examples/desktop/holistic_tracking:holistic_tracking_cpu --local_ram_resources=1500 --local_cpu_resources=1
./bazel-bin/mediapipe/examples/desktop/holistic_tracking:holistic_tracking_cpu -calculator_graph_config_file=./mediapipe/graphs/holistic_tracking/holistic_tracking_cpu.pbtxt -input_video_path=./TestVideos/out.mp4
进入mediapipe的源码根目录开始编译运行GPU版本。
bazel build --copt -DMESA_EGL_NO_X11_HEADERS --copt -DEGL_NO_X11 mediapipe/examples/desktop/holistic_tracking:holistic_tracking_gpu --local_ram_resources=1500 --local_cpu_resources=1
./bazel-bin/mediapipe/examples/desktop/holistic_tracking:holistic_tracking_gpu -calculator_graph_config_file=./ mediapipe/graphs/holistic_tracking/holistic_tracking_gpu.pbtxt -input_video_path=./TestVideos/out.mp4
运行之后,就会弹一个框开始检测了。
如上文所说,如果不修改,将会出现undefined symbol xxx 或者 找不到opencv_xxx的库的问题。
根据官方说明,需要Opengl ES 3.1 或者 Opengl ES 3.1+的版本,我这里遇到的问题是,如果版本比这个低,运行GPU版本的demo 的时候,会报错。
注意:rk的libmali-rk-midgard-t86x-r14p0版本就支持 Opengl ES 3.1及以上,现在rk已经更新了libmali-rk-midgard-t86x-r18p0,可以暂时不更新也可以用。因为一旦更新了gpu驱动,内核也必须更新,配套的文件系统也得更新。因为我 xxxxxxxxxxxxxx 踩过.
关于Opengl ES 版本查看问题,可以使用glxinfo,如果没有这个命令推荐安装。如果使用的ssh来运行glxinfo,保证启用X11-forwarding. 查看到的版本如下图(glxinfo|grep “OpenGL ES”):
可能会出现以下问题,这个问题好像是多个xclient和xserver连接导致的问题,我也不确定。但是要不报这个错误,请直接将板卡接显示器使用。
I20210417 17:49:08 4016 demo_run_graph_main_gpu.cc:58] Initialize the calculator graph.
I20210417 17:49:08 4016 demo_run_graph_main_gpu.cc:62] Initialize the GPU.
XIO: fatal IO error 11 (Resource temporarily unavailable) on X server "localhost:12.0"
after 6 requests (6 known processed) with 0 events remaining.
请在gflags::ParseCommandLineFlags之后,设置如下三个变量的值。。。,这个值是调试出来的。
gflags::ParseCommandLineFlags(&argc, &argv, true);
FLAGS_minloglevel = 0;
FLAGS_stderrthreshold = 0;
FLAGS_alsologtostderr = 1;
关于这个问题,我这里只想问问Firefly的厂家,咋们修改开源库的时候,能不能认真点??? “尽量添加,不要删除”,难道这不是修改开源库的一大原则吗?
看下边两个文件内容:
EGL/eglplatform.h 来至于deb包 libmali-rk-dev 1.7-1 http://wiki.t-firefly.com/firefly-rk3399-repo bionic/main arm64 Packages
#ifndef __eglplatform_h_ #define __eglplatform_h_ /* ** Copyright (c) 2007-2016 The Khronos Group Inc. ** ** Permission is hereby granted, free of charge, to any person obtaining a ** copy of this software and/or associated documentation files (the ** "Materials"), to deal in the Materials without restriction, including ** without limitation the rights to use, copy, modify, merge, publish, ** distribute, sublicense, and/or sell copies of the Materials, and to ** permit persons to whom the Materials are furnished to do so, subject to ** the following conditions: ** ** The above copyright notice and this permission notice shall be included ** in all copies or substantial portions of the Materials. ** ** THE MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, ** EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF ** MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. ** IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY ** CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, ** TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE ** MATERIALS OR THE USE OR OTHER DEALINGS IN THE MATERIALS. */ /* Platform-specific types and definitions for egl.h * $Revision: 30994 $ on $Date: 2015-04-30 13:36:48 -0700 (Thu, 30 Apr 2015) $ * * Adopters may modify khrplatform.h and this file to suit their platform. * You are encouraged to submit all modifications to the Khronos group so that * they can be included in future versions of this file. Please submit changes * by sending them to the public Khronos Bugzilla (http://khronos.org/bugzilla) * by filing a bug against product "EGL" component "Registry". */ #include <KHR/khrplatform.h> /* Macros used in EGL function prototype declarations. * * EGL functions should be prototyped as: * * EGLAPI return-type EGLAPIENTRY eglFunction(arguments); * typedef return-type (EXPAPIENTRYP PFNEGLFUNCTIONPROC) (arguments); * * KHRONOS_APICALL and KHRONOS_APIENTRY are defined in KHR/khrplatform.h */ #ifndef EGLAPI #define EGLAPI KHRONOS_APICALL #endif #ifndef EGLAPIENTRY #define EGLAPIENTRY KHRONOS_APIENTRY #endif #define EGLAPIENTRYP EGLAPIENTRY* /* The types NativeDisplayType, NativeWindowType, and NativePixmapType * are aliases of window-system-dependent types, such as X Display * or * Windows Device Context. They must be defined in platform-specific * code below. The EGL-prefixed versions of Native*Type are the same * types, renamed in EGL 1.3 so all types in the API start with "EGL". * * Khronos STRONGLY RECOMMENDS that you use the default definitions * provided below, since these changes affect both binary and source * portability of applications using EGL running on different EGL * implementations. */ struct gbm_device; struct gbm_surface; #if defined(WL_EGL_PLATFORM) typedef struct wl_display *EGLNativeDisplayType; typedef struct wl_egl_pixmap *EGLNativePixmapType; typedef struct wl_egl_window *EGLNativeWindowType; #elif defined(__GBM__) typedef struct gbm_device * EGLNativeDisplayType; typedef struct gbm_surface * EGLNativeWindowType; typedef void * EGLNativePixmapType; #elif defined(__unix__) #include <X11/Xlib.h> #include <X11/Xutil.h> typedef Display *EGLNativeDisplayType; typedef Pixmap EGLNativePixmapType; typedef Window EGLNativeWindowType; #endif /* EGL 1.2 types, renamed for consistency in EGL 1.3 */ typedef EGLNativeDisplayType NativeDisplayType; typedef EGLNativePixmapType NativePixmapType; typedef EGLNativeWindowType NativeWindowType; /* Define EGLint. This must be a signed integral type large enough to contain * all legal attribute names and values passed into and out of EGL, whether * their type is boolean, bitmask, enumerant (symbolic constant), integer, * handle, or other. While in general a 32-bit integer will suffice, if * handles are 64 bit types, then EGLint should be defined as a signed 64-bit * integer type. */ typedef khronos_int32_t EGLint; /* C++ / C typecast macros for special EGL handle values */ #if defined(__cplusplus) #define EGL_CAST(type, value) (static_cast<type>(value)) #else #define EGL_CAST(type, value) ((type) (value)) #endif #endif /* __eglplatform_h */
EGL/eglplatform.h 来至于deb包 libegl1-mesa-dev ubuntu 官方,甚至 https://github.com/rockchip-linux/libmali 人家rockchip官方带的东西至少没有乱删东西吧。
#ifndef __eglplatform_h_ #define __eglplatform_h_ /* ** Copyright (c) 2007-2016 The Khronos Group Inc. ** ** Permission is hereby granted, free of charge, to any person obtaining a ** copy of this software and/or associated documentation files (the ** "Materials"), to deal in the Materials without restriction, including ** without limitation the rights to use, copy, modify, merge, publish, ** distribute, sublicense, and/or sell copies of the Materials, and to ** permit persons to whom the Materials are furnished to do so, subject to ** the following conditions: ** ** The above copyright notice and this permission notice shall be included ** in all copies or substantial portions of the Materials. ** ** THE MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, ** EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF ** MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. ** IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY ** CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, ** TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE ** MATERIALS OR THE USE OR OTHER DEALINGS IN THE MATERIALS. */ /* Platform-specific types and definitions for egl.h * $Revision: 30994 $ on $Date: 2015-04-30 13:36:48 -0700 (Thu, 30 Apr 2015) $ * * Adopters may modify khrplatform.h and this file to suit their platform. * You are encouraged to submit all modifications to the Khronos group so that * they can be included in future versions of this file. Please submit changes * by sending them to the public Khronos Bugzilla (http://khronos.org/bugzilla) * by filing a bug against product "EGL" component "Registry". */ #include <KHR/khrplatform.h> /* Macros used in EGL function prototype declarations. * * EGL functions should be prototyped as: * * EGLAPI return-type EGLAPIENTRY eglFunction(arguments); * typedef return-type (EXPAPIENTRYP PFNEGLFUNCTIONPROC) (arguments); * * KHRONOS_APICALL and KHRONOS_APIENTRY are defined in KHR/khrplatform.h */ #ifndef EGLAPI #define EGLAPI KHRONOS_APICALL #endif #ifndef EGLAPIENTRY #define EGLAPIENTRY KHRONOS_APIENTRY #endif #define EGLAPIENTRYP EGLAPIENTRY* #if defined(MESA_EGL_NO_X11_HEADERS) && !defined(EGL_NO_X11) #warning "`MESA_EGL_NO_X11_HEADERS` is deprecated, and doesn't work with the unmodified Khronos header" #warning "Please use `EGL_NO_X11` instead, as `MESA_EGL_NO_X11_HEADERS` will be removed soon" #define EGL_NO_X11 #endif /* The types NativeDisplayType, NativeWindowType, and NativePixmapType * are aliases of window-system-dependent types, such as X Display * or * Windows Device Context. They must be defined in platform-specific * code below. The EGL-prefixed versions of Native*Type are the same * types, renamed in EGL 1.3 so all types in the API start with "EGL". * * Khronos STRONGLY RECOMMENDS that you use the default definitions * provided below, since these changes affect both binary and source * portability of applications using EGL running on different EGL * implementations. */ #if defined(_WIN32) || defined(__VC32__) && !defined(__CYGWIN__) && !defined(__SCITECH_SNAP__) /* Win32 and WinCE */ #ifndef WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN 1 #endif #include <windows.h> typedef HDC EGLNativeDisplayType; typedef HBITMAP EGLNativePixmapType; typedef HWND EGLNativeWindowType; #elif defined(__EMSCRIPTEN__) typedef int EGLNativeDisplayType; typedef int EGLNativePixmapType; typedef int EGLNativeWindowType; #elif defined(__WINSCW__) || defined(__SYMBIAN32__) /* Symbian */ typedef int EGLNativeDisplayType; typedef void *EGLNativePixmapType; typedef void *EGLNativeWindowType; #elif defined(WL_EGL_PLATFORM) typedef struct wl_display *EGLNativeDisplayType; typedef struct wl_egl_pixmap *EGLNativePixmapType; typedef struct wl_egl_window *EGLNativeWindowType; #elif defined(__GBM__) typedef struct gbm_device *EGLNativeDisplayType; typedef struct gbm_bo *EGLNativePixmapType; typedef void *EGLNativeWindowType; #elif defined(__ANDROID__) || defined(ANDROID) struct ANativeWindow; struct egl_native_pixmap_t; typedef void* EGLNativeDisplayType; typedef struct egl_native_pixmap_t* EGLNativePixmapType; typedef struct ANativeWindow* EGLNativeWindowType; #elif defined(USE_OZONE) typedef intptr_t EGLNativeDisplayType; typedef intptr_t EGLNativePixmapType; typedef intptr_t EGLNativeWindowType; #elif defined(__unix__) && defined(EGL_NO_X11) typedef void *EGLNativeDisplayType; typedef khronos_uintptr_t EGLNativePixmapType; typedef khronos_uintptr_t EGLNativeWindowType; #elif defined(__unix__) || defined(USE_X11) /* X11 (tentative) */ #include <X11/Xlib.h> #include <X11/Xutil.h> typedef Display *EGLNativeDisplayType; typedef Pixmap EGLNativePixmapType; typedef Window EGLNativeWindowType; #elif defined(__APPLE__) typedef int EGLNativeDisplayType; typedef void *EGLNativePixmapType; typedef void *EGLNativeWindowType; #elif defined(__HAIKU__) #include <kernel/image.h> typedef void *EGLNativeDisplayType; typedef khronos_uintptr_t EGLNativePixmapType; typedef khronos_uintptr_t EGLNativeWindowType; #else #error "Platform not recognized" #endif /* EGL 1.2 types, renamed for consistency in EGL 1.3 */ typedef EGLNativeDisplayType NativeDisplayType; typedef EGLNativePixmapType NativePixmapType; typedef EGLNativeWindowType NativeWindowType; /* Define EGLint. This must be a signed integral type large enough to contain * all legal attribute names and values passed into and out of EGL, whether * their type is boolean, bitmask, enumerant (symbolic constant), integer, * handle, or other. While in general a 32-bit integer will suffice, if * handles are 64 bit types, then EGLint should be defined as a signed 64-bit * integer type. */ typedef khronos_int32_t EGLint; /* C++ / C typecast macros for special EGL handle values */ #if defined(__cplusplus) #define EGL_CAST(type, value) (static_cast<type>(value)) #else #define EGL_CAST(type, value) ((type) (value)) #endif #endif /* __eglplatform_h */
这文件被删改的谁都不认识了好吧!!!你说气不气,关键是Mediapipe编译GPU 版本的时候,还得用到这个EGL_NO_X11宏。我真的是无fuck说。
出现问题的根源是:
/* In file EGL/egl.h: #include <EGL/eglplatform.h> In file EGL/eglplatform.h: #include <X11/Xlib.h> In file X11/Xlib.h(line:83): #define Status int 当我们include 了 egl.h 这个文件后,会引入一个Status的宏。 在我们使用google的工程的时候,还有一个class 叫做 absl::Status。 如果项目中定义一个变量: absl::Status TestVal; 会被预编译为: absl::int TestVal; 然后就会编译报错。 */
好了,这就是我初探Mediapipe的故事。初次接触的话,我觉得最恼火的还是它的编译问题(对我来说:bazel是真的难用),其实编译问题解决了,还是很友好的,这些代码都封装的很好。
PS: 请尊重原创,不喜勿喷。
PS: 要转载请注明出处,本人版权所有。
PS: 有问题请留言,看到后我会第一时间回复。
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。