devbox
你好赵伟
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

HarmonyOS通过OpenGL渲染显示yuv数据_鸿蒙 camera 通过opegl 渲染

作者：你好赵伟 | 2024-03-26 05:14:53

赞

踩

鸿蒙 camera 通过opegl 渲染

一、UI页面render_page.ets

  1. Column() {
  2.           XComponent({
  3.             id: "XComponent88",
  4.             type: XCompentConstants.XCOMPONENT_TYPE,
  5.             libraryname: XCompentConstants.XCOMPONENT_LIBRARY_NAME
  6.           }).width(640).height(360)
  7.             .onLoad((xComponentContext?: object | Record<string, () => void>) => {
  8.               if (xComponentContext) {
  9.                 SRLog.i(TAG, "xComponentContext======")
  10.                 this.xComponentContext = xComponentContext as Record<string, (userid: number, streamid: number) => void>;
  11.                 this.xComponentContext.setBindUser(123, 1);
  12.               }
  13.             })
  14. }

 二、通过NAPI创建RenderManager管理Xcompent的windows窗口

1、NAPI初始化：init_napi.cpp

  1. #include "util/Log.h"
  2. #include "napi/native_api.h"
  3. #include "rtc/SRRtcVideoEngineNapi.h"
  4. #include "SRWindowManager.h"
  5. #include "net/SRHttpManager.h"
  6.  
  7.  
  8. EXTERN_C_START
  9. static napi_value Init(napi_env env, napi_value exports) {
  10.     napi_property_descriptor desc[] = {
  11.         {"testRender", nullptr, SRRtcVideoEngineNapi::testRender, nullptr, nullptr, nullptr, napi_default, nullptr},
  12.         {"getNativeLifeCycle", nullptr, SRWindowManager::getNativeLifeCycle, nullptr, nullptr, nullptr, napi_default,
  13.          nullptr}
  14.  
  15.  
  16.     };
  17.  
  18.     if (napi_ok != napi_define_properties(env, exports, sizeof(desc) / sizeof(desc[0]), desc)) {
  19.         LogE("init_napi===napi_define_properties failed");
  20.         return nullptr;
  21.     }
  22.     SRWindowManager::GetInstance()->Export(env, exports);
  23.     return exports;
  24. }
  25. EXTERN_C_END
  26.  
  27. static napi_module demoModule = {
  28.     .nm_version = 1,
  29.     .nm_flags = 0,
  30.     .nm_filename = nullptr,
  31.     .nm_register_func = Init,
  32.     .nm_modname = "rtcvideo",
  33.     .nm_priv = ((void *)0),
  34.     .reserved = {0},
  35. };
  36.  
  37. extern "C" __attribute__((constructor)) void RegisterRtcvideoModule(void) { napi_module_register(&demoModule); }

2、绑定XCompendNative的实现：SRWindowManager.cpp

  1. std::unordered_map<std::string, OH_NativeXComponent *> SRWindowManager::m_nativeXComponentMap;
  2. std::unordered_map<std::string, SRGLRender *> SRWindowManager::m_SRGLRenderMap;
  3. void SRWindowManager::Export(napi_env env, napi_value exports) {
  4.     LogE("PluginManager::Export====");
  5.     if ((nullptr == env) || (nullptr == exports)) {
  6.         LogE("PluginManager::Export: env or exports is null");
  7.         return;
  8.     }
  9.  
  10.     napi_value exportInstance = nullptr;
  11.     if (napi_ok != napi_get_named_property(env, exports, OH_NATIVE_XCOMPONENT_OBJ, &exportInstance)) {
  12.         LogE("PluginManager::Export: napi_get_named_property fail");
  13.         return;
  14.     }
  15.  
  16.     OH_NativeXComponent *nativeXComponent = nullptr;
  17.     if (napi_ok != napi_unwrap(env, exportInstance, reinterpret_cast<void **>(&nativeXComponent))) {
  18.         LogE("PluginManager::Export: napi_unwrap fail");
  19.         return;
  20.     }
  21.  
  22.     char idStr[OH_XCOMPONENT_ID_LEN_MAX + 1] = {'\0'};
  23.     uint64_t idSize = OH_XCOMPONENT_ID_LEN_MAX + 1;
  24.     if (OH_NATIVEXCOMPONENT_RESULT_SUCCESS != OH_NativeXComponent_GetXComponentId(nativeXComponent, idStr, &idSize)) {
  25.         LogE("Export: OH_NativeXComponent_GetXComponentId fail");
  26.         return;
  27.     }
  28.  
  29.     std::string id(idStr);
  30.     auto context = SRWindowManager::GetInstance();
  31.     if ((nullptr != context) && (nullptr != nativeXComponent)) {
  32.         context->SetNativeXComponent(id, nativeXComponent);
  33.         auto render = context->GetRender(id);
  34.         OH_NativeXComponent_RegisterCallback(nativeXComponent, &SRGLRender::m_callback);
  35.         if (nullptr != render) {
  36.             render->Export(env, exports);
  37.         }
  38.     }
  39. }
  40.  
  41. void SRWindowManager::SetNativeXComponent(std::string &id, OH_NativeXComponent *nativeXComponent) {
  42.     if (nullptr == nativeXComponent) {
  43.         return;
  44.     }
  45.  
  46.     if (m_nativeXComponentMap.find(id) == m_nativeXComponentMap.end()) {
  47.         m_nativeXComponentMap[id] = nativeXComponent;
  48.         return;
  49.     }
  50.  
  51.     if (m_nativeXComponentMap[id] != nativeXComponent) {
  52.         OH_NativeXComponent *tmp = m_nativeXComponentMap[id];
  53.         delete tmp;
  54.         tmp = nullptr;
  55.         m_nativeXComponentMap[id] = nativeXComponent;
  56.     }
  57. }
  58.  
  59. SRGLRender *SRWindowManager::GetRender(std::string &id) {
  60.     if (m_SRGLRenderMap.find(id) == m_SRGLRenderMap.end()) {
  61.         SRGLRender *instance = SRGLRender::GetInstance(id);
  62.         m_SRGLRenderMap[id] = instance;
  63.         return instance;
  64.     }
  65.  
  66.     return m_SRGLRenderMap[id];
  67. }

三、OpenGL渲染模块

1、定义Shader

  1. const char VERTEX_SHADER11[] = "attribute vec4 vPosition;\n"
  2.                                "attribute vec2 a_texCoord;\n"
  3.                                "varying vec2 tc;\n"
  4.                                " uniform mat4 uMVPMatrix;\n"
  5.                                "void main() {\n"
  6.                                "gl_Position =uMVPMatrix * vPosition  ;\n"
  7.                                "tc = a_texCoord;\n"
  8.                                "}\n";
  9.  
  10. const char FRAGMENT_SHADER11[] = "precision mediump float;\n"
  11.                                  "uniform sampler2D tex_y;\n"
  12.                                  "uniform sampler2D tex_u;\n"
  13.                                  "uniform sampler2D tex_v;\n"
  14.                                  "varying vec2 tc;\n"
  15.                                  "void main() {\n"
  16.                                  "vec4 c = vec4((texture2D(tex_y, tc).r - 16./255.) * 1.164);\n"
  17.                                  "vec4 U = vec4(texture2D(tex_u, tc).r - 128./255.);\n"
  18.                                  "vec4 V = vec4(texture2D(tex_v, tc).r - 128./255.);\n"
  19.                                  "c += V * vec4(1.596, -0.813, 0, 0);\n"
  20.                                  "c += U * vec4(0, -0.392, 2.017, 0);\n"
  21.                                  "c.a = 1.0;\n"
  22.                                  "gl_FragColor = c;\n"
  23.                                  "}\n";

2、初始化EglContextInit

  1. bool EGLCore::EglContextInit(void *window, int width, int height) {
  2.     LogE("EGLCore::EglContextInit execute===width:%d,height:%d", width, height);
  3.     if (nullptr == window) {
  4.         LogE("EGLCore::EglContextInit window=nullptr");
  5.     }
  6.     if ((nullptr == window) || (0 >= width) || (0 >= height)) {
  7.         LogE("EGLCore::EglContextInit: param error");
  8.         return false;
  9.     }
  10.     w_width = width;
  11.     w_height = height;
  12.     LogE("EGLCore::EglContextInit: param m_width:%d;%d", w_width, w_height);
  13.     m_eglWindow = reinterpret_cast<EGLNativeWindowType>(window);
  14.     // Init display.
  15.     m_eglDisplay = eglGetDisplay(EGL_DEFAULT_DISPLAY);
  16.     if (EGL_NO_DISPLAY == m_eglDisplay) {
  17.         LogE("EGLCore:: eglGetDisplay: unable to get EGL display");
  18.         return false;
  19.     }
  20.     EGLint majorVersion;
  21.     EGLint minorVersion;
  22.     if (!eglInitialize(m_eglDisplay, &majorVersion, &minorVersion)) {
  23.         LogE("EGLCore:: eglInitialize: unable to get initialize EGL display");
  24.         return false;
  25.     }
  26.  
  27.     // Select configuration.
  28.     const EGLint maxConfigSize = 1;
  29.     EGLint numConfigs;
  30.     if (!eglChooseConfig(m_eglDisplay, ATTRIB_LIST, &m_eglConfig, maxConfigSize, &numConfigs)) {
  31.         LogE("EGLCore::eglChooseConfig: unable to choose configs");
  32.         return false;
  33.     }
  34.  
  35.     bool isCreateEnvironment = CreateEnvironment();
  36.     if (isCreateEnvironment) {
  37.         glClearColor(0.0, 0.0, 0.0, 0.0);
  38.         GLuint textures[1];
  39.         glGenTextures(1, &textures[0]);
  40.     }
  41.     return isCreateEnvironment;
  42. }
  43.  
  44. bool EGLCore::CreateEnvironment() {
  45.     // Create surface.
  46.     m_eglSurface = eglCreateWindowSurface(m_eglDisplay, m_eglConfig, m_eglWindow, NULL);
  47.  
  48.     if (nullptr == m_eglSurface) {
  49.         LogE("EGLCore:: eglCreateWindowSurface: unable to create surface");
  50.         return false;
  51.     }
  52.  
  53.     // Create context.
  54.     m_eglContext = eglCreateContext(m_eglDisplay, m_eglConfig, EGL_NO_CONTEXT, CONTEXT_ATTRIBS);
  55.     if (!eglMakeCurrent(m_eglDisplay, m_eglSurface, m_eglSurface, m_eglContext)) {
  56.         LogE("EGLCore::eglMakeCurrent failed");
  57.         return false;
  58.     }
  59.  
  60.     // Create program.
  61.     m_program = CreateProgram(VERTEX_SHADER11, FRAGMENT_SHADER11);
  62.     if (PROGRAM_ERROR == m_program) {
  63.         LogE("EGLCore::CreateProgram: unable to create program");
  64.         return false;
  65.     }
  66.     // 获取着色器的aPostion和a_textCoord
  67.     _positionHandle = glGetAttribLocation(m_program, "vPosition");
  68.     mMatrixHandle = glGetUniformLocation(m_program, "uMVPMatrix");
  69.     _coordHandle = glGetAttribLocation(m_program, "a_texCoord");
  70.     _yhandle = glGetUniformLocation(m_program, "tex_y");
  71.     _uhandle = glGetUniformLocation(m_program, "tex_u");
  72.     _vhandle = glGetUniformLocation(m_program, "tex_v");
  73.     LogE("EGLCore::CreateProgram: "
  74.          "_positionHandle:%d;mMatrixHandle:%d;_coordHandle:%d;_yhandle:%d;_uhandle:%d;_vhandle:%d",
  75.          _positionHandle, mMatrixHandle, _coordHandle, _yhandle, _uhandle, _vhandle);
  76.     _textureI = GL_TEXTURE0;
  77.     _textureII = GL_TEXTURE1;
  78.     _textureIII = GL_TEXTURE2;
  79.     LogE("EGLCore::CreateProgram: _textureI:%d;_textureII:%d;_textureIII:%d", _textureI, _textureII, _textureIII);
  80.     return true;
  81. }
  82.  
  83. GLuint EGLCore::CreateProgram(const char *vertexShader, const char *fragShader) {
  84.     if ((nullptr == vertexShader) || (nullptr == fragShader)) {
  85.         LogE("EGLCore:: createProgram: vertexShader or fragShader is null");
  86.         return PROGRAM_ERROR;
  87.     }
  88.     GLuint vertex_shader;
  89.     vertex_shader = LoadShader(GL_VERTEX_SHADER, vertexShader);
  90.     if (PROGRAM_ERROR == vertex_shader) {
  91.         LogE("EGLCore:: createProgram vertex error");
  92.         return PROGRAM_ERROR;
  93.     }
  94.  
  95.     GLuint fragment_shader;
  96.     fragment_shader = LoadShader(GL_FRAGMENT_SHADER, fragShader);
  97.     if (PROGRAM_ERROR == fragment_shader) {
  98.         LogE("EGLCore:: createProgram fragment error");
  99.         return PROGRAM_ERROR;
  100.     }
  101.     GLint vCompiled[4] = {0};
  102. //    glGetShaderiv(vertex_shader, GL_COMPILE_STATUS, vCompiled);
  103. //    if (vCompiled[0] == 0) {
  104. //        glDeleteShader(vertex_shader);
  105. //    }
  106.  
  107.     glGetShaderiv(vertex_shader, GL_COMPILE_STATUS, vCompiled);
  108.     if (vCompiled[0] != GL_TRUE) {
  109.         LogE("EGLCore:: createProgram vertex_shader error");
  110.         //        DeleteProgram();
  111.         return PROGRAM_ERROR;
  112.     }
  113.     glGetShaderiv(fragment_shader, GL_COMPILE_STATUS, vCompiled);
  114.     if (vCompiled[0] != GL_TRUE) {
  115.         LogE("EGLCore:: createProgram fragment_shader error");
  116.         //        DeleteProgram();
  117.         return PROGRAM_ERROR;
  118.     }
  119.  
  120.  
  121.     GLuint program;
  122.     program = glCreateProgram();
  123.     if (PROGRAM_ERROR == program) {
  124.         LogE("EGLCore:: createProgram program error");
  125.         glDeleteShader(vertex_shader);
  126.         glDeleteShader(fragment_shader);
  127.         return PROGRAM_ERROR;
  128.     } else {
  129.         LogE("EGLCore:: createProgram 加载着色器");
  130.         glAttachShader(program, vertex_shader);
  131.         glAttachShader(program, fragment_shader);
  132.         glLinkProgram(program);
  133.         // 存放链接成功的program的数组
  134.         GLint linkedStatus[4] = {0};
  135.         glGetProgramiv(program, GL_LINK_STATUS, linkedStatus);
  136.         if (linkedStatus[0] != GL_TRUE) {
  137.             LogE("EGLCore:: createProgram linkedStatus==error=linkedStatus[0]:%d", linkedStatus[0]);
  138.             glDeleteShader(program);
  139.             return PROGRAM_ERROR;
  140.         }
  141.         glUseProgram(program);
  142.     }
  143.     glDeleteShader(vertex_shader);
  144.     glDeleteShader(fragment_shader);
  145.     LogE("EGLCore:: createProgram linked success");
  146.     return program;
  147. }
  148.  
  149. GLuint EGLCore::LoadShader(GLenum type, const char *shaderSrc) {
  150.     if ((0 >= type) || (nullptr == shaderSrc)) {
  151.         LogE("EGLCore:: glCreateShader type or shaderSrc error");
  152.         return PROGRAM_ERROR;
  153.     }
  154.  
  155.     GLuint shader;
  156.     shader = glCreateShader(type);
  157.     if (0 == shader) {
  158.         LogE("EGLCore:: glCreateShader unable to load shader");
  159.         return PROGRAM_ERROR;
  160.     }
  161.  
  162.     // The gl function has no return value.
  163.     glShaderSource(shader, 1, &shaderSrc, nullptr);
  164.     glCompileShader(shader);
  165.     return shader;
  166. }

3、渲染实现

  1. void EGLCore::onRenderData(void *yData, void *uData, void *vData, int width, int height) {
  2.     if ((nullptr == m_eglDisplay) || (nullptr == m_eglSurface) || (nullptr == m_eglContext) ||
  3.         (!eglMakeCurrent(m_eglDisplay, m_eglSurface, m_eglSurface, m_eglContext))) {
  4.         LogE("EGLCore:: PrepareDraw: param error");
  5.         return;
  6.     }
  7.     // The gl function has no return value.
  8.     LogE("EGLCore:: PrepareDraw: param m_height:%d,m_width:%d", w_height, w_width);
  9.     glViewport(0, 0, w_width, w_height);
  10.     glClearColor(0, 0, 0, 1.0);
  11.     glClear(GL_COLOR_BUFFER_BIT);
  12.     glUseProgram(m_program);
  13.     // 开始渲染
  14.     LogE("EGLCore::draw: "
  15.          "_positionHandle:%d;mMatrixHandle:%d;_coordHandle:%d;_yhandle:%d;_uhandle:%d;_vhandle:%d",
  16.          _positionHandle, mMatrixHandle, _coordHandle, _yhandle, _uhandle, _vhandle);
  17.     // y
  18.     if (m_glTexture[0] < 0) {
  19.         glGenTextures(1, &m_glTexture[0]);
  20.         LogE("EGLCore::CreateProgram:  _ytid ", m_glTexture[0]);
  21.     }
  22.     glBindTexture(GL_TEXTURE_2D, m_glTexture[0]);
  23.     glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, width, height, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, yData);
  24.     glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
  25.     glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
  26.     glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
  27.     glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
  28.     // u
  29.     if (m_glTexture[1] < 0) {
  30.         glGenTextures(1, &m_glTexture[1]);
  31.         LogE("EGLCore::draw:  _utid ", m_glTexture[1]);
  32.     }
  33.     int w1 = (width + 1) / 2;
  34.     int h1 = (height + 1) / 2;
  35.     glBindTexture(GL_TEXTURE_2D, m_glTexture[1]);
  36.     glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, w1, h1, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, uData);
  37.     glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
  38.     glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
  39.     glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
  40.     glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
  41.  
  42.  
  43.     // v
  44.     if (m_glTexture[2] < 0) {
  45.         glGenTextures(1, &m_glTexture[2]);
  46.         LogE("EGLCore::CreateProgram:  _vtid ", m_glTexture[2]);
  47.     }
  48.  
  49.     glBindTexture(GL_TEXTURE_2D, m_glTexture[2]);
  50.     glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, w1, h1, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, vData);
  51.     glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
  52.     glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
  53.     glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
  54.     glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
  55.     // draw
  56.     //  使用shader程序
  57.     glUseProgram(m_program);
  58.     // 将最终变换矩阵传入shader程序
  59.     glUniformMatrix4fv(mMatrixHandle, 1, false, mMMatrix);
  60.     // 顶点位置数据传入着色器
  61.     glVertexAttribPointer(_positionHandle, 2, GL_FLOAT, false, 8, vertices);
  62.     glEnableVertexAttribArray(_positionHandle);
  63.     glVertexAttribPointer(_coordHandle, 2, GL_FLOAT, false, 8, texCoords);
  64.     // 允许使用顶点坐标数组
  65.     glEnableVertexAttribArray(_coordHandle);
  66.  
  67.     // bind textures
  68.     LogE("EGLCore::draw: _textureI:%d;_textureII:%d;_textureIII:%d", _textureI, _textureII, _textureIII);
  69.     // 绑定纹理
  70.     glActiveTexture(_textureI);
  71.     glBindTexture(GL_TEXTURE_2D, m_glTexture[0]);
  72.     glUniform1i(_yhandle, _tIindex);
  73.  
  74.     glActiveTexture(_textureII);
  75.     glBindTexture(GL_TEXTURE_2D, m_glTexture[1]);
  76.     glUniform1i(_uhandle, _tIIindex);
  77.  
  78.     glActiveTexture(_textureIII);
  79.     glBindTexture(GL_TEXTURE_2D, m_glTexture[2]);
  80.     glUniform1i(_vhandle, _tIIIindex);
  81.     // 图形绘制
  82.     glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
  83.     //    glFinish();
  84.     glDisableVertexAttribArray(_positionHandle);
  85.     glDisableVertexAttribArray(_coordHandle);
  86.     glFlush();
  87.     glFinish();
  88.     eglSwapBuffers(m_eglDisplay, m_eglSurface);
  89. }

4、测试代码实现

  1.  std::string id(idStr);
  2.     SRGLRender *render = SRGLRender::GetInstance(id);
  3.     if (nullptr != render) {
  4.         int width = 960;
  5.         int height = 540;
  6.         int uWidth = width / 2;
  7.         int uHeight = height / 2;
  8.         unsigned char *yData = new unsigned char[width * height];
  9.         unsigned char *uData = new unsigned char[(width * height) / 2];
  10.         unsigned char *vData = new unsigned char[(width * height) / 2];
  11.         for (int i = 0; i < width * height; ++i) {
  12.             yData[i] = color;
  13.         }
  14.         // 填充Y数据为红色
  15.         for (int i = 0; i < (uWidth * uHeight); ++i) {
  16.             uData[i] = color;
  17.             vData[i] = color;
  18.         }
  19.         render->m_eglCore->onRenderData(yData, uData, vData, width, height);
  20.     }

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/你好赵伟/article/detail/314868
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号