赞
踩
刚开始做camera相关的开发时,对YUV_420_888这个格式很懵,不同平台的机型,从Image中转换出RGB的方法好像也不同,在终于初步了解YUV420格式后,写下本文,权当笔记总结。
YUV420转RGBA系列共三篇:
本文是其中的第一篇。
在了解YUV_420_888之前,我们先来了解一下YUV。我们知道,RGB是一种颜色编码方法,一个像素分别以R、G、B三个分量来表示。YUV也是一种颜色编码方法,一个像素分别以Y、U、V三个分量来表示。Y表示明亮度(Luminance、Luma),U和V则是色度、浓度(Chrominance、Chroma)。
YUV发明于彩色电视与黑白电视的过渡时期。黑白电视使用的黑白图像只有Y(Luma,Luminance)分量,也就是灰阶值。到了彩色电视规格的制定,是以YUV的格式来处理彩色电视图像,把UV视作表示彩度的C(Chrominance或Chroma),如果忽略C信号,那么剩下的Y(Luma)信号就跟之前的黑白电视频号相同,这样一来便解决了彩色电视机与黑白电视机的兼容问题。YUV在对照片或影片编码时,考虑到人类的感知能力,允许降低色度的采样。根据UV的采样不同,YUV分为多种格式:
降低UV的采样后,传输时只需占用极少的带宽。(本节内容来自维基百科:https://zh.wikipedia.org/wiki/YUV)
YUV根据Y、U、V存储方式的不同,可以分成两个格式:
平面格式(planar)又分为:
采样方式采用YUV420、存储方式采用平面格式(planar)称为YUV420P。YUV420P根据U和V顺序不同又分为:
采样方式采用YUV420、存储方式采用半平面格式(semi-planar)称为YUV420SP,YUV420SP根据U和V顺序不同又分为:
我们先来看Android API中对YUV_420_888的介绍(https://developer.android.google.cn/reference/android/graphics/ImageFormat?hl=en#YUV_420_888):
Multi-plane Android YUV 420 format
This format is a generic YCbCr format, capable of describing any 4:2:0 chroma-subsampled planar or semiplanar buffer (but not fully interleaved), with 8 bits per color sample.
Images in this format are always represented by three separate buffers of data, one for each color plane. Additional information always accompanies the buffers, describing the row stride and the pixel stride for each plane.
The order of planes in the array returned by Image#getPlanes() is guaranteed such that plane #0 is always Y, plane #1 is always U (Cb), and plane #2 is always V (Cr).
The Y-plane is guaranteed not to be interleaved with the U/V planes (in particular, pixel stride is always 1 in yPlane.getPixelStride()).
The U/V planes are guaranteed to have the same row stride and pixel stride (in particular, uPlane.getRowStride() == vPlane.getRowStride() and uPlane.getPixelStride() == vPlane.getPixelStride(); ).
Google翻译下:
多平面Android YUV 420格式
此格式是通用的YCbCr格式,能够描述任何4:2:0色度采样的平面或半平面缓冲区(但不完全交织),每个颜色样本有8位。
这种格式的图像始终由三个单独的数据缓冲区表示,每个颜色缓冲区一个。 缓冲区中始终会附带其他信息,描述每个平面的行步长和像素步长。
确保Image#getPlanes()返回的数组中平面的顺序,使得平面#0始终为Y,平面#1始终为U(Cb),平面#2始终为V(Cr)。
保证Y平面不与U / V平面交错(特别是yPlane.getPixelStride()中的像素步长始终为1)。
确保U / V平面具有相同的行步长和像素步长(尤其是uPlane.getRowStride()== vPlane.getRowStride()和uPlane.getPixelStride()== vPlane.getPixelStride();)。
根据API中的介绍,我们可以知道,YUV_420_888是可以兼容所有YUV420P和YUV420SP格式的。也就是说上面提到的I420、YV12、NV12、NV21都可以是YUV_420_888的具体实现。虽然同样是Image对象,不同平台的机型可能有不同的实现。
根据本人测试,创建ImageReader时参数format传入YUV_420_888,Image.getPlanes()后,分别获取Plane0、Plane1、Plane2的buffer:
//MTK
width:3264, height:2448,
yRowStride:3264, yPixelStride:1, yPixelSize:7990272,
uRowStride:1632, uPixelStride:1, uPixelSize:1997568,
vRowStride:1632, vPixelStride:1, vPixelSize:1997568
可以看到 Y:U:V = 4:1:1,说明这是YUV420采样格式。实测:
展锐平台就比较奇怪了,即使创建ImageReader时传了YUV_420_888,用的似乎还是YUV422,但这明显不符合Andoid API的设计意图。并且用的是YUV422就算了,又还缺了一个UV色度分量。
//SPRD
width:3264, height:2448,
yRowStride:3264, yPixelStride:1, yPixelSize:7990272,
uRowStride:3264, uPixelStride:2, uPixelSize:3995135, //正常是3995136
vRowStride:3264, vPixelStride:2, vPixelSize:3995135 //正常是3995136
可以看到 Y:U:V = 4:2:2,说明这是YUV422采样格式。所以展锐平台的转换有两种方法:
其它平台暂未测试,有兴趣的童鞋可以自行测试下。采用YUV422虽然也能正常使用,但是个人认为这个明显是不合理的,YUV422不符合YUV_420_888的API描述,并且采用YUV420处理速度、存储速度都会比YUV422更快,传输带宽占用更少。
根据 API 我们可以知道,创建ImageReader时参数format除了YUV_420_888,还可以传,YUV_422_888、YUV_444_888这样就可以得到不同YUV采样模式的图像,但根据展锐平台的异常现象,我们也可以知道这都依赖于平台实现。
I420、YV12、NV12、N21转换时都有一些共性:
因此,我们只要找到每个像素Y、U、V分量的对应关系就可以进行转换。
i420.png
yv12.png
nv12.png
nv21.png
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/android/src/org/tensorflow/demo/中有相关的YUV转RGB和RGB转YUV的代码。本文也是参考其中的转换代码,分别封装了一下I420转RGBA、YV12转RGBA、NV12转RGBA、NV21转RGBA。
- const int K_MAX_CHANNEL_VALUE = 262143;
- const int YUV420P_I420 = 1;
- const int YUV420P_YV12 = 2;
- const int YUV420SP_NV12 = 3;
- const int YUV420SP_NV21 = 4;
- /*
- * This function come from:
- * https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/android/jni/yuv2rgb.cc
- */
- static inline int YUV2RGBA(int nY, int nU, int nV) {
- nY -= 16;
- nU -= 128;
- nV -= 128;
- if (nY < 0) nY = 0;
-
- // This is the floating point equivalent. We do the conversion in integer
- // because some Android devices do not have floating point in hardware.
- // nR = (int)(1.164 * nY + 2.018 * nU);
- // nG = (int)(1.164 * nY - 0.813 * nV - 0.391 * nU);
- // nB = (int)(1.164 * nY + 1.596 * nV);
-
- int nR = 1192 * nY + 1634 * nV;
- int nG = 1192 * nY - 833 * nV - 400 * nU;
- int nB = 1192 * nY + 2066 * nU;
-
- nR = nR > K_MAX_CHANNEL_VALUE ? K_MAX_CHANNEL_VALUE : (nR < 0 ? 0 : nR);
- nG = nG > K_MAX_CHANNEL_VALUE ? K_MAX_CHANNEL_VALUE : (nG < 0 ? 0 : nG);
- nB = nB > K_MAX_CHANNEL_VALUE ? K_MAX_CHANNEL_VALUE : (nB < 0 ? 0 : nB);
-
- nR = (nR >> 10) & 0xff;
- nG = (nG >> 10) & 0xff;
- nB = (nB >> 10) & 0xff;
- return 0xff000000 | (nR << 16) | (nG << 8) | nB;
- }
-
- /*
- * int order: ARGB
- * byte order: RGBA
- */
- static inline void rgbaIntToBytes(int rgba, unsigned char *b) {
- b[0] = (unsigned char) ((rgba >> 16) & 0xff); //R
- b[1] = (unsigned char) ((rgba >> 8) & 0xff); //G
- b[2] = (unsigned char) (rgba & 0xff); //B
- b[3] = (unsigned char) ((rgba >> 24) & 0xff); //A
- }
- void YUV420PToRGBAByte(unsigned char *src, unsigned char *dst, int width, int height,
- int yRowStride, int uvRowStride, int uvPixelStride, int format) {
- if (format == YUV420P_I420 || format == YUV420P_YV12) {
- unsigned char *pRGBA = dst;
- unsigned char *pY = src;
- unsigned char *pU;
- unsigned char *pV;
- if (format == YUV420P_I420) {
- pU = src + width * height;
- pV = src + width * height / 4 * 5;
- } else {
- pU = src + width * height / 4 * 5;
- pV = src + width * height;
- }
- for (int y = 0; y < height; y++) {
- //const int yRowStart = yRowStride * y;
- const int uvRowStart = uvRowStride * (y >> 1);
- for (int x = 0; x < width; x++) {
- const int uvRowOffset = (x >> 1) * uvPixelStride;
- rgbaIntToBytes(YUV2RGBA(*pY++,//pY[yRowStart + x],
- pU[uvRowStart + uvRowOffset],
- pV[uvRowStart + uvRowOffset]),
- pRGBA);
- pRGBA += 4;
- }
- }
- }
- }
-
- void YUV420PToRGBAInt(unsigned char *src, int *dst, int width, int height,
- int yRowStride, int uvRowStride, int uvPixelStride, int format) {
- if (format == YUV420P_I420 || format == YUV420P_YV12) {
- unsigned char *pY = src;
- unsigned char *pU;
- unsigned char *pV;
- int rgbaIndex = 0;
- if (format == YUV420P_I420) {
- pU = src + width * height;
- pV = src + width * height / 4 * 5;
- } else {
- pU = src + width * height / 4 * 5;
- pV = src + width * height;
- }
- for (int y = 0; y < height; y++) {
- //const int yRowStart = yRowStride * y;
- const int uvRowStart = uvRowStride * (y >> 1);
- for (int x = 0; x < width; x++) {
- const int uvRowOffset = (x >> 1) * uvPixelStride;
- dst[rgbaIndex++] = YUV2RGBA(*pY++,//pY[yRowStart + x],
- pU[uvRowStart + uvRowOffset],
- pV[uvRowStart + uvRowOffset]);
- }
- }
- }
- }
- void YUV420SPToRGBAByte(unsigned char *src, unsigned char *dst, int width, int height,
- int format) {
- if (format == YUV420SP_NV12 || format == YUV420SP_NV21) {
- unsigned char *pRGBA = dst;
- unsigned char *pY = src;
- unsigned char *pUV = src + width * height;
- for (int y = 0; y < height; y++) {
- for (int x = 0; x < width; x++) {
- int uvOffset = (y >> 1) * width + 2 * (x >> 1);
- int uOffset;
- int vOffset;
- if (format == YUV420SP_NV12) {
- uOffset = uvOffset;
- vOffset = uvOffset + 1;
- } else {
- uOffset = uvOffset + 1;
- vOffset = uvOffset;
- }
- rgbaIntToBytes(YUV2RGBA(*pY++, pUV[uOffset], pUV[vOffset]), pRGBA);
- pRGBA += 4;
- }
- }
- }
- }
-
- void YUV420SPToRGBAInt(unsigned char *src, int *dst, int width, int height,
- int format) {
- if (format == YUV420SP_NV12 || format == YUV420SP_NV21) {
- unsigned char *pY = src;
- unsigned char *pUV = src + width * height;
- int rgbaIndex = 0;
- for (int y = 0; y < height; y++) {
- for (int x = 0; x < width; x++) {
- int uvOffset = (y >> 1) * width + 2 * (x >> 1);
- int uOffset;
- int vOffset;
- if (format == YUV420SP_NV12) {
- uOffset = uvOffset;
- vOffset = uvOffset + 1;
- } else {
- uOffset = uvOffset + 1;
- vOffset = uvOffset;
- }
- dst[rgbaIndex++] = YUV2RGBA(*pY++, pUV[uOffset], pUV[vOffset]);
- }
- }
- }
- }
- extern "C"
- JNIEXPORT void JNICALL
- Java_com_qxt_yuv420_NativeUtils_I420ToRGBAByte(JNIEnv *env, jclass clazz,
- jbyteArray src, jbyteArray dst,
- jint width, jint height, jint yRowStride,
- jint uvRowStride, jint uvPixelStride) {
- jbyte *_src = env->GetByteArrayElements(src, nullptr);
- jbyte *_dst = env->GetByteArrayElements(dst, nullptr);
- YUV420PToRGBAByte(reinterpret_cast<unsigned char *>(_src),
- reinterpret_cast<unsigned char *>(_dst),
- width, height, yRowStride, uvRowStride, uvPixelStride, YUV420P_I420);
- env->ReleaseByteArrayElements(src, _src, JNI_ABORT);
- env->ReleaseByteArrayElements(dst, _dst, 0);
- }
-
- extern "C"
- JNIEXPORT void JNICALL
- Java_com_qxt_yuv420_NativeUtils_I420ToRGBAInt(JNIEnv *env, jclass clazz,
- jbyteArray src, jintArray dst,
- jint width, jint height, jint yRowStride,
- jint uvRowStride, jint uvPixelStride) {
- jbyte *_src = env->GetByteArrayElements(src, NULL);
- jint *_dst = env->GetIntArrayElements(dst, NULL);
- YUV420PToRGBAInt(reinterpret_cast<unsigned char *>(_src), _dst, width, height, yRowStride,
- uvRowStride, uvPixelStride, YUV420P_I420);
- env->ReleaseByteArrayElements(src, _src, JNI_ABORT);
- env->ReleaseIntArrayElements(dst, _dst, 0);
- }
-
- extern "C"
- JNIEXPORT void JNICALL
- Java_com_qxt_yuv420_NativeUtils_YV12ToRGBAByte(JNIEnv *env, jclass clazz,
- jbyteArray src, jbyteArray dst,
- jint width, jint height, jint yRowStride,
- jint uvRowStride, jint uvPixelStride) {
- jbyte *_src = env->GetByteArrayElements(src, nullptr);
- jbyte *_dst = env->GetByteArrayElements(dst, nullptr);
- YUV420PToRGBAByte(reinterpret_cast<unsigned char *>(_src),
- reinterpret_cast<unsigned char *>(_dst),
- width, height, yRowStride, uvRowStride, uvPixelStride, YUV420P_YV12);
- env->ReleaseByteArrayElements(src, _src, JNI_ABORT);
- env->ReleaseByteArrayElements(dst, _dst, 0);
- }
-
- extern "C"
- JNIEXPORT void JNICALL
- Java_com_qxt_yuv420_NativeUtils_YV12ToRGBAInt(JNIEnv *env, jclass clazz,
- jbyteArray src, jintArray dst,
- jint width, jint height, jint yRowStride,
- jint uvRowStride, jint uvPixelStride) {
- jbyte *_src = env->GetByteArrayElements(src, NULL);
- jint *_dst = env->GetIntArrayElements(dst, NULL);
- YUV420PToRGBAInt(reinterpret_cast<unsigned char *>(_src),
- _dst, width, height, yRowStride, uvRowStride, uvPixelStride, YUV420P_YV12);
- env->ReleaseByteArrayElements(src, _src, JNI_ABORT);
- env->ReleaseIntArrayElements(dst, _dst, 0);
- }
-
- extern "C"
- JNIEXPORT void JNICALL
- Java_com_qxt_yuv420_NativeUtils_NV12ToRGBAByte(JNIEnv *env, jclass clazz,
- jbyteArray src, jbyteArray dst,
- jint width, jint height) {
- jbyte *_src = env->GetByteArrayElements(src, NULL);
- jbyte *_dst = env->GetByteArrayElements(dst, NULL);
- YUV420SPToRGBAByte(reinterpret_cast<unsigned char *>(_src),
- reinterpret_cast<unsigned char *>(_dst), width, height, YUV420SP_NV12);
- env->ReleaseByteArrayElements(src, _src, JNI_ABORT);
- env->ReleaseByteArrayElements(dst, _dst, 0);
- }
-
- extern "C"
- JNIEXPORT void JNICALL
- Java_com_qxt_yuv420_NativeUtils_NV12ToRGBAInt(JNIEnv *env, jclass clazz,
- jbyteArray src, jintArray dst,
- jint width, jint height) {
- jbyte *_src = env->GetByteArrayElements(src, NULL);
- jint *_dst = env->GetIntArrayElements(dst, NULL);
- YUV420SPToRGBAInt(reinterpret_cast<unsigned char *>(_src), _dst, width, height, YUV420SP_NV12);
- env->ReleaseByteArrayElements(src, _src, JNI_ABORT);
- env->ReleaseIntArrayElements(dst, _dst, 0);
- }
-
- extern "C"
- JNIEXPORT void JNICALL
- Java_com_qxt_yuv420_NativeUtils_NV21ToRGBAByte(JNIEnv *env, jclass clazz,
- jbyteArray src, jbyteArray dst,
- jint width, jint height) {
- jbyte *_src = env->GetByteArrayElements(src, NULL);
- jbyte *_dst = env->GetByteArrayElements(dst, NULL);
- YUV420SPToRGBAByte(reinterpret_cast<unsigned char *>(_src),
- reinterpret_cast<unsigned char *>(_dst), width, height, YUV420SP_NV21);
- env->ReleaseByteArrayElements(src, _src, JNI_ABORT);
- env->ReleaseByteArrayElements(dst, _dst, 0);
- }
-
- extern "C"
- JNIEXPORT void JNICALL
- Java_com_qxt_yuv420_NativeUtils_NV21ToRGBAInt(JNIEnv *env, jclass clazz,
- jbyteArray src, jintArray dst,
- jint width, jint height) {
- jbyte *_src = env->GetByteArrayElements(src, NULL);
- jint *_dst = env->GetIntArrayElements(dst, NULL);
- YUV420SPToRGBAInt(reinterpret_cast<unsigned char *>(_src), _dst, width, height, YUV420SP_NV21);
- env->ReleaseByteArrayElements(src, _src, JNI_ABORT);
- env->ReleaseIntArrayElements(dst, _dst, 0);
- }
- package com.qxt.yuv420;
-
-
- /**
- * @author Tyler Qiu
- * @date: 2020/05/09
- */
- public class NativeUtils {
- /*
- * YYYYYYYY UU VV =>I420 =>YUV420P
- * YYYYYYYY VV UU =>YV12 =>YUV420P
- * YYYYYYYY UV UV =>NV12 =>YUV420SP
- * YYYYYYYY VU VU =>NV21 =>YUV420SP
- */
- static {
- System.loadLibrary("NativeUtils");
- }
-
- /**
- * convert I420 to ARGB_8888
- *
- * @param src src I420 byte array
- * @param dst dst RGBA byte array, the length of the dst array must be >= width*height*4
- * @param width image width
- * @param height image height
- * @param yRowStride The row stride of plane y.
- * @param uvRowStride The row stride of plane u or v.
- * @param uvPixelStride The pixel stride of plane u or v.
- */
- public static native void I420ToRGBAByte(byte[] src, byte[] dst, int width, int height,
- int yRowStride, int uvRowStride, int uvPixelStride);
-
- /**
- * convert I420 to ARGB_8888
- *
- * @param src src I420 byte array
- * @param dst dst RGBA int array, the length of the dst array must be >= width*height
- * @param width image width
- * @param height image height
- * @param yRowStride The row stride of plane y.
- * @param uvRowStride The row stride of plane u or v.
- * @param uvPixelStride The pixel stride of plane u or v.
- */
- public static native void I420ToRGBAInt(byte[] src, int[] dst, int width, int height,
- int yRowStride, int uvRowStride, int uvPixelStride);
-
- /**
- * convert YV12 to ARGB_8888
- *
- * @param src src YV12 byte array
- * @param dst dst RGBA byte array, the length of the dst array must be >= width*height*4
- * @param width image width
- * @param height image height
- * @param yRowStride The row stride of plane y.
- * @param uvRowStride The row stride of plane u or v.
- * @param uvPixelStride The pixel stride of plane u or v.
- */
- public static native void YV12ToRGBAByte(byte[] src, byte[] dst, int width, int height,
- int yRowStride, int uvRowStride, int uvPixelStride);
-
- /**
- * convert YV12 to ARGB_8888
- *
- * @param src src YV12 byte array
- * @param dst dst RGBA int array, the length of the dst array must be >= width*height
- * @param width image width
- * @param height image height
- * @param yRowStride The row stride of plane y.
- * @param uvRowStride The row stride of plane u or v.
- * @param uvPixelStride The pixel stride of plane u or v.
- */
- public static native void YV12ToRGBAInt(byte[] src, int[] dst, int width, int height,
- int yRowStride, int uvRowStride, int uvPixelStride);
-
- /**
- * convert NV12 to ARGB_8888
- *
- * @param src src NV12 byte array
- * @param dst dst RGBA byte array, the length of the dst array must be >= width*height*4
- * @param width image width
- * @param height image height
- */
- public static native void NV12ToRGBAByte(byte[] src, byte[] dst, int width, int height);
-
- /**
- * convert NV12 to ARGB_8888
- *
- * @param src src NV12 byte array
- * @param dst dst RGBA int array, the length of the dst array must be >= width*height
- * @param width image width
- * @param height image height
- */
- public static native void NV12ToRGBAInt(byte[] src, int[] dst, int width, int height);
-
- /**
- * convert NV21 to ARGB_8888
- *
- * @param src src NV21 byte array
- * @param dst dst RGBA byte array, the length of the dst array must be >= width*height*4
- * @param width image width
- * @param height image height
- */
- public static native void NV21ToRGBAByte(byte[] src, byte[] dst, int width, int height);
-
- /**
- * convert NV21 to ARGB_8888
- *
- * @param src src NV21 byte array
- * @param dst dst RGBA int array, the length of the dst array must be >= width*height
- * @param width image width
- * @param height image height
- */
- public static native void NV21ToRGBAInt(byte[] src, int[] dst, int width, int height);
- }
本文中的代码和i420/yv12/nv12/n21文件已经上传到github:https://github.com/qiuxintai/YUV420Converter,如果本文代码对你有帮助,烦请在github上给我一个小小的star。除转换外,YUV420Converter中还有旋转RGB和YUV420的代码,如果还想了解RGB和YUV420的旋转,请戳:
RGB和YUV420旋转90/180/270度
除了用代码转换,我们还可以用7yuv工具来查看和转换YUV格式的图像。
7yuv下载地址:https://www.onlinedown.net/soft/1225925.htm
打开7yuv,File -> open 或者直接将yuv图像拖入7yuv界面进行查看:
7yuv使用1.png
选择 Edit -> Convert Format 转换:
7yuv使用2.png
链接:https://www.jianshu.com/p/944ede616261
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。