赞
踩
https://gitee.com/open-ascend/atlas_mindxsdk_samples/raw/master/contrib/cv/object_detection/video_yolov5
https://gitee.com/ai_samples/pytorch_models/tree/master/cv/object_detection/yolov5
存放到 vidio_yolov5/data/models/yolov5 目录下
bash atc_310.sh
bash atc_310P3.sh
export MX_SDK_HOME=/usr/local/sdk_home/mxVision
bash run_cpp.sh
bash run_python.sh
wget http://www.live555.com/liveMedia/public/live555-latest.tar.gz
tar xzf live555-latest.tar.gz
cd live
./genMakefiles linux-64bit #注意后面这个参数是根据当前文件夹下config.<后缀>获取得到的
make
cd mediaServer
./live555MediaServer
下面上传文件到 live555MediaServer 同级目录
[root@localhost mediaServer]#
[root@localhost mediaServer]# ls
COPYING DynamicRTSPServer.cpp DynamicRTSPServer.o live555MediaServer.cpp Makefile Makefile.tail video.264
COPYING.LESSER DynamicRTSPServer.hh live555MediaServer live555MediaServer.o Makefile.head version.hh
[root@localhost mediaServer]#
我上传了一个 video.264 那么,使用http协议访问的地址就行 http://你的ip:8000/文件名
例:http://127.0.0.1:8000/video.264;
使用rtsp协议访问的地址 http://你的ip/文件名
例:rtsp://127.0.0.1/video.264
然后请准备一个流媒体播放器,我下载的是vlc播放器
输入测试地址:
输入测试地址后点击播放,播放的时候能看到视频,说明rtsp server功能正常。
先安装昇腾驱动,昇腾驱动请参考各个产品安装手册,安装完成后npu-smi info 显示安装成功
[root@localhost ~]#
[root@localhost ~]# npu-smi info
+-------------------------------------------------------------------------------------------------+
| npu-smi 22.0.2 Version: 22.0.2 |
+------------------+--------------+---------------------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) |
| Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) |
+==================+==============+===============================================================+
| 1 310 | OK | 12.8 45 0 / 0 |
| 0 0 | 0000:05:00.0 | 0 2621 / 8192 |
+==================+==============+===============================================================+
https://www.hiascend.com/document/detail/zh/mind-sdk/30rc3/quickstart/visionquickstart/visionquickstart_0000.html
./Ascend-mindxsdk-mxvision_3.0.RC2_linux-aarch64.run --install --install-path=/usr/local/sdk_home
–install-path为指定安装的路径
Installing collected packages:mindx
Successfully installed mindx-3.0.RC2
[root@localhost sdk_home]#
[root@localhost sdk_home]# pwd
/usr/local/sdk_home
[root@localhost sdk_home]# ls
mxVision mxVision-3.0.RC2
[root@localhost sdk_home]#
[root@localhost sdk_home]#
bash /usr/local/sdk_home/mxVision/operators/opencvosd/generate_osd_om.sh
执行成功后,显示如下效果
[root@localhost ~]# bash /usr/local/sdk_home/mxVision/operators/opencvosd/generate_osd_om.sh
ASCEND_HOME is set to /usr/local/Ascend by user
Set ASCEND_VERSION to the default value:ascend-toolkit/latest
ATC start working now,please wait for a moment.
ATC run success, welcome to the next use.
The model has been successfully converted to om,please get it under /usr/local/sdk_home/mxVision/operators/opencvosd.
[root@localhost ~]#
.bashrc文件添加以下环境变量
# 安装mxVision时配置
. /usr/local/sdk_home/mxVision/set_env.sh
用户也可以通过修改~/.bashrc文件方式设置永久环境变量,操作如下:
样例模型快速下载地址:
https://gitee.com/ai_samples/pytorch_models/tree/master/cv/object_detection/yolov5
python3.7 pytorch 1.5 建议搭配yolov5 2.0
下载ultralytics-2.0(软件包名为yolov5-2.0.tar.gz)。
wget https://github.com/ultralytics/yolov5/archive/v2.0.tar.gz
运行如下命令,解压软件包并修改转换脚本,目前ATC(om文件转换工具)支持的onnx算子版本为opset11。
tar -xzf yolov5-2.0.tar.gz
vi yolov5-2.0/models/export.py
:set number
i
# 修改第48行opset_version为11
:wq!
在yolov5-2.0目录运行如下命令:
python models/export.py --weights ./yolov5s.pt --img 640 --batch 1
运行结果:生成yolov5s.onnx文件。
首先对导出的onnx图使用onnx-simplifer工具进行简化。在yolov5-2.0目录运行如下命令:
python -m onnxsim --skip-optimization yolov5s.onnx yolov5s_sim.onnx
运行结果:生成yolov5s_sim.onnx文件。
然后利用附件脚本video_yolov5/data/models/yolov5/modify_yolov5s_2.0_slice.py修改模型Slice算子。将附件脚本上传yolov5-2.0目录,运行如下命令:
python modify_yolov5s_2.0_slice.py yolov5s_sim.onnx
运行结果:生成yolov5s_sim_t.onnx文件。
python3.8 pytorch 1.7 建议搭配yolov5 5.0
下载ultralytics-5.0(软件包名为yolov5-5.0.tar.gz)。
wget https://github.com/ultralytics/yolov5/archive/v5.0.tar.gz
运行如下命令,解压软件包并修改转换脚本,目前ATC(om文件转换工具)支持的onnx算子版本为opset11。
tar -xzf yolov5-5.0.tar.gz
vi yolov5-5.0/models/export.py
:set number
i
# 修改第77行opset_version为11
:wq!
在yolov5-5.0目录运行如下命令:
python models/export.py --weights ./yolov5s.pt --img 640 --batch 1
运行结果:生成yolov5s.onnx文件。
首先对导出的onnx图使用onnx-simplifer工具进行简化。在yolov5-5.0目录运行如下命令:
python -m onnxsim --skip-optimization yolov5s.onnx yolov5s_sim.onnx
运行结果:生成yolov5s_sim.onnx文件。
然后利用附件脚本video_yolov5/data/models/yolov5/modify_yolov5s_5.0_slice.py修改模型Slice算子。将附件脚本上传yolov5-5.0目录,运行如下命令:
python modify_yolov5s_5.0_slice.py yolov5s_sim.onnx
运行结果:生成yolov5s_sim_t.onnx文件。
[root@localhost yolov5]#
[root@localhost yolov5]# ls
aipp_yolov5.cfg atc_310.sh atc_310P3.sh coco2014.names modify_yolov5s_2.0_slice.py modify_yolov5s_5.0_slice.py yolov5.cfg yolov5.pt yolov5s_sim_t.onnx
[root@localhost yolov5]#
Ascend310芯片模型转换命令如下:
atc \
--model=./yolov5s_sim_t.onnx \
--framework=5 \
--output=./yolov5s \
--input_format=NCHW \
--input_shape="images:1,3,640,640" \
--enable_small_channel=1 \
--insert_op_conf=./aipp_yolov5.cfg \
--soc_version=Ascend310 \
--log=info
Ascend310P3芯片模型转换命令如下:
atc \
--model=./yolov5s_sim_t.onnx \
--framework=5 \
--output=./yolov5s \
--input_format=NCHW \
--input_shape="images:1,3,640,640" \
--enable_small_channel=1 \
--insert_op_conf=./aipp_yolov5.cfg \
--soc_version=Ascend310P3 \
--log=info
参数说明:
–model:待转换的ONNX模型。
–framework:5代表ONNX模型。
–output:输出的om模型。
–input_format:输入数据的格式。
–input_shape:输入数据的shape。
–insert_op_conf=./aipp_yolov5.cfg:AIPP插入节点,通过config文件配置算子信息,功能包括图片色域转换、裁剪、归一化,主要用于处理原图输入数据,常与DVPP配合使用,详见下文数据预处理。
详细ATC命令转换学习请参考:
https://support.huawei.com/enterprise/zh/doc/EDOC1100234054?idPath=23710424%7C251366513%7C22892968%7C251168373
[root@localhost yolov5]#
[root@localhost yolov5]# ls
aipp_yolov5.cfg atc_310.sh atc_310P3.sh coco2014.names modify_yolov5s_2.0_slice.py modify_yolov5s_5.0_slice.py yolov5.cfg yolov5.om yolov5.pt yolov5s_sim_t.onnx
[root@localhost yolov5]#
export MX_SDK_HOME=/usr/local/sdk_home/mxVision
"rtspUrl": "rtsp://127.0.0.1/video.264"
bash run_cpp.sh
bash run_python.sh
打开视频,视频流目标检测符合预期。
{ "classification": { "stream_config": { ##设置业务流在哪个芯片上处理 "deviceId": "0" }, "mxpi_rtspsrc0": { ##拉取视频流 "factory": "mxpi_rtspsrc", "props": { "rtspUrl": "rtsp://127.0.0.1/video.264", "channelId": "0" }, "next": "mxpi_videodecoder0" }, "mxpi_videodecoder0": { ##视频解码(纯硬件) "factory": "mxpi_videodecoder", "props": { "inputVideoFormat": "H264", "outputImageFormat": "YUV420SP_NV12", "vdecChannelId": "0" }, "next": "tee0" }, "tee0": { ##分流插件 "factory": "tee", "next": [ "queue1", "queue2" ] }, "queue1": { ##缓存插件 "props": { "max-size-buffers": "100" }, "factory": "queue", "next": "mxpi_imageresize0" }, "queue2": { ##缓存插件 "props": { "max-size-buffers": "100" }, "factory": "queue", "next": "mxpi_opencvosd_plugin:0" }, "mxpi_imageresize0": { ##图像缩放(纯硬件) "props": { "dataSource": "mxpi_videodecoder0", "resizeHeight": "640", "resizeWidth": "640", "resizeType": "Resizer_KeepAspectRatio_Fit" }, "factory": "mxpi_imageresize", "next": "queue3" }, "queue3": { ##缓存插件 "props": { "max-size-buffers": "100" }, "factory": "queue", "next": "mxpi_modelinfer0" }, "mxpi_modelinfer0": { ##模型推理 "props": { "dataSource": "mxpi_imageresize0", "modelPath": "data/models/yolov5/yolov5s.om", ##模型路径 "postProcessConfigPath": "data/models/yolov5/yolov5.cfg", "labelPath": "data/models/yolov5/coco2014.names", "postProcessLibPath": "libMpYOLOv5PostProcessor.so" }, "factory": "mxpi_modelinfer", "next": "queue4" }, "queue4": { ##缓存插件 "props": { "max-size-buffers": "100" }, "factory": "queue", "next": "mxpi_object2osdinstances0" }, "mxpi_object2osdinstances0": { ##配置OSD绘框参数 "props": { "dataSource": "mxpi_modelinfer0", "colorMap":"100,100,100|200,200,200|0,128,255|255,128,0", "fontFace":"16", "fontScale":"0.5", "fontThickness":"2", "fontLineType":"16", "rectThickness":"2", "rectLineType":"16" }, "factory":"mxpi_object2osdinstances", "next": "queue5" }, "queue5": { ##缓存插件 "props": { "max-size-buffers": "100" }, "factory": "queue", "next": "mxpi_opencvosd_plugin:1" }, "mxpi_opencvosd_plugin":{ ##绘制边框和目标检测类型 "factory":"mxpi_opencvosd", "next":"queue6" }, "queue6": { ##缓存插件 "props": { "max-size-buffers": "100" }, "factory": "queue", "next": "mxpi_videoencoder0" }, "mxpi_videoencoder0": { ##视频编码(纯硬件) "props": { "imageHeight": "1080", "imageWidth": "1920", "inputFormat": "YUV420SP_NV12", "outputFormat": "H264", "fps": "1", "iFrameInterval": "50" }, "factory": "mxpi_videoencoder", "next": "queue7" }, "queue7": { ##缓存插件 "props": { "max-size-buffers": "100" }, "factory": "queue", "next": "appsink0" }, "appsink0": { ##视频流输出 "factory": "appsink" } } }
int main(int argc, char* argv[]) { // 读取pipeline配置文件 std::string pipelineConfigPath = "data/pipeline/Sample.pipeline"; std::string pipelineConfig = ReadPipelineConfig(pipelineConfigPath); if (pipelineConfig == "") { LogError << "Read pipeline failed."; return APP_ERR_COMM_INIT_FAIL; } // 初始化 stream manager 资源 MxStream::MxStreamManager mxStreamManager; APP_ERROR ret = mxStreamManager.InitManager(); if (ret != APP_ERR_OK) { LogError << "Failed to init Stream manager, ret = " << ret << "."; return ret; } // 根据指定的pipeline配置创建Stream ret = mxStreamManager.CreateMultipleStreams(pipelineConfig); if (ret != APP_ERR_OK) { LogError << "Failed to create Stream, ret = " << ret << "."; return ret; } // 创建空的h264文件 FILE *fp = fopen("./out.h264", "wb"); if (fp == nullptr) { LogError << "Failed to open file."; return APP_ERR_COMM_OPEN_FAIL; } bool m_bFoundFirstIDR = false; bool bIsIDR = false; uint32_t frameCount = 0; uint32_t MaxframeCount = 5000; std::string streamName = "classification"; int inPluginId = 0; while (1) { // 获取视频编码后的视频帧 MxStream::MxstDataOutput* output = mxStreamManager.GetResult(streamName, inPluginId, 200000); if (output == nullptr) { LogError << "Failed to get pipeline output."; return ret; } // H264视频格式,第一帧写入必须是IDR帧 bIsIDR = (output->dataSize > 1); if(!m_bFoundFirstIDR) { if(!bIsIDR) { continue; } else { m_bFoundFirstIDR = true; } } // 把视频编码后的数据帧写入h264文件 if (fwrite(output->dataPtr, output->dataSize, 1, fp) != 1) { LogInfo << "write frame to file fail"; } LogInfo << "Dealing frame id:" << frameCount; frameCount++; if (frameCount > MaxframeCount) { LogInfo << "write frame to file done"; break; } delete output; } // 关闭h264文件 fclose(fp); // 销毁Streams mxStreamManager.DestroyAllStreams(); return 0; }
if __name__ == '__main__': # 初始化 Stream manager 资源 streamManagerApi = StreamManagerApi() ret = streamManagerApi.InitManager() if ret != 0: print("Failed to init Stream manager, ret=%s" % str(ret)) exit() # 根据指定的pipeline配置创建Stream with open("data/pipeline/Sample.pipeline", 'rb') as f: pipelineStr = f.read() ret = streamManagerApi.CreateMultipleStreams(pipelineStr) if ret != 0: print("Failed to create Stream, ret=%s" % str(ret)) exit() # 创建空的h264文件 fo = open("./out.h264", mode='wb') m_bFoundFirstIDR = False bIsIDR = False frameCount = 0 MaxframeCount = 1000 streamName = b'classification' uniqueId = 0 while 1 : # 获取视频编码后的视频帧 Result = streamManagerApi.GetResult(streamName, uniqueId, 200000) if Result.errorCode != 0: print("GetResult error. errorCode=%d, errorMsg=%s" % (Result.errorCode, Result.data.decode())) exit() # H264视频格式,第一帧写入必须是IDR帧 bIsIDR = (len(Result.data) > 1) if m_bFoundFirstIDR == False : if bIsIDR == False : continue else : m_bFoundFirstIDR = True # 把视频编码后的数据帧写入h264文件 if fo.write(Result.data) == 0 : print("write frame to file fail") break print("Dealing frame id: %d" % frameCount) frameCount = frameCount + 1 if frameCount > MaxframeCount : print("write frame to file done") break # 关闭h264文件 fo.close() # 销毁Streams streamManagerApi.DestroyAllStreams()
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。