RK3588 yolo v8实例分割部署

环境：
yolo v8版本：8.0.151

1.训练自己的模型（省）

from ultralytics import YOLO

if __name__ == '__main__':
    # 加载模型
    model = YOLO(r'yolov8-seg.yaml')  # 不使用预训练权重训练
    # model = YOLO(r'yolov8-seg.yaml').load("yolov8n-seg.pt")  # 使用预训练权重训练
    # 训练参数 ----------------------------------------------------------------------------------------------
    model.train(
        data=r'coco128bak-seg.yaml',
        epochs=300,  # (int) 训练的周期数
        batch=64,  # (int) 每批次的图像数量（-1 为自动批处理）
        device='',  # (int | str | list, optional) 运行的设备，例如 cuda device=0 或 device=0,1,2,3 或 device=cpu
        workers=8,  # (int) 数据加载的工作线程数（每个DDP进程）
    )
1
2
3
4
5
6
7
8
9
10
11
12
13
14

2.pt模型转onnx模型再转rknn模型

a.修改 ultralytics/cfg/defualt.yaml中的model字段
model: 你训练后生成的模型文件路径/best.pt 

b.执行ultralytics/engine/exporter.py将.pt模型转为onnx模型,生成的模型会在打印信息中给出。
python ultralytics/engine/exporter.py

c.借助toolkit2将onnx转为rknn模型，此处省略toolkit2的相关安装，参见:[RK git](https://github.com/rockchip-linux/rknn-toolkit2)
转换命令例如：python convert.py yolov8-seg.onnx rk3588
1
2
3
4
5
6
7
8

3.交叉编译C++部署代码
rk已经提供了参考的部署代码：rknn_model_zoo仓库
clone代码下来，在rknn_model_zoo/build-linux.sh中修改交叉编译工具链路径就可以编译官方deme了
例如本人修改下：
执行：

chmod 777 build-linux.sh
./build-linux.sh -t rk3588 -a aarch64 -d yolov8
1
2

编译的文件会全部打包到install，修改install/rk3588_linux_aarch64/rknn_yolov8_seg_demo/model/coco_80_labels_list.txt中的内容，修改为自己的类别，将上述所转换的rknn模型与该文件压缩打包到rk3588平台上就可以了

4.部署到rk3588平台
在平台上根据程序执行的参数格式运行程序

./rknn_yolov8_seg_demo model/yolov8-seg.rknn test.jpg

执行效果：
load lable ./model/coco_80_labels_list.txt
model input num: 1, output num: 13
input tensors:
  index=0, name=images, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
output tensors:
  index=0, name=375, n_dims=4, dims=[1, 64, 80, 80], n_elems=409600, size=409600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-3, scale=0.113323
  index=1, name=onnx::ReduceSum_383, n_dims=4, dims=[1, 10, 80, 80], n_elems=64000, size=64000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003346
  index=2, name=388, n_dims=4, dims=[1, 1, 80, 80], n_elems=6400, size=6400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003505
  index=3, name=354, n_dims=4, dims=[1, 32, 80, 80], n_elems=204800, size=204800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-9, scale=0.023833
  index=4, name=395, n_dims=4, dims=[1, 64, 40, 40], n_elems=102400, size=102400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=7, scale=0.094574
  index=5, name=onnx::ReduceSum_403, n_dims=4, dims=[1, 10, 40, 40], n_elems=16000, size=16000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003012
  index=6, name=407, n_dims=4, dims=[1, 1, 40, 40], n_elems=1600, size=1600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003036
  index=7, name=361, n_dims=4, dims=[1, 32, 40, 40], n_elems=51200, size=51200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-5, scale=0.022464
  index=8, name=414, n_dims=4, dims=[1, 64, 20, 20], n_elems=25600, size=25600, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=3, scale=0.068694
  index=9, name=onnx::ReduceSum_422, n_dims=4, dims=[1, 10, 20, 20], n_elems=4000, size=4000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.002477
  index=10, name=426, n_dims=4, dims=[1, 1, 20, 20], n_elems=400, size=400, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.002487
  index=11, name=368, n_dims=4, dims=[1, 32, 20, 20], n_elems=12800, size=12800, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-3, scale=0.018722
  index=12, name=347, n_dims=4, dims=[1, 32, 160, 160], n_elems=819200, size=819200, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-117, scale=0.025950
model is NHWC input fmt
model input height=640, width=640, channel=3
origin size=3984x1840 crop size=3984x1840
input image: 3984 x 1840, subsampling: 4:2:0, colorspace: YCbCr, orientation: 1
scale=0.160643 dst_box=(0 172 639 465) allow_slight_change=1 _left_offset=0 _top_offset=172 padding_w=0 padding_h=346
src width=3984 height=1840 fmt=0x1 virAddr=0x0x7fbbf9e010 fd=0
dst width=640 height=640 fmt=0x1 virAddr=0x0x38ec4000 fd=0
src_box=(0 0 3983 1839)
dst_box=(0 172 639 465)
color=0x72
rga_api version 1.10.0_[2]
fill dst image (x y w h)=(0 0 640 640) with color=0x72727272
 RgaCollorFill(1756) RGA_COLORFILL fail: Invalid argument
 RgaCollorFill(1757) RGA_COLORFILL fail: Invalid argument
230 im2d_rga_impl rga_task_submit(2100): Failed to call RockChipRga interface, please use 'dmesg' command to view driver error log.
230 im2d_rga_impl rga_dump_channel_info(1452): src_channel: 
  rect[x,y,w,h] = [0, 0, 0, 0]
  image[w,h,ws,hs,f] = [0, 0, 0, 0, rgba8888]
  buffer[handle,fd,va,pa] = [0, 0, 0, 0]
  color_space = 0x0, global_alpha = 0x0, rd_mode = 0x0

230 im2d_rga_impl rga_dump_channel_info(1452): dst_channel: 
  rect[x,y,w,h] = [0, 0, 640, 640]
  image[w,h,ws,hs,f] = [640, 640, 640, 640, rgb888]
  buffer[handle,fd,va,pa] = [108, 0, 0, 0]
  color_space = 0x0, global_alpha = 0xff, rd_mode = 0x1

230 im2d_rga_impl rga_dump_opt(1502): opt version[0x0]:

230 im2d_rga_impl rga_dump_opt(1503): set_core[0x0], priority[0]

230 im2d_rga_impl rga_dump_opt(1506): color[0x72727272] 
230 im2d_rga_impl rga_dump_opt(1515): 

230 im2d_rga_impl rga_task_submit(2109): acquir_fence[-1], release_fence_ptr[0x0], usage[0x280000]

rknn_run
carpet @ (2975 902 3965 1475) 0.768
wire @ (1170 547 2203 1425) 0.768
null @ (946 31 1145 168) 0.512
write_image path: out.png width=3984 height=1840 channel=3 data=0x7fbbf9e010

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

参考：
https://doc.embedfire.com/linux/rk356x/Ai/zh/latest/README.html
参考的yolo v8代码：https://gitee.com/ysxgitee/rk3588_npu_yolov8_2.git