devbox
喵喵爱编程
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

使用YOLOv5-Lite在树莓派4b上部署车辆检测模型（三）——在树莓派4b上部署模型_v5lite-e.pt

作者：喵喵爱编程 | 2024-07-13 13:23:25

赞

踩

v5lite-e.pt

目录

一、前言

二、树莓派环境配置

三、MNN框架编译及模型转换

四、推理

参考博文

一、前言

YOLOv5-Lite项目：https://github.com/ppogg/YOLOv5-Lite

使用YOLOv5-Lite在树莓派4b上部署车辆检测模型（一）——UA-DETRAC车辆检测数据集的处理-CSDN博客

使用YOLOv5-Lite在树莓派4b上部署车辆检测模型（二）——使用数据集训练模型-CSDN博客

        在前面的两篇文章中，我们处理好了UA-DETRAC数据集，并使用这个数据集训练了v5Lite-e模型。现在我们有了训练出的pt格式的权重文件，暂且称为v5Lite-e.pt，接下来需要将这个权重文件转换格式，然后在树莓派4B上部署。

二、树莓派环境配置

        系统镜像源推荐使用官方下载器Raspberry Pi Imager，下载网址：https://www.raspberrypi.com/software/icon-default.png?t=N7T8https://www.raspberrypi.com/software/        系统选择raspbian，不同大版本的raspbian环境有微妙的区别，建议选择64位的bullseye版本系统。官方镜像源网站：

https://www.raspberrypi.com/software/operating-systems/#raspberry-pi-os-legacy-64-biticon-default.png?t=N7T8https://www.raspberrypi.com/software/operating-systems/#raspberry-pi-os-legacy-64-bit        烧录好镜像源后，在树莓派上执行下面的命令安装依赖：

  1. sudo apt-get install libprotobuf-dev protobuf-compiler
  2. sudo apt-get install cmake
  3. sudo apt-get install libopencv-dev

        如果要使用摄像头，执行sudo raspi-config打开配置面板，激活摄像头。

三、MNN框架编译及模型转换

        MNN框架选择2.7.1版本，执行下面命令进行编译：

  1. mkdir build
  2. cd build
  3. cmake .. -DMNN_BUILD_CONVERTER=ON -DMNN_BUILD_TOOL=ON -DMNN_BUILD_QUANTOOLS=ON -DMNN_EVALUATION=ON -DMNN_SUPPORT_BF16=ON -DMNN_ARM82=ON -DMNN_BUILD_OPENCV=ON -DMNN_USE_OPENCV=ON
  4. make -j

       cmake的参数介绍见MNN项目wiki：https://github.com/alibaba/MNN/wiki/cmake#%E7%BC%96%E8%AF%91%E5%AE%8F%E4%BB%8B%E7%BB%8D

        编译过程可能会报错：

/home/pi/MNN-2.7.1/source/backend/cpu/arm/arm64/bf16/ARMV86_MNNPackedMatMulRemain_BF16.S:158: Fatal error: macros nested too deeply

        这是因为源码的汇编指令嵌套过深，会导致编译时定义的宏无法展开，需要将报错文件中的FMAX和FMIN这两个指令展开。

        将ARMV86_MNNPackedMatMul_BF16.S、ARMV86_MNNPackedMatMulRemain_BF16.S这两个文件的内容替换为下面给出的即可解决报错。

        ARMV86_MNNPackedMatMul_BF16.S

  1. //
  2. //  ARMV86_MNNPackedMatMul_BF16.S
  3. //  MNN
  4. //
  5. //  Created by MNN on 2022/10/09.
  6. //  Copyright © 2018-2021 Alibaba Group Holding Limited
  7. //
  8. #ifdef __aarch64__
  9.  
  10. #include "MNNAsmGlobal.h"
  11.  
  12. .text
  13. .align 5
  14.  
  15. .macro SET_ZERO d0, d1, d2, d3
  16.     movi \d0\().4s, #0
  17.     movi \d1\().4s, #0
  18.     movi \d2\().4s, #0
  19.     movi \d3\().4s, #0
  20. .endm
  21.  
  22. .macro Float32ToBf16 d0, d1, d2, d3
  23.     shrn \d0\().4h, \d0\().4s, #16
  24.     shrn \d1\().4h, \d1\().4s, #16
  25.     shrn \d2\().4h, \d2\().4s, #16
  26.     shrn \d3\().4h, \d3\().4s, #16
  27. .endm
  28.  
  29.  
  30.  
  31. .macro SET_BIAS s, d0, d1, d2, d3
  32.     mov \d0\().16b, \s\().16b
  33.     mov \d1\().16b, \s\().16b
  34.     mov \d2\().16b, \s\().16b
  35.     mov \d3\().16b, \s\().16b
  36. .endm
  37.  
  38. // 12 * 8 * 4 MatMul
  39. asm_function ARMV86_MNNPackedMatMul_BF16
  40. //void ARMV86_MNNPackedMatMul_BF16(float* C, const float* A, const float* B, const size_t* parameter, const float* postParameters, const float* bias);
  41. // x0: C, x1:A, x2:B, x3:parameter, x4: postParameters, x5:bias
  42. stp d14, d15, [sp, #-64]!
  43. stp d12, d13, [sp, #16]
  44. stp d10, d11, [sp, #32]
  45. stp d8,  d9,  [sp, #48]
  46.  
  47. //ldr x8, [x3, #0] // deprecated
  48. ldr x9, [x3, #8] // l
  49. ldr x10, [x3, #16] // h
  50. mov x11, #64  // B_stride = LP * HP = 4 * 8 * sizeof(int16_t)
  51.  
  52. ldr x13, [x3, #24] // cStride
  53. ldr x7, [x3, #40] // bExtraStride
  54.  
  55. add x10, x10, #3
  56. lsr x10, x10, #2
  57. add x9, x9, #3
  58. lsr x9, x9, #2
  59.  
  60. cbz x4, Start
  61. ld1 {v5.4s}, [x4]
  62. mov w17, v5.s[2] // min value
  63. mov w18, v5.s[3] // max value
  64.  
  65. Start:
  66.     cmp x10, #2
  67.     blt LH4
  68. LH8:
  69.     sub x14, x13, #96 // cStride - 96
  70. LoopH:
  71.     mov x15, x1
  72.     mov x12, x9
  73.     cbz x5, NoBiasH8
  74.     ld1 {v0.4h, v1.4h}, [x5], #16 // 8 * sizeof(int16_t)
  75.     shll v0.4s, v0.4h, #16
  76.     shll v1.4s, v1.4h, #16
  77.     mov v2.16b, v0.16b
  78.     mov v3.16b, v1.16b
  79.     uzp1 v18.2d, v0.2d, v2.2d   // bias_0, bias_1, bias_0, bias_1
  80.     uzp2 v19.2d, v0.2d, v2.2d   // bias_2, bias_3, bias_2, bias_3
  81.     uzp1 v30.2d, v1.2d, v3.2d   // bias_0, bias_1, bias_0, bias_1
  82.     uzp2 v31.2d, v1.2d, v3.2d   // bias_2, bias_3, bias_2, bias_3
  83.     SET_BIAS v18, v8, v10, v12, v14
  84.     mov v16.16b, v18.16b
  85.     SET_BIAS v19, v9, v11, v13, v15
  86.     mov v17.16b, v19.16b
  87.     SET_BIAS v30, v20, v22, v24, v26
  88.     mov v28.16b, v30.16b
  89.     SET_BIAS v31, v21, v23, v25, v27
  90.     mov v29.16b, v31.16b
  91.     b LoopL
  92.     NoBiasH8:
  93.         SET_ZERO v8, v9, v10, v11
  94.         SET_ZERO v12, v13, v14, v15
  95.         SET_ZERO v16, v17, v18, v19
  96.         SET_ZERO v20, v21, v22, v23
  97.         SET_ZERO v24, v25, v26, v27
  98.         SET_ZERO v28, v29, v30, v31
  99.     LoopL:
  100.         // A [12, 4, bf16] : rn = 6  : v2 - v7
  101.         // B [ 8, 4, bf16] : rn = 2  : v0 - v1
  102.         // C [12, 8, fp32] : rn = 24 : v8 - v31
  103.         ld1 {v2.8h, v3.8h, v4.8h, v5.8h}, [x15], #64 // A: 8 * 4 * sizeof(int16_t)
  104.         ld1 {v6.8h, v7.8h}, [x15], #32               // A: 4 * 4 * sizeof(int16_t)
  105.         ld1 {v0.8h, v1.8h}, [x2],  #32               // B: 4 * 4 * sizeof(int16_t) 
  106.         .inst 0x6e40ec48 // bfmmla v8.4s, v2.8h, v0.8h
  107.         .inst 0x6e41ec49 // bfmmla v9.4s, v2.8h, v1.8h
  108.         .inst 0x6e40ec6a // bfmmla v10.4s, v3.8h, v0.8h
  109.         .inst 0x6e41ec6b // bfmmla v11.4s, v3.8h, v1.8h
  110.         .inst 0x6e40ec8c // bfmmla v12.4s, v4.8h, v0.8h
  111.         .inst 0x6e41ec8d // bfmmla v13.4s, v4.8h, v1.8h
  112.         .inst 0x6e40ecae // bfmmla v14.4s, v5.8h, v0.8h
  113.         .inst 0x6e41ecaf // bfmmla v15.4s, v5.8h, v1.8h
  114.         .inst 0x6e40ecd0 // bfmmla v16.4s, v6.8h, v0.8h
  115.         .inst 0x6e41ecd1 // bfmmla v17.4s, v6.8h, v1.8h
  116.         .inst 0x6e40ecf2 // bfmmla v18.4s, v7.8h, v0.8h
  117.         .inst 0x6e41ecf3 // bfmmla v19.4s, v7.8h, v1.8h
  118.         ld1 {v0.8h, v1.8h}, [x2],  #32               // B: 4 * 4 * sizeof(int16_t) 
  119.         .inst 0x6e40ec54 // bfmmla v20.4s, v2.8h, v0.8h
  120.         .inst 0x6e41ec55 // bfmmla v21.4s, v2.8h, v1.8h
  121.         .inst 0x6e40ec76 // bfmmla v22.4s, v3.8h, v0.8h
  122.         .inst 0x6e41ec77 // bfmmla v23.4s, v3.8h, v1.8h
  123.         .inst 0x6e40ec98 // bfmmla v24.4s, v4.8h, v0.8h
  124.         .inst 0x6e41ec99 // bfmmla v25.4s, v4.8h, v1.8h
  125.         .inst 0x6e40ecba // bfmmla v26.4s, v5.8h, v0.8h
  126.         .inst 0x6e41ecbb // bfmmla v27.4s, v5.8h, v1.8h
  127.         .inst 0x6e40ecdc // bfmmla v28.4s, v6.8h, v0.8h
  128.         .inst 0x6e41ecdd // bfmmla v29.4s, v6.8h, v1.8h
  129.         .inst 0x6e40ecfe // bfmmla v30.4s, v7.8h, v0.8h
  130.         .inst 0x6e41ecff // bfmmla v31.4s, v7.8h, v1.8h
  131.         subs x12, x12, #1
  132.         bgt LoopL
  133.     LoopLEnd:
  134.         uzp1 v7.2d, v8.2d, v9.2d
  135.         uzp2 v8.2d, v8.2d, v9.2d
  136.         uzp1 v9.2d, v10.2d, v11.2d
  137.         uzp2 v10.2d, v10.2d, v11.2d
  138.         uzp1 v11.2d, v12.2d, v13.2d
  139.         uzp2 v12.2d, v12.2d, v13.2d
  140.         uzp1 v13.2d, v14.2d, v15.2d
  141.         uzp2 v14.2d, v14.2d, v15.2d
  142.         uzp1 v15.2d, v16.2d, v17.2d
  143.         uzp2 v16.2d, v16.2d, v17.2d
  144.         uzp1 v17.2d, v18.2d, v19.2d
  145.         uzp2 v18.2d, v18.2d, v19.2d
  146.         uzp1 v19.2d, v20.2d, v21.2d
  147.         uzp2 v20.2d, v20.2d, v21.2d
  148.         uzp1 v21.2d, v22.2d, v23.2d
  149.         uzp2 v22.2d, v22.2d, v23.2d
  150.         uzp1 v23.2d, v24.2d, v25.2d
  151.         uzp2 v24.2d, v24.2d, v25.2d
  152.         uzp1 v25.2d, v26.2d, v27.2d
  153.         uzp2 v26.2d, v26.2d, v27.2d
  154.         uzp1 v27.2d, v28.2d, v29.2d
  155.         uzp2 v28.2d, v28.2d, v29.2d
  156.         uzp1 v29.2d, v30.2d, v31.2d
  157.         uzp2 v30.2d, v30.2d, v31.2d
  158.         cbz x4, StoreLH8
  159.     PostTreatLH8:
  160.         dup v5.4s, w17
  161.         dup v6.4s, w18
  162.  
  163.  
  164.         fmax v7.4s,  v7.4s,  v5.4s
  165.         fmax v8.4s,  v8.4s,  v5.4s
  166.         fmax v9.4s,  v9.4s,  v5.4s
  167.         fmax v10.4s, v10.4s, v5.4s
  168.  
  169.  
  170.         fmax v11.4s, v11.4s, v5.4s
  171.         fmax v12.4s, v12.4s, v5.4s
  172.         fmax v13.4s, v13.4s, v5.4s
  173.         fmax v14.4s, v14.4s, v5.4s
  174.  
  175.  
  176.         fmax v15.4s, v15.4s, v5.4s
  177.         fmax v16.4s, v16.4s, v5.4s
  178.         fmax v17.4s, v17.4s, v5.4s
  179.         fmax v18.4s, v18.4s, v5.4s
  180.  
  181.  
  182.         fmax v19.4s, v19.4s, v5.4s
  183.         fmax v20.4s, v20.4s, v5.4s
  184.         fmax v21.4s, v21.4s, v5.4s
  185.         fmax v22.4s, v22.4s, v5.4s
  186.  
  187.  
  188.         fmax v23.4s, v23.4s, v5.4s
  189.         fmax v24.4s, v24.4s, v5.4s
  190.         fmax v25.4s, v25.4s, v5.4s
  191.         fmax v26.4s, v26.4s, v5.4s
  192.  
  193.         fmax v27.4s, v27.4s, v5.4s
  194.         fmax v28.4s, v28.4s, v5.4s
  195.         fmax v29.4s, v29.4s, v5.4s
  196.         fmax v30.4s, v30.4s, v5.4s
  197.  
  198.  
  199.  
  200.  
  201.  
  202.  
  203.         fmin v7.4s,  v7.4s,  v6.4s
  204.         fmin v8.4s,  v8.4s,  v6.4s
  205.         fmin v9.4s,  v9.4s,  v6.4s
  206.         fmin v10.4s, v10.4s, v6.4s
  207.  
  208.  
  209.         fmin v11.4s, v11.4s, v6.4s
  210.         fmin v12.4s, v12.4s, v6.4s
  211.         fmin v13.4s, v13.4s, v6.4s
  212.         fmin v14.4s, v14.4s, v6.4s
  213.  
  214.  
  215.         fmin v15.4s, v15.4s, v6.4s
  216.         fmin v16.4s, v16.4s, v6.4s
  217.         fmin v17.4s, v17.4s, v6.4s
  218.         fmin v18.4s, v18.4s, v6.4s
  219.  
  220.  
  221.         fmin v19.4s, v19.4s, v6.4s
  222.         fmin v20.4s, v20.4s, v6.4s
  223.         fmin v21.4s, v21.4s, v6.4s
  224.         fmin v22.4s, v22.4s, v6.4s
  225.  
  226.  
  227.         fmin v23.4s, v23.4s, v6.4s
  228.         fmin v24.4s, v24.4s, v6.4s
  229.         fmin v25.4s, v25.4s, v6.4s
  230.         fmin v26.4s, v26.4s, v6.4s
  231.     StoreLH8:
  232.         Float32ToBf16 v7, v8, v9, v10
  233.         Float32ToBf16 v11, v12, v13, v14
  234.         Float32ToBf16 v15, v16, v17, v18
  235.         Float32ToBf16 v19, v20, v21, v22
  236.         Float32ToBf16 v23, v24, v25, v26
  237.         Float32ToBf16 v27, v28, v29, v30
  238.         st1 {v7.4h, v8.4h, v9.4h, v10.4h},    [x0], #32 // 16 * sizeof(int16_t)
  239.         st1 {v11.4h, v12.4h, v13.4h, v14.4h}, [x0], #32 // 16 * sizeof(int16_t)
  240.         st1 {v15.4h, v16.4h, v17.4h, v18.4h}, [x0], #32 // 16 * sizeof(int16_t)
  241.         add x0, x0, x14
  242.         st1 {v19.4h, v20.4h, v21.4h, v22.4h}, [x0], #32 // 16 * sizeof(int16_t)
  243.         st1 {v23.4h, v24.4h, v25.4h, v26.4h}, [x0], #32 // 16 * sizeof(int16_t)
  244.         st1 {v27.4h, v28.4h, v29.4h, v30.4h}, [x0], #32 // 16 * sizeof(int16_t)
  245.         add x0, x0, x14
  246.         add x2, x2, x7 // weight stride
  247.         sub x10, x10, #2
  248.         cmp x10, #2
  249.         bge LoopH
  250. LH4:
  251. cbz x10, End
  252. LoopHR:
  253.     mov x15, x1
  254.     mov x12, x9
  255.     cbz x5, NoBiasH4
  256.     ld1 {v0.4h}, [x5], #8 // 8 * sizeof(int16_t)
  257.     shll v0.4s, v0.4h, #16
  258.     mov v2.16b, v0.16b
  259.     uzp1 v18.2d, v0.2d, v2.2d   // bias_0, bias_1, bias_0, bias_1
  260.     uzp2 v19.2d, v0.2d, v2.2d   // bias_2, bias_3, bias_2, bias_3
  261.     SET_BIAS v18, v8, v10, v12, v14
  262.     mov v16.16b, v18.16b
  263.     SET_BIAS v19, v9, v11, v13, v15
  264.     mov v17.16b, v19.16b
  265.     b LoopLR
  266.     NoBiasH4:
  267.         SET_ZERO v8, v9, v10, v11
  268.         SET_ZERO v12, v13, v14, v15
  269.         SET_ZERO v16, v17, v18, v19
  270.     LoopLR:
  271.         // A [12, 4, bf16] : rn = 6  : v2 - v7
  272.         // B [ 4, 4, bf16] : rn = 2  : v0 - v1
  273.         // C [12, 4, fp32] : rn = 12 : v8 - v19
  274.         ld1 {v2.8h, v3.8h, v4.8h, v5.8h}, [x15], #64 // A: 8 * 4 * sizeof(int16_t)
  275.         ld1 {v6.8h, v7.8h}, [x15], #32               // A: 4 * 4 * sizeof(int16_t)
  276.         ld1 {v0.8h, v1.8h}, [x2],  x11               // B: 4 * 4 * sizeof(int16_t)
  277.         .inst 0x6e40ec48 // bfmmla v8.4s, v2.8h, v0.8h
  278.         .inst 0x6e41ec49 // bfmmla v9.4s, v2.8h, v1.8h
  279.         .inst 0x6e40ec6a // bfmmla v10.4s, v3.8h, v0.8h
  280.         .inst 0x6e41ec6b // bfmmla v11.4s, v3.8h, v1.8h
  281.         .inst 0x6e40ec8c // bfmmla v12.4s, v4.8h, v0.8h
  282.         .inst 0x6e41ec8d // bfmmla v13.4s, v4.8h, v1.8h
  283.         .inst 0x6e40ecae // bfmmla v14.4s, v5.8h, v0.8h
  284.         .inst 0x6e41ecaf // bfmmla v15.4s, v5.8h, v1.8h
  285.         .inst 0x6e40ecd0 // bfmmla v16.4s, v6.8h, v0.8h
  286.         .inst 0x6e41ecd1 // bfmmla v17.4s, v6.8h, v1.8h
  287.         .inst 0x6e40ecf2 // bfmmla v18.4s, v7.8h, v0.8h
  288.         .inst 0x6e41ecf3 // bfmmla v19.4s, v7.8h, v1.8h
  289.         subs x12, x12, #1
  290.         bgt LoopLR
  291.     LoopLREnd:
  292.         add x2, x2, x7 // weight stride
  293.         uzp1 v7.2d, v8.2d, v9.2d
  294.         uzp2 v8.2d, v8.2d, v9.2d
  295.         uzp1 v9.2d, v10.2d, v11.2d
  296.         uzp2 v10.2d, v10.2d, v11.2d
  297.         uzp1 v11.2d, v12.2d, v13.2d
  298.         uzp2 v12.2d, v12.2d, v13.2d
  299.         uzp1 v13.2d, v14.2d, v15.2d
  300.         uzp2 v14.2d, v14.2d, v15.2d
  301.         uzp1 v15.2d, v16.2d, v17.2d
  302.         uzp2 v16.2d, v16.2d, v17.2d
  303.         uzp1 v17.2d, v18.2d, v19.2d
  304.         uzp2 v18.2d, v18.2d, v19.2d
  305.         cbz x4, StoreLH4
  306.     PostTreatLH4:
  307.         dup v5.4s, w17
  308.         dup v6.4s, w18
  309.         fmax v7.4s,  v7.4s,  v5.4s
  310.         fmax v8.4s,  v8.4s,  v5.4s
  311.         fmax v9.4s,  v9.4s,  v5.4s
  312.         fmax v10.4s, v10.4s, v5.4s
  313.  
  314.  
  315.         fmax v11.4s, v11.4s, v5.4s
  316.         fmax v12.4s, v12.4s, v5.4s
  317.         fmax v13.4s, v13.4s, v5.4s
  318.         fmax v14.4s, v14.4s, v5.4s
  319.  
  320.  
  321.         fmax v15.4s, v15.4s, v5.4s
  322.         fmax v16.4s, v16.4s, v5.4s
  323.         fmax v17.4s, v17.4s, v5.4s
  324.         fmax v18.4s, v18.4s, v5.4s
  325.         fmin v7.4s,  v7.4s,  v6.4s
  326.         fmin v8.4s,  v8.4s,  v6.4s
  327.         fmin v9.4s,  v9.4s,  v6.4s
  328.         fmin v10.4s, v10.4s, v6.4s
  329.  
  330.  
  331.         fmin v11.4s, v11.4s, v6.4s
  332.         fmin v12.4s, v12.4s, v6.4s
  333.         fmin v13.4s, v13.4s, v6.4s
  334.         fmin v14.4s, v14.4s, v6.4s
  335.  
  336.  
  337.         fmin v15.4s, v15.4s, v6.4s
  338.         fmin v16.4s, v16.4s, v6.4s
  339.         fmin v17.4s, v17.4s, v6.4s
  340.         fmin v18.4s, v18.4s, v6.4s
  341.     StoreLH4:
  342.         Float32ToBf16 v7, v8, v9, v10
  343.         Float32ToBf16 v11, v12, v13, v14
  344.         Float32ToBf16 v15, v16, v17, v18
  345.         st1 {v7.4h, v8.4h, v9.4h, v10.4h},    [x0], #32 // 16 * sizeof(int16_t)
  346.         st1 {v11.4h, v12.4h, v13.4h, v14.4h}, [x0], #32 // 16 * sizeof(int16_t)
  347.         st1 {v15.4h, v16.4h, v17.4h, v18.4h}, [x0], #32 // 16 * sizeof(int16_t)
  348. End:
  349. ldp d8,  d9,  [sp, #48]
  350. ldp d10, d11, [sp, #32]
  351. ldp d12, d13, [sp, #16]
  352. ldp d14, d15, [sp], #64
  353. ret
  354.  
  355. #endif

        ARMV86_MNNPackedMatMulRemain_BF16.S

  1. //
  2. //  ARMV86_MNNPackedMatMulRemain_BF16.S
  3. //  MNN
  4. //
  5. //  Created by MNN on 2022/10/09.
  6. //  Copyright © 2018-2021 Alibaba Group Holding Limited
  7. //
  8.  
  9. #ifdef __aarch64__
  10.  
  11. #include "MNNAsmGlobal.h"
  12.  
  13. .text
  14. .align 5
  15. .macro SET_ZERO d0, d1, d2, d3
  16.     movi \d0\().4s, #0
  17.     movi \d1\().4s, #0
  18.     movi \d2\().4s, #0
  19.     movi \d3\().4s, #0
  20. .endm
  21.  
  22. .macro Float32ToBf16 d0, d1, d2, d3
  23.     shrn \d0\().4h, \d0\().4s, #16
  24.     shrn \d1\().4h, \d1\().4s, #16
  25.     shrn \d2\().4h, \d2\().4s, #16
  26.     shrn \d3\().4h, \d3\().4s, #16
  27. .endm
  28.  
  29.  
  30.  
  31. .macro SET_BIAS s, d0, d1, d2
  32.     mov \d0\().16b, \s\().16b
  33.     mov \d1\().16b, \s\().16b
  34.     mov \d2\().16b, \s\().16b
  35. .endm
  36.  
  37. // 12 * 8 * 4 MatMul
  38. asm_function ARMV86_MNNPackedMatMulRemain_BF16
  39. //void ARMV86_MNNPackedMatMulRemain_BF16(float* C, const float* A, const float* B, size_t eSize, const size_t* parameter, const float* postParameters, const float* bias);
  40. //Auto x0: C, x1:A, x2:B, x3:eSize, x4:parameter, x5:postParameters, x6:bias
  41. sub sp, sp, #32
  42. str x19, [sp, #0]
  43. str x20, [sp, #8]
  44. str x21, [sp, #16]
  45. ldr x11, [x4, #0] // aStride
  46. ldr x9, [x4, #8] // l
  47. ldr x10, [x4, #16] // h
  48. lsl x11, x11, #2 // aStride * 4
  49. mov x16, #64  // B_stride = LP * HP = 4 * 8 * sizeof(int16_t)
  50.  
  51. ldr x7, [x4, #24] // cStride
  52. ldr x19, [x4, #40] // bExtraStride
  53.  
  54. add x10, x10, #3
  55. lsr x10, x10, #2
  56. add x9, x9, #3
  57. lsr x9, x9, #2
  58.  
  59. cbz x5, Start
  60. ld1 {v5.4s}, [x5]
  61. dup v9.4s, v5.s[2] // Min Value
  62. dup v10.4s, v5.s[3] // Max Value
  63.  
  64. Start:
  65.  
  66. E8:
  67. cmp x3, #8
  68. blt E4
  69.  
  70. LoopE8: // e, TILE_BLOCK size is 8
  71.     mov x20, x6 // bias
  72.     mov x8, x10 // updiv(h, 4)
  73.     mov x21, x0 // dest, C
  74.     mov x13, x2 // weight, B
  75.  
  76.     LH8:
  77.     cmp x8, #2 // h/4 > 2
  78.     blt LH4
  79.     sub x14, x7, #64 // cStride - 64
  80.     LoopH8x8:
  81.         mov x15, x1 // src, A
  82.         mov x12, x9 // l
  83.         cbz x5, NoBiasLH8
  84.         ld1 {v0.4h, v1.4h}, [x20], #16 // 8 * sizeof(int16_t)
  85.         shll v0.4s, v0.4h, #16
  86.         shll v1.4s, v1.4h, #16
  87.         mov v2.16b, v0.16b
  88.         mov v3.16b, v1.16b
  89.         uzp1 v16.2d, v0.2d, v2.2d   // bias_0, bias_1, bias_0, bias_1
  90.         uzp2 v17.2d, v0.2d, v2.2d   // bias_2, bias_3, bias_2, bias_3
  91.         uzp1 v24.2d, v1.2d, v3.2d   // bias_0, bias_1, bias_0, bias_1
  92.         uzp2 v25.2d, v1.2d, v3.2d   // bias_2, bias_3, bias_2, bias_3
  93.         SET_BIAS v16, v18, v20, v22
  94.         SET_BIAS v17, v19, v21, v23
  95.         SET_BIAS v24, v26, v28, v30
  96.         SET_BIAS v25, v27, v29, v31
  97.         b LoopL
  98.         NoBiasLH8:
  99.             SET_ZERO v16, v17, v18, v19
  100.             SET_ZERO v20, v21, v22, v23
  101.             SET_ZERO v24, v25, v26, v27
  102.             SET_ZERO v28, v29, v30, v31
  103.         LoopL:
  104.             // A [8, 4, bf16] : rn = 4  : v4 - v7
  105.             // B [8, 4, bf16] : rn = 4  : v0 - v3
  106.             // C [8, 8, fp32] : rn = 16 : v16 - v31
  107.             ld1 {v4.8h, v5.8h, v6.8h, v7.8h}, [x15], x11 // A: 8 * 4 * sizeof(int16_t)
  108.             ld1 {v0.8h, v1.8h, v2.8h, v3.8h}, [x13], x16 // B: 8 * 4 * sizeof(int16_t)
  109.             .inst 0x6e40ec90 // bfmmla v16.4s, v4.8h, v0.8h
  110.             .inst 0x6e41ec91 // bfmmla v17.4s, v4.8h, v1.8h
  111.             .inst 0x6e40ecb2 // bfmmla v18.4s, v5.8h, v0.8h
  112.             .inst 0x6e41ecb3 // bfmmla v19.4s, v5.8h, v1.8h
  113.             .inst 0x6e40ecd4 // bfmmla v20.4s, v6.8h, v0.8h
  114.             .inst 0x6e41ecd5 // bfmmla v21.4s, v6.8h, v1.8h
  115.             .inst 0x6e40ecf6 // bfmmla v22.4s, v7.8h, v0.8h
  116.             .inst 0x6e41ecf7 // bfmmla v23.4s, v7.8h, v1.8h
  117.             .inst 0x6e42ec98 // bfmmla v24.4s, v4.8h, v2.8h
  118.             .inst 0x6e43ec99 // bfmmla v25.4s, v4.8h, v3.8h
  119.             .inst 0x6e42ecba // bfmmla v26.4s, v5.8h, v2.8h
  120.             .inst 0x6e43ecbb // bfmmla v27.4s, v5.8h, v3.8h
  121.             .inst 0x6e42ecdc // bfmmla v28.4s, v6.8h, v2.8h
  122.             .inst 0x6e43ecdd // bfmmla v29.4s, v6.8h, v3.8h
  123.             .inst 0x6e42ecfe // bfmmla v30.4s, v7.8h, v2.8h
  124.             .inst 0x6e43ecff // bfmmla v31.4s, v7.8h, v3.8h
  125.             subs x12, x12, #1
  126.             bgt LoopL
  127.         LoopLEnd:
  128.             uzp1 v15.2d, v16.2d, v17.2d
  129.             uzp2 v16.2d, v16.2d, v17.2d
  130.             uzp1 v17.2d, v18.2d, v19.2d
  131.             uzp2 v18.2d, v18.2d, v19.2d
  132.             uzp1 v19.2d, v20.2d, v21.2d
  133.             uzp2 v20.2d, v20.2d, v21.2d
  134.             uzp1 v21.2d, v22.2d, v23.2d
  135.             uzp2 v22.2d, v22.2d, v23.2d
  136.             uzp1 v23.2d, v24.2d, v25.2d
  137.             uzp2 v24.2d, v24.2d, v25.2d
  138.             uzp1 v25.2d, v26.2d, v27.2d
  139.             uzp2 v26.2d, v26.2d, v27.2d
  140.             uzp1 v27.2d, v28.2d, v29.2d
  141.             uzp2 v28.2d, v28.2d, v29.2d
  142.             uzp1 v29.2d, v30.2d, v31.2d
  143.             uzp2 v30.2d, v30.2d, v31.2d
  144.             cbz x5, StoreLH8
  145.         PostTreatLH8:
  146.             fmax v15.4s, v15.4s, v9.4s
  147.             fmax v16.4s, v16.4s, v9.4s
  148.             fmax v17.4s, v17.4s, v9.4s
  149.             fmax v18.4s, v18.4s, v9.4s
  150.  
  151.  
  152.             fmax v19.4s, v19.4s, v9.4s
  153.             fmax v20.4s, v20.4s, v9.4s
  154.             fmax v21.4s, v21.4s, v9.4s
  155.             fmax v22.4s, v22.4s, v9.4s
  156.  
  157.  
  158.             fmax v23.4s, v23.4s, v9.4s
  159.             fmax v24.4s, v24.4s, v9.4s
  160.             fmax v25.4s, v25.4s, v9.4s
  161.             fmax v26.4s, v26.4s, v9.4s
  162.  
  163.  
  164.             fmax v27.4s, v27.4s, v9.4s
  165.             fmax v28.4s, v28.4s, v9.4s
  166.             fmax v29.4s, v29.4s, v9.4s
  167.             fmax v30.4s, v30.4s, v9.4s
  168.  
  169.  
  170.             fmin v15.4s, v15.4s, v10.4s
  171.             fmin v16.4s, v16.4s, v10.4s
  172.             fmin v17.4s, v17.4s, v10.4s
  173.             fmin v18.4s, v18.4s, v10.4s
  174.  
  175.  
  176.             fmin v19.4s, v19.4s, v10.4s
  177.             fmin v20.4s, v20.4s, v10.4s
  178.             fmin v21.4s, v21.4s, v10.4s
  179.             fmin v22.4s, v22.4s, v10.4s
  180.  
  181.  
  182.             fmin v23.4s, v23.4s, v10.4s
  183.             fmin v24.4s, v24.4s, v10.4s
  184.             fmin v25.4s, v25.4s, v10.4s
  185.             fmin v26.4s, v26.4s, v10.4s
  186.  
  187.  
  188.             fmin v27.4s, v27.4s, v10.4s
  189.             fmin v28.4s, v28.4s, v10.4s
  190.             fmin v29.4s, v29.4s, v10.4s
  191.             fmin v30.4s, v30.4s, v10.4s
  192.         StoreLH8:
  193.             Float32ToBf16 v15, v16, v17, v18
  194.             Float32ToBf16 v19, v20, v21, v22
  195.             Float32ToBf16 v23, v24, v25, v26
  196.             Float32ToBf16 v27, v28, v29, v30
  197.             st1 {v15.4h, v16.4h, v17.4h, v18.4h}, [x0], #32 // 16 * sizeof(int16_t)
  198.             st1 {v19.4h, v20.4h, v21.4h, v22.4h}, [x0], #32 // 16 * sizeof(int16_t)
  199.             add x0, x0, x14
  200.             st1 {v23.4h, v24.4h, v25.4h, v26.4h}, [x0], #32 // 16 * sizeof(int16_t)
  201.             st1 {v27.4h, v28.4h, v29.4h, v30.4h}, [x0], #32 // 16 * sizeof(int16_t)
  202.             add x0, x0, x14
  203.             add x13, x13, x19 // weight stride
  204.             sub x8, x8, #2
  205.             cmp x8, #2
  206.             bge LoopH8x8
  207.     LH4:
  208.     cbz x8, E8End
  209.     LoopHRemain:
  210.         mov x15, x1
  211.         mov x12, x9
  212.         cbz x5, NoBiasHRemain
  213.         ld1 {v0.4h}, [x20]
  214.         shll v0.4s, v0.4h, #16
  215.         mov v2.16b, v0.16b
  216.         uzp1 v16.2d, v0.2d, v2.2d   // bias_0, bias_1, bias_0, bias_1
  217.         uzp2 v17.2d, v0.2d, v2.2d   // bias_2, bias_3, bias_2, bias_3
  218.         SET_BIAS v16, v18, v20, v22
  219.         SET_BIAS v17, v19, v21, v23
  220.         b LoopLR
  221.         NoBiasHRemain:
  222.             SET_ZERO v16, v17, v18, v19
  223.             SET_ZERO v20, v21, v22, v23
  224.         LoopLR:
  225.             // A [8, 4, bf16] : rn = 4  : v4 - v7
  226.             // B [4, 4, bf16] : rn = 2  : v0 - v1
  227.             // C [8, 4, fp32] : rn = 8  : v16 - v23
  228.             ld1 {v4.8h, v5.8h, v6.8h, v7.8h}, [x15], x11 // A: 8 * 4 * sizeof(int16_t)
  229.             ld1 {v0.8h, v1.8h}, [x13],  x16              // B: 4 * 4 * sizeof(int16_t)
  230.             .inst 0x6e40ec90 // bfmmla v16.4s, v4.8h, v0.8h
  231.             .inst 0x6e41ec91 // bfmmla v17.4s, v4.8h, v1.8h
  232.             .inst 0x6e40ecb2 // bfmmla v18.4s, v5.8h, v0.8h
  233.             .inst 0x6e41ecb3 // bfmmla v19.4s, v5.8h, v1.8h
  234.             .inst 0x6e40ecd4 // bfmmla v20.4s, v6.8h, v0.8h
  235.             .inst 0x6e41ecd5 // bfmmla v21.4s, v6.8h, v1.8h
  236.             .inst 0x6e40ecf6 // bfmmla v22.4s, v7.8h, v0.8h
  237.             .inst 0x6e41ecf7 // bfmmla v23.4s, v7.8h, v1.8h
  238.             subs x12, x12, #1
  239.             bne LoopLR
  240.         LoopLREnd:
  241.             uzp1 v15.2d, v16.2d, v17.2d
  242.             uzp2 v16.2d, v16.2d, v17.2d
  243.             uzp1 v17.2d, v18.2d, v19.2d
  244.             uzp2 v18.2d, v18.2d, v19.2d
  245.             uzp1 v19.2d, v20.2d, v21.2d
  246.             uzp2 v20.2d, v20.2d, v21.2d
  247.             uzp1 v21.2d, v22.2d, v23.2d
  248.             uzp2 v22.2d, v22.2d, v23.2d
  249.             cbz x5, StoreLH8x4
  250.         PostTreatLH8x4:
  251.             fmax v15.4s, v15.4s, v9.4s
  252.             fmax v16.4s, v16.4s, v9.4s
  253.             fmax v17.4s, v17.4s, v9.4s
  254.             fmax v18.4s, v18.4s, v9.4s
  255.  
  256.  
  257.             fmax v19.4s, v19.4s, v9.4s
  258.             fmax v20.4s, v20.4s, v9.4s
  259.             fmax v21.4s, v21.4s, v9.4s
  260.             fmax v22.4s, v22.4s, v9.4s
  261.  
  262.             fmin v15.4s, v15.4s, v10.4s
  263.             fmin v16.4s, v16.4s, v10.4s
  264.             fmin v17.4s, v17.4s, v10.4s
  265.             fmin v18.4s, v18.4s, v10.4s
  266.  
  267.  
  268.             fmin v19.4s, v19.4s, v10.4s
  269.             fmin v20.4s, v20.4s, v10.4s
  270.             fmin v21.4s, v21.4s, v10.4s
  271.             fmin v22.4s, v22.4s, v10.4s
  272.         StoreLH8x4:
  273.             Float32ToBf16 v15, v16, v17, v18
  274.             Float32ToBf16 v19, v20, v21, v22
  275.             st1 {v15.4h, v16.4h, v17.4h, v18.4h}, [x0], #32 // 16 * sizeof(int16_t)
  276.             st1 {v19.4h, v20.4h, v21.4h, v22.4h}, [x0], #32 // 16 * sizeof(int16_t)
  277.     E8End:
  278.         sub x3, x3, #8
  279.         cmp x3, #8
  280.         add x0, x21, #64 // move dest address of 8 * 4 * sizeof(int16_t)
  281.         add x1, x1, #64  // move A matrix address of 8 * 4 * sizeof(int16_t)
  282.         bge LoopE8
  283.  
  284. E4:
  285. cmp x3, #4
  286. mov x20, x6
  287. blt E2
  288.  
  289. mov x8, x10
  290. mov x21, x0
  291. mov x13, x2
  292.  
  293. cmp x8, #2
  294. blt E4LH4
  295. E4LH8:
  296.     E4LoopH8:
  297.         mov x15, x1
  298.         mov x12, x9
  299.         cbz x5, NoBiasE4
  300.         ld1 {v0.4h, v1.4h}, [x20], #16 // 8 * sizeof(int16_t)
  301.         shll v0.4s, v0.4h, #16
  302.         shll v1.4s, v1.4h, #16
  303.         mov v2.16b, v0.16b
  304.         mov v3.16b, v1.16b
  305.         uzp1 v16.2d, v0.2d, v2.2d   // bias_0, bias_1, bias_0, bias_1
  306.         uzp2 v17.2d, v0.2d, v2.2d   // bias_2, bias_3, bias_2, bias_3
  307.         uzp1 v20.2d, v1.2d, v3.2d   // bias_0, bias_1, bias_0, bias_1
  308.         uzp2 v21.2d, v1.2d, v3.2d   // bias_2, bias_3, bias_2, bias_3
  309.         mov v18.16b, v16.16b
  310.         mov v19.16b, v17.16b
  311.         mov v22.16b, v20.16b
  312.         mov v23.16b, v21.16b
  313.         b E4LoopL
  314.         NoBiasE4:
  315.             SET_ZERO v16, v17, v18, v19
  316.             SET_ZERO v20, v21, v22, v23
  317.         E4LoopL:
  318.             // A [4, 4, bf16] : rn = 4  : v4 - v5
  319.             // B [8, 4, bf16] : rn = 4  : v0 - v3
  320.             // C [4, 8, fp32] : rn = 8  : v16 - v23
  321.             ld1 {v4.8h, v5.8h}, [x15], x11               // A: 4 * 4 * sizeof(int16_t)
  322.             ld1 {v0.8h, v1.8h, v2.8h, v3.8h}, [x13], x16 // B: 8 * 4 * sizeof(int16_t)
  323.             .inst 0x6e40ec90 // bfmmla v16.4s, v4.8h, v0.8h
  324.             .inst 0x6e41ec91 // bfmmla v17.4s, v4.8h, v1.8h
  325.             .inst 0x6e40ecb2 // bfmmla v18.4s, v5.8h, v0.8h
  326.             .inst 0x6e41ecb3 // bfmmla v19.4s, v5.8h, v1.8h
  327.             .inst 0x6e42ec94 // bfmmla v20.4s, v4.8h, v2.8h
  328.             .inst 0x6e43ec95 // bfmmla v21.4s, v4.8h, v3.8h
  329.             .inst 0x6e42ecb6 // bfmmla v22.4s, v5.8h, v2.8h
  330.             .inst 0x6e43ecb7 // bfmmla v23.4s, v5.8h, v3.8h
  331.             subs x12, x12, #1
  332.             bgt E4LoopL
  333.         E4LoopLEnd:
  334.             uzp1 v15.2d, v16.2d, v17.2d
  335.             uzp2 v16.2d, v16.2d, v17.2d
  336.             uzp1 v17.2d, v18.2d, v19.2d
  337.             uzp2 v18.2d, v18.2d, v19.2d
  338.             uzp1 v19.2d, v20.2d, v21.2d
  339.             uzp2 v20.2d, v20.2d, v21.2d
  340.             uzp1 v21.2d, v22.2d, v23.2d
  341.             uzp2 v22.2d, v22.2d, v23.2d
  342.             cbz x5, StoreLH4x8
  343.         PostTreatLH4x8:
  344.             fmax v15.4s, v15.4s, v9.4s
  345.             fmax v16.4s, v16.4s, v9.4s
  346.             fmax v17.4s, v17.4s, v9.4s
  347.             fmax v18.4s, v18.4s, v9.4s
  348.  
  349.  
  350.             fmax v19.4s, v19.4s, v9.4s
  351.             fmax v20.4s, v20.4s, v9.4s
  352.             fmax v21.4s, v21.4s, v9.4s
  353.             fmax v22.4s, v22.4s, v9.4s
  354.  
  355.             fmin v15.4s, v15.4s, v10.4s
  356.             fmin v16.4s, v16.4s, v10.4s
  357.             fmin v17.4s, v17.4s, v10.4s
  358.             fmin v18.4s, v18.4s, v10.4s
  359.  
  360.  
  361.             fmin v19.4s, v19.4s, v10.4s
  362.             fmin v20.4s, v20.4s, v10.4s
  363.             fmin v21.4s, v21.4s, v10.4s
  364.             fmin v22.4s, v22.4s, v10.4s
  365.         StoreLH4x8:
  366.             Float32ToBf16 v15, v16, v17, v18
  367.             Float32ToBf16 v19, v20, v21, v22          
  368.             st1 {v15.4h, v16.4h, v17.4h, v18.4h}, [x0], x7 // 16 * sizeof(int16_t)
  369.             st1 {v19.4h, v20.4h, v21.4h, v22.4h}, [x0], x7 // 16 * sizeof(int16_t)
  370.             add x13, x13, x19 // weight stride
  371.             sub x8, x8, #2
  372.             cmp x8, #2
  373.             bge E4LoopH8
  374.     E4LH4:
  375.         cbz x8, E4End
  376.         mov x15, x1
  377.         mov x12, x9
  378.         cbz x5, NoBiasE4R
  379.         ld1 {v0.4h}, [x20]
  380.         shll v0.4s, v0.4h, #16
  381.         mov v2.16b, v0.16b
  382.         uzp1 v16.2d, v0.2d, v2.2d   // bias_0, bias_1, bias_0, bias_1
  383.         uzp2 v17.2d, v0.2d, v2.2d   // bias_2, bias_3, bias_2, bias_3
  384.         mov v18.16b, v16.16b
  385.         mov v19.16b, v17.16b
  386.         b E4LoopLR
  387.         NoBiasE4R:
  388.             SET_ZERO v16, v17, v18, v19
  389.         E4LoopLR:
  390.             // A [4, 4, bf16] : rn = 4  : v4 - v5
  391.             // B [4, 4, bf16] : rn = 4  : v0 - v1
  392.             // C [4, 4, fp32] : rn = 4  : v16 - v19
  393.             ld1 {v4.8h, v5.8h}, [x15], x11 // A: 4 * 4 * sizeof(int16_t)
  394.             ld1 {v0.8h, v1.8h}, [x13], x16 // B: 4 * 4 * sizeof(int16_t)
  395.             .inst 0x6e40ec90 // bfmmla v16.4s, v4.8h, v0.8h
  396.             .inst 0x6e41ec91 // bfmmla v17.4s, v4.8h, v1.8h
  397.             .inst 0x6e40ecb2 // bfmmla v18.4s, v5.8h, v0.8h
  398.             .inst 0x6e41ecb3 // bfmmla v19.4s, v5.8h, v1.8h
  399.             subs x12, x12, #1
  400.             bgt E4LoopLR
  401.         E4LoopLREnd:
  402.             uzp1 v15.2d, v16.2d, v17.2d
  403.             uzp2 v16.2d, v16.2d, v17.2d
  404.             uzp1 v17.2d, v18.2d, v19.2d
  405.             uzp2 v18.2d, v18.2d, v19.2d
  406.             cbz x5, StoreLH4x4
  407.         PostTreatLH4x4:
  408.  
  409.             fmax v15.4s, v15.4s, v9.4s
  410.             fmax v16.4s, v16.4s, v9.4s
  411.             fmax v17.4s, v17.4s, v9.4s
  412.             fmax v18.4s, v18.4s, v9.4s
  413.             fmin v19.4s, v19.4s, v10.4s
  414.             fmin v20.4s, v20.4s, v10.4s
  415.             fmin v21.4s, v21.4s, v10.4s
  416.             fmin v22.4s, v22.4s, v10.4s
  417.  
  418.         StoreLH4x4:
  419.             Float32ToBf16 v15, v16, v17, v18
  420.             st1 {v15.4h, v16.4h, v17.4h, v18.4h}, [x0] // 16 * sizeof(int16_t)
  421.     E4End:
  422.         sub x3, x3, #4
  423.         add x0, x21, #32 // move dest address of 4 * 4 * sizeof(int16_t)
  424.         add x1, x1, #32 // move dest address of 4 * 4 * sizeof(int16_t)
  425.  
  426. E2:
  427. cmp x3, #2
  428. mov x20, x6
  429. blt E1
  430.  
  431. mov x8, x10
  432. mov x21, x0
  433. mov x13, x2
  434.  
  435. cmp x8, #2
  436. blt E2LH4
  437. E2LH8:
  438.     E2LoopH8:
  439.         mov x15, x1
  440.         mov x12, x9
  441.         cbz x5, NoBiasE2
  442.         ld1 {v0.4h, v1.4h}, [x20], #16
  443.         shll v0.4s, v0.4h, #16
  444.         shll v1.4s, v1.4h, #16
  445.         mov v2.16b, v0.16b
  446.         mov v3.16b, v1.16b
  447.         uzp1 v16.2d, v0.2d, v2.2d   // bias_0, bias_1, bias_0, bias_1
  448.         uzp2 v17.2d, v0.2d, v2.2d   // bias_2, bias_3, bias_2, bias_3
  449.         uzp1 v18.2d, v1.2d, v3.2d   // bias_0, bias_1, bias_0, bias_1
  450.         uzp2 v19.2d, v1.2d, v3.2d   // bias_2, bias_3, bias_2, bias_3
  451.         b E2LoopL
  452.         NoBiasE2:
  453.             SET_ZERO v16, v17, v18, v19
  454.         E2LoopL:
  455.             // A [2, 4, bf16] : rn = 1  : v4
  456.             // B [8, 4, bf16] : rn = 2  : v0 - v3
  457.             // C [2, 8, fp32] : rn = 4  : v16 - v19
  458.             ld1 {v4.8h}, [x15], x11                      // A: 2 * 4 * sizeof(int16_t)
  459.             ld1 {v0.8h, v1.8h, v2.8h, v3.8h}, [x13], x16 // B: 8 * 4 * sizeof(int16_t)
  460.             .inst 0x6e40ec90 // bfmmla v16.4s, v4.8h, v0.8h
  461.             .inst 0x6e41ec91 // bfmmla v17.4s, v4.8h, v1.8h
  462.             .inst 0x6e42ec92 // bfmmla v18.4s, v4.8h, v2.8h
  463.             .inst 0x6e43ec93 // bfmmla v19.4s, v4.8h, v3.8h
  464.             subs x12, x12, #1
  465.             bgt E2LoopL
  466.         E2LoopLEnd:
  467.             uzp1 v15.2d, v16.2d, v17.2d
  468.             uzp2 v16.2d, v16.2d, v17.2d
  469.             uzp1 v17.2d, v18.2d, v19.2d
  470.             uzp2 v18.2d, v18.2d, v19.2d
  471.             cbz x5, StoreLH2x8
  472.         PostTreatLH2x8:
  473.             fmax v15.4s, v15.4s, v9.4s
  474.             fmax v16.4s, v16.4s, v9.4s
  475.             fmax v17.4s, v17.4s, v9.4s
  476.             fmax v18.4s, v18.4s, v9.4s
  477.             fmin v15.4s, v15.4s, v10.4s
  478.             fmin v16.4s, v16.4s, v10.4s
  479.             fmin v17.4s, v17.4s, v10.4s
  480.             fmin v18.4s, v18.4s, v10.4s
  481.         StoreLH2x8:
  482.             Float32ToBf16 v15, v16, v17, v18
  483.             st1 {v15.4h, v16.4h}, [x0], x7 // 8 * sizeof(int16_t)
  484.             st1 {v17.4h, v18.4h}, [x0], x7 // 8 * sizeof(int16_t)
  485.             add x13, x13, x19 // weight stride
  486.             sub x8, x8, #2
  487.             cmp x8, #2
  488.             bge E2LoopH8
  489.     E2LH4:
  490.         cbz x8, E2End
  491.         mov x15, x1
  492.         mov x12, x9
  493.         cbz x5, NoBiasE2R
  494.         ld1 {v0.4h}, [x20]
  495.         shll v0.4s, v0.4h, #16
  496.         mov v2.16b, v0.16b
  497.         uzp1 v16.2d, v0.2d, v2.2d   // bias_0, bias_1, bias_0, bias_1
  498.         uzp2 v17.2d, v0.2d, v2.2d   // bias_2, bias_3, bias_2, bias_3
  499.         b E2LoopLR
  500.         NoBiasE2R:
  501.             movi v16.4s, #0
  502.             movi v17.4s, #0
  503.         E2LoopLR:
  504.             // A [2, 4, bf16] : rn = 1  : v4
  505.             // B [4, 4, bf16] : rn = 2  : v0 - v1
  506.             // C [2, 4, fp32] : rn = 2  : v16 - v17
  507.             ld1 {v4.8h}, [x15], x11        // A: 2 * 4 * sizeof(int16_t)
  508.             ld1 {v0.8h, v1.8h}, [x13], x16 // B: 4 * 4 * sizeof(int16_t) 
  509.             .inst 0x6e40ec90 // bfmmla v16.4s, v4.8h, v0.8h
  510.             .inst 0x6e41ec91 // bfmmla v17.4s, v4.8h, v1.8h
  511.             subs x12, x12, #1
  512.             bgt E2LoopLR
  513.         E2LoopLREnd:
  514.             uzp1 v15.2d, v16.2d, v17.2d
  515.             uzp2 v16.2d, v16.2d, v17.2d
  516.             cbz x5, StoreLH2x4
  517.         PostTreatLH2x4:
  518.             fmax v15.4s, v15.4s, v9.4s
  519.             fmax v16.4s, v16.4s, v9.4s
  520.             fmin v15.4s, v15.4s, v10.4s
  521.             fmin v16.4s, v16.4s, v10.4s
  522.         StoreLH2x4:
  523.             shrn v15.4h, v15.4s, #16
  524.             shrn v16.4h, v16.4s, #16
  525.             st1 {v15.4h, v16.4h}, [x0] // 8 * sizeof(int16_t)
  526.     E2End:
  527.         sub x3, x3, #2
  528.         add x0, x21, #16 // move dest address of 2 * 4 * sizeof(int16_t)
  529.         add x1, x1, #16 // move dest address of 2 * 4 * sizeof(int16_t)
  530.  
  531. E1:
  532. cmp x3, #0
  533. beq End
  534.  
  535. LoopE1:
  536.     mov x20, x6
  537.     mov x8, x10
  538.     mov x21, x0
  539.     mov x13, x2
  540.  
  541.     cmp x8, #2
  542.     blt E1LH4
  543.  
  544.     E1LH8:
  545.     E1LoopH8:
  546.         mov x15, x1
  547.         mov x12, x9
  548.         cbz x5, NoBiasE1
  549.         ld1 {v0.4h, v1.4h}, [x20], #16
  550.         shll v0.4s, v0.4h, #16
  551.         shll v1.4s, v1.4h, #16
  552.         mov v2.16b, v0.16b
  553.         mov v3.16b, v1.16b
  554.         uzp1 v16.2d, v0.2d, v2.2d   // bias_0, bias_1, bias_0, bias_1
  555.         uzp2 v17.2d, v0.2d, v2.2d   // bias_2, bias_3, bias_2, bias_3
  556.         uzp1 v18.2d, v1.2d, v3.2d   // bias_0, bias_1, bias_0, bias_1
  557.         uzp2 v19.2d, v1.2d, v3.2d   // bias_2, bias_3, bias_2, bias_3
  558.         b E1LoopL
  559.         NoBiasE1:
  560.             SET_ZERO v16, v17, v18, v19
  561.         E1LoopL:
  562.             // A [1, 4, bf16] : rn = 1  : v4
  563.             // B [8, 4, bf16] : rn = 4  : v0 - v3
  564.             // C [1, 8, fp32] : rn = 4  : v16 - v19
  565.             ld1 {v4.4h}, [x15], x11                      // A: 1 * 4 * sizeof(int16_t)
  566.             ld1 {v0.8h, v1.8h, v2.8h, v3.8h}, [x13], x16 // B: 8 * 4 * sizeof(int16_t)
  567.             .inst 0x6e40ec90 // bfmmla v16.4s, v4.8h, v0.8h
  568.             .inst 0x6e41ec91 // bfmmla v17.4s, v4.8h, v1.8h
  569.             .inst 0x6e42ec92 // bfmmla v18.4s, v4.8h, v2.8h
  570.             .inst 0x6e43ec93 // bfmmla v19.4s, v4.8h, v3.8h
  571.             subs x12, x12, #1
  572.             bgt E1LoopL
  573.         E1LoopLEnd:
  574.             // v16-v19: [r0, r1, 0, 0]
  575.             uzp1 v15.2d, v16.2d, v17.2d
  576.             uzp1 v16.2d, v18.2d, v19.2d
  577.             cbz x5, StoreLH1x8
  578.         PostTreatLH1x8:
  579.             fmax v15.4s, v15.4s, v9.4s
  580.             fmax v16.4s, v16.4s, v9.4s
  581.             fmin v15.4s, v15.4s, v10.4s
  582.             fmin v16.4s, v16.4s, v10.4s
  583.         StoreLH1x8:
  584.             shrn v15.4h, v15.4s, #16
  585.             shrn v16.4h, v16.4s, #16
  586.             st1 {v15.4h}, [x0], x7
  587.             st1 {v16.4h}, [x0], x7
  588.             add x13, x13, x19
  589.             sub x8, x8, #2
  590.             cmp x8, #2
  591.             bge E1LoopH8
  592.  
  593.     E1LH4:
  594.     cbz x8, E1End
  595.     mov x15, x1
  596.     mov x12, x9
  597.     cbz x5, NoBiasE1R
  598.     ld1 {v0.4h}, [x20]
  599.     shll v0.4s, v0.4h, #16
  600.     mov v2.16b, v0.16b
  601.     uzp1 v16.2d, v0.2d, v2.2d   // bias_0, bias_1, bias_0, bias_1
  602.     uzp2 v17.2d, v0.2d, v2.2d   // bias_2, bias_3, bias_2, bias_3
  603.     b E1LoopLR
  604.     NoBiasE1R:
  605.         movi v16.4s, #0
  606.         movi v17.4s, #0
  607.     E1LoopLR:
  608.         // A [1, 4, bf16] : rn = 1  : v4
  609.         // B [4, 4, bf16] : rn = 2  : v0 - v1
  610.         // C [1, 8, fp32] : rn = 4  : v16 - v17
  611.         ld1 {v4.4h}, [x15], x11        // A: 1 * 4 * sizeof(int16_t)
  612.         ld1 {v0.8h, v1.8h}, [x13], x16 // B: 4 * 4 * sizeof(int16_t)
  613.         .inst 0x6e40ec90 // bfmmla v16.4s, v4.8h, v0.8h
  614.         .inst 0x6e41ec91 // bfmmla v17.4s, v4.8h, v1.8h
  615.         subs x12, x12, #1
  616.         bgt E1LoopLR
  617.     E1LoopLREnd:
  618.         uzp1 v15.2d, v16.2d, v17.2d
  619.         cbz x5, StoreLH1x4
  620.     PostTreatLH1x4:
  621.         fmax v15.4s, v15.4s, v9.4s
  622.         fmin v15.4s, v15.4s, v10.4s
  623.     StoreLH1x4:
  624.         shrn v15.4h, v15.4s, #16
  625.         st1 {v15.4h}, [x0]
  626.     E1End:
  627.         subs x3, x3, #1
  628.         add x0, x21, #8
  629.         add x1, x1, #8
  630.         bne LoopE1
  631. End:
  632. ldr x19, [sp, #0]
  633. ldr x20, [sp, #8]
  634. ldr x21, [sp, #16]
  635. add sp, sp, #32
  636.  
  637. ret
  638. #endif

        编译完成后得到了三个动态库：libMNN.so、libMNNOpenCV.so、libMNN_Express.so，这三个库后面会用到。

        在前面我们已经训练得到了权重文件v5Lite-e.pt，现在要把它先转成onnx，再从onnx格式转成mnn格式。

        YOLOv5-Lite作者已提供export.py程序，执行：

  1. python export.py --mnnd --weight weights/v5lite-e.pt
  2. python -m onnxsim weights/v5lite-e.onnx weights/v5lite-e-mnnd_sim.onnx

        得到onnx格式的权重文件v5lite-e-mnnd_sim.onnx，将它放到树莓派上，在MNN的目录下执行：

python ./tools/script/testMNNFromOnnx.py YOLOv5-Lite/v5Lite-e-mnnd_sim.onnx

        出现TEST_SUCCESS未报错，继续执行： 

./build/MNNConvert -f ONNX --modelFile YOLOv5-Lite/mnnd/v5lite-e-mnnd_sim.onnx --MNNModel YOLOv5-Lite/mnnd/v5lite-e-mnnd.mnn --optimizeLevel 1 --optimizePrefer 2 --bizCode MNN --saveStaticModel --testdir val_test

        转换完成，得到mnn格式的模型权重。

四、推理

        使用YOLOv5-Lite作者提供的检测代码，位置在项目目录下的./cpp_demo/mnn，在其中的lib目录内放入前面编译MNN框架得到的三个动态库，命令行执行下面命令即可编译得到在树莓派上进行检测的程序。

  1. mkdir build && cd build
  2. cmake ..
  3. make

        MNN框架在树莓派上的推理速度比较快，我使用的是8GB的树莓派4B，用MNN框架推理能有16帧左右，而使用onnxruntime只有5到6帧。

参考博文

https://zhuanlan.zhihu.com/p/672633849

香橙派--编译MNN报错，关于汇编的嵌套展开_mnn 编译报错-CSDN博客

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/喵喵爱编程/article/detail/819707
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号