当前位置: article > 正文

华为CANN训练营笔记——应用开发全流程 [3]（with 代码版）_getdevicebufferoffile

作者：小小林熬夜学编程 | 2024-05-04 10:14:45

踩

getdevicebufferoffile

1.5 模型加载与执行

1.5.1 模型推理三大流程

1. 模型加载流程

在这里插入图片描述

aclmdlLoadFromMemWithMem：从HOST侧内存加载权重到aclrtMalloc申请的Device内存上

2. 模型推理整体流程

在这里插入图片描述

主要讲解同步推理

3. 准备推理输入&输出数据

在这里插入图片描述

转换成om模型时，batch_size已固定。

1.5.2 模型推理资源准备相关接口

1. 自动管理模型

aclError aclmdlLoadFromFile(const char *modelPath, uint32_t *modelId)：从文件加载om模型，返回模型ID（确保om有权限访问）

aclError aclmdlLoadFromMem(const void *model, size_t modelSize, uint32 _t *modelId)：从内存加载om模型。此处输入参数model是模型的内存地址，需要申请运行模式的内存（HOST / Device）

2. 手动管理模型

模型在文件中
aclError aclmdlQuerySize(const char *fileName, size_t *workSize, size_t *weightSize)：返回权值内存大小和工作内存大小

aclError aclmdlLoadFromFileWithMem(const char *modelPath, uint32_t *modelId, void *workPtr, size_t workSize, void *weightPtr, size_t weightSize)

workPtr：Device上模型工作所需内存，需要调用aclrtMalloc由用户自行申请，管理。
weightPtr：Device上模型权值所需内存

模型在内存中
aclError aclmdlQuerySizeFromMem(const void *model, size_t modelSize, size_t *workSize, size_t *weightSize)

aclError aclmdlLoadFromMemWithMem(const void *model, size_t modelSize, uint32_t *modelId, void *workPtr, size_t workSize, void *weightPtr, size_t weightSize)：同样两个Ptr需要申请

3. 卸载

aclError aclmdlUnload(uint32_t modelId)

4. 模型描述

获取模型描述信息
aclmdlDesc* aclmdlCreateDesc()：创建描述信息对象；用aclError aclmdlDestroyDesc(aclmdlDesc *modelDesc)销毁

aclError aclmdlGetDesc(aclmdlDesc* modelDesc, uint32_t modelId)：根据模型ID获取模型描述信息

模型描述信息

5. 输入输出数据

创建数据结构
aclmdlDataset* aclmdlCreateDataset()；用aclError aclmdlDestroyDataset(const aclmdlDataset *dataset)销毁
向Dataset添加DataBuffer
aclError aclmdlAddDatasetBuffer(aclmdlDataset* dataset, aclDataBuffer * dataBuffer)：向aclmdlDataset中增加DataBuffer
- dataset：待增加Buffer的Dataset地址
- dataBuffer：输入，databuffer地址
获取dataset中buffer数
size_t aclmdlGetDatasetNumBuffer(const aclmdlDataset* dataset)：常用于输出数据的获取阶段
获取aclmdlDataset中第index个DataBuffer
aclDataBuffer* aclmdlGetDatasetBuffer(const aclmdlDataset *dataset, size_t index)：常用于输出数据的获取阶段
创建DataBuffer
aclDataBuffer *aclCreateDataBuffer(void *data, size_t size)：data是由aclrtMalloc申请的device上内存。aclError aclDestroyDataBuffer(const alcDataBuffer *dataBuffer)仅销毁DataBuffer，data内存没有被释放
获取DataBuffer内容
void *aclGetDataBufferAddr(const acIDataBuffer *dataBuffer)：返回数据的内存地址
size_t aclGetDataBufferSize(const acIDataBuffer *dataBuffer)：返回数据的内存大小

1.5.3 模型推理执行接口

同步
aclError aclmdlExecute(uint32_t modelId, const aclmdlDataset *input, aclmdlDataset *output)
- modelId: 模型Id
- input: 输入Dataset
- output: 输出Dataset
异步
aclError aclmdlExecuteAsync(uint32_t modelId, const aclmdlDataset *input, aclmdlDataset *output, aclrtStream stream)

1.5.4 Demo！！！

1. 目录结构

- caffe_model
	- .prototxt  模型结构文件
- data
	- 1.jpg
	- 2.jpg
- inc
	- model_process.h
	- sample_process.h
	- utils.h
- src
	- acl.json
	- CMakeLists.txt
	- main.cpp		主函数
	- model_process.cpp
	- sample_process.cpp
	- utils.cpp
- script
	- transferPic.py
- .project
- CMakeLists.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

2. utils.h

#pragma once
#include <iostream>

#define INFO_LOG(fmt, args...) fprintf(stdout, "[INFO]  " fmt "\n", ##args)
#define WARN_LOG(fmt, args...) fprintf(stdout, "[WARN]  " fmt "\n", ##args)
#define ERROR_LOG(fmt, args...) fprintf(stdout, "[ERROR] " fmt "\n", ##args)

typedef enum Result {
    SUCCESS = 0,
    FAILED = 1
} Result;

/**
* Utils
*/
class Utils {
public:
    /**
    * @brief create device buffer of file
    * @param [in] fileName: file name
    * @param [out] fileSize: size of file
    * @return device buffer of file
    */
    static void *GetDeviceBufferOfFile(std::string fileName, uint32_t &fileSize);

    /**
    * @brief create buffer of file
    * @param [in] fileName: file name
    * @param [out] fileSize: size of file
    * @return buffer of pic
    */
    static void* ReadBinFile(std::string fileName, uint32_t& fileSize);
};

#pragma once

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

3. sample_process.h

#pragma once
#include "utils.h"
#include "acl/acl.h"

/**
* SampleProcess
*/
class SampleProcess {
public:
    /**
    * @brief Constructor
    */
    SampleProcess();

    /**
    * @brief Destructor
    */
    ~SampleProcess();

    /**
    * @brief init reousce
    * @return result
    */
    Result InitResource();

    /**
    * @brief sample process
    * @return result
    */
    Result Process();

private:
    void DestroyResource();

    int32_t deviceId_;
    aclrtContext context_;
    aclrtStream stream_;
};

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

4. model_process.h

#pragma once
#include <iostream>
#include "utils.h"
#include "acl/acl.h"

/**
* ModelProcess
*/
class ModelProcess {
public:
    /**
    * @brief Constructor
    */
    ModelProcess();

    /**
    * @brief Destructor
    */
    ~ModelProcess();

    /**
    * @brief load model from file with mem
    * @param [in] modelPath: model path
    * @return result
    */
    Result LoadModelFromFileWithMem(const char *modelPath);

    /**
    * @brief unload model
    */
    void Unload();

    /**
    * @brief create model desc
    * @return result
    */
    Result CreateDesc();

    /**
    * @brief destroy desc
    */
    void DestroyDesc();

    /**
    * @brief create model input
    * @param [in] inputDataBuffer: input buffer
    * @param [in] bufferSize: input buffer size
    * @return result
    */
    Result CreateInput(void *inputDataBuffer, size_t bufferSize);

    /**
    * @brief destroy input resource
    */
    void DestroyInput();

    /**
    * @brief create output buffer
    * @return result
    */
    Result CreateOutput();

    /**
    * @brief destroy output resource
    */
    void DestroyOutput();

    /**
    * @brief model execute
    * @return result
    */
    Result Execute();

    /**
    * @brief get model output result
    */
    void OutputModelResult();

   	// void DumpModelOutputResult();

private:
    uint32_t modelId_;
    size_t modelMemSize_;
    size_t modelWeightSize_;
    void *modelMemPtr_;
    void *modelWeightPtr_;
    bool loadFlag_;  // model load flag
    aclmdlDesc *modelDesc_;
    aclmdlDataset *input_;
    aclmdlDataset *output_;
};
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91

5. transferPic.py

import numpy as np
import os
from PIL import Image

def process(input_path):
    try:
        input_image = Image.open(input_path)
        input_image = input_image.resize((256, 256))
        # hwc
        img = np.array(input_image)
        height = img.shape[0]
        width = img.shape[1]
        h_off = int((height-224)/2)
        w_off = int((width-224)/2)
        crop_img = img[h_off:height-h_off, w_off:width-w_off, :]
        # rgb to bgr
        img = crop_img[:, :, ::-1]
        shape = img.shape
        img = img.astype("float16")
        img[:, :, 0] -= 104
        img[:, :, 1] -= 117
        img[:, :, 2] -= 123
        img = img.reshape([1] + list(shape))
        result = img.transpose([0, 3, 1, 2])
        output_name = input_path.split('.')[0] + ".bin"
        result.tofile(output_name)
    except Exception as except_err:
        print(except_err)
        return 1
    else:
        return 0
if __name__ == "__main__":
    count_ok = 0
    count_ng = 0
    images = os.listdir(r'./')
    for image_name in images:
        if not image_name.endswith("jpg"):
            continue
        print("start to process image {}....".format(image_name))
        ret = process(image_name)
        if ret == 0:
            print("process image {} successfully".format(image_name))
            count_ok = count_ok + 1
        elif ret == 1:
            print("failed to process image {}".format(image_name))
            count_ng = count_ng + 1
    print("{} images in total, {} images process successfully, {} images process failed"
          .format(count_ok + count_ng, count_ok, count_ng))
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

6. main.cpp

# include <iostream>
# include "sample_process.h"
# include "utils.h"
using namespace std;
bool g_isDevice = false; // 全局变量赋初值

int main(){
	SampleProcess processSample;
	Result ret = processSample.InitResource(); // 运行资源初始化
	if (ret != SUCCESS){
		ERROR_LOG("sample init resource failed");
		return FAILED;
	}
	
	ret = processSample.Process();
	if (ret != SUCCESS){
		ERROR_LOG("sample process failed");
		return FAILED;
	}
	INFO_LOG("execute sample success");
	return SUCCESS;
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

7. sample_process.cpp

# include <iostream>
# include "sample_process.h"
# include "utils.h"
# include "model_process.h"
# include "acl/acl.h"
using namespace std;
extern bool g_isDevice;

// 构造方法，为三个成员变量赋初值
SampleProcess::SampleProcess() :deviceId_(0), context_(nullptr), stream_(nullptr) 
{
}

// 析构
SampleProcess::~SampleProcess(){
	DestroyResource();
}

// 初始化函数
Result SampleProcess::InitResource(){
	// ACL初始化
	const char *aclConfigPath = "../src/acl.json";
	aclError ret = aclInit(aclConfigPath);
	if (ret != ACL_ERROR_NONE){	// 判断初始化成功
		ERROR_LOG("acl init failed");
	}
	INFO_LOG("acl init success");

	// open device
	ret = aclrtSetDevice(deviceId_);
	if (ret != ACL_ERROR_NONE){
		ERROR_LOG("acl open device %d failed", deviceId_);
		return FAILED;
	}
	INFO_LOG("acl open device %d success", deviceId_);

	// create context
	ret = aclrtCreateContext(&context_, deviceId_);
	if (ret != ACL_ERROR_NONE){
		ERROR_LOG("acl create context failed");
		return FAILED;
	}
	INFO_LOG("acl create contxt success");

	// create stream
	ret = aclrtCreateStream(&stream_);
	if (ret != ACL_ERROR_NONE){
		ERROR_LOG("acl create stream failed");
		return FAILED;
	}
	INFO_LOG("acl create stream success");
	
	// get run mode
	aclrtRunMode runMode;
	ret = aclrtGetRunMode(&runMode);
	if (ret != ACL_ERROR_NONE){
		ERROR_LOG("acl get run mode failed");
		return FAILED;
	}
	g_isDevice = (runMode == ACL_DEVICE);
	INFO_LOG("get run mode success");
	return SUCCESS;
}


Result SampleProcess::Process(){
	// 创建一个modelProcess对象, 这个类构造于model_process.cpp
	ModelProcess processModel;
	const char* omModelPath = "../model/resnet50.om";	// 模型文件路径
	// 手动管理内存, 从文件加载模型
	Result ret = processModel.LoadModelFromFileWithMem(omModelPath);
	if (ret != SUCCESS){
		ERROR_LOG("excute LoadModelFromFileWithMem failed");
		return FAILED;
	}
	// 创建模型描述信息
	ret = processModel.CreateDesc();
    if (ret != SUCCESS) {
        ERROR_LOG("execute CreateModelDesc failed");
        return FAILED;
    }
	// 对于一个模型, 他的输出应该是固定的
	ret = processModel.CreateOutput();
    if (ret != SUCCESS) {
        ERROR_LOG("execute CreateOutput failed");
        return FAILED;
    }
	
	// =================
	// 		图片数据
	// =================
	string testFile[] = {
        "../data/dog1_1024_683.bin",
        "../data/dog2_1024_683.bin"
    };

	for (size_t index = 0; index < sizeof(testFile) / sizeof(testFile[0]); ++index) {
		INFO_LOG("start to process file:%s", testFile[index].c_str());
		
		// 处理模型
		// 不同于resnet50_imageclassification工程; 这里将获取输入放在了循环里
		uint32_t devBufferSize;
		// 申请这个文件在device上的内存 并返回内存地址
		void *picDevBuffer = Utils::GetDeviceBufferOfFile(testFile[index], devBufferSize);
		if (picDevBuffer == nullptr){
			ERROR_LOG("get pic device buffer failed, index is %zu", index);
			return FAILED;
		}

		// 图片数据内存地址放入input
		ret = processModel.CreateInput(picDevBuffer, devBufferSize);
		if (ret != SUCCESS){
			ERROR_LOG("get pic device buffer failed, index is %zu", index);
			aclrtFree(picDevBuffer); // 失败及时销毁
			return FAILED;
		}
		
		// 推理；结果保存在processModel的成员变量中
		ret = processModel.Execute();
		if (ret != SUCCESS){
			ERROR_LOG("execute inference failed");
			aclrtFree(picDevBuffer);
			return FAILED;
		}
	
		processModel.OutputModelResult();
		aclrtFree(picDevBuffer);
		processModel.DestroyInput();
	}
	return SUCCESS;
}

void SampleProcess::DestroyResource()
{	
	// 销毁顺序：stream -> context -> device -> finalize
    aclError ret;
    if (stream_ != nullptr) {
        ret = aclrtDestroyStream(stream_);
        if (ret != ACL_ERROR_NONE) {
            ERROR_LOG("destroy stream failed");
        }
        stream_ = nullptr;
    }
    INFO_LOG("end to destroy stream");

    if (context_ != nullptr) {
        ret = aclrtDestroyContext(context_);
        if (ret != ACL_ERROR_NONE) {
            ERROR_LOG("destroy context failed");
        }
        context_ = nullptr;
    }
    INFO_LOG("end to destroy context");

    ret = aclrtResetDevice(deviceId_);
    if (ret != ACL_ERROR_NONE) {
        ERROR_LOG("reset device failed");
    }
    INFO_LOG("end to reset device %d", deviceId_);

    ret = aclFinalize();
    if (ret != ACL_ERROR_NONE) {
        ERROR_LOG("finalize acl failed");
    }
    INFO_LOG("end to finalize acl");
}


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168

8. model_process.cpp

手动管理内存

# include <iostream>
# include "model_process.h"
# include <map>
# include <sstream>
# include <algorithm>
# include "utils.h"
using namespace std;
extern bool g_isDevice;

// 构造函数 loadFlag_ 1表示模型已加载
ModelProcess::ModelProcess() :modelId_(0), modelMemSize_(0), modelWeightSize_(0), modelMemPtr_(nullptr), modelWeightPtr_(nullptr), loadFlag_(false), modelDesc_(nullptr), input_(nullptr), output_(nullptr)
{
}

ModelProcess :: ~ModelProcess(){
	Unload(); // 卸载模型
	DestroyDesc();
	DestroyInput();
	DestroyOutput();
}

// 从文件加载模型文件至手动管理内存
Result ModelProcess::LoadModelFromFileWithMem(const char* modelPath){
	if (loadFlag_){
		ERROR_LOG("has already loaded a model");
		return FAILED;
	}
	
	// 从文件中加载模型内存大小与权重大小
	aclError ret = aclmdlQuerySize(modelPath, &modelMemSize_, &modelWeightSize_);
	if (ret != ACL_ERROR_NONE){
		ERROR_LOG("query mem of model file %s failed", modelPath);
		return FAILED;
	}
	
	// 根据查询的模型内存大小分配空间
	ret = aclrtMalloc(&modelMemPtr_, modelMemSize_, ACL_MEM_MALLOC_HUGE_FIRST);
	if (ret != ACL_ERROR_NONE){
		ERROR_LOG("malloc buffer for mem failed, required size is %zu", modelMemSize_);
		return FAILED;
	}

	// 根据查询的模型参数内存大小分配空间
	ret = aclrtMalloc(&modelWeightPtr_, modelWeightSize_, ACL_MEM_MALLOC_HUGE_FIRST);
	if (ret != ACL_ERROR_NONE){
		ERROR_LOG("malloc buffer for weight failed, required size is %zu", modelMemSize_);
		return FAILED;
	}
	
	// 调用acl接口，从文件加载模型
	ret = aclmdlLoadFromFileWithMem(modelPath, &modelId_, modelMemPtr_, modelMemSize_, modelWeightPtr_, modelWeightSize_);
	if (ret != ACL_ERROR_NONE){
		ERROR_LOG("load model from file %s failed", modelPath );
		return FAILED;
	}
	
	// 加载模型结束
	loadFlag_ = true;
	INFO_LOG("load model %s success", modelPath);
	return SUCCESS;
}

// 创建模型描述信息
Result ModelProcess::CreateDesc()
{
    modelDesc_ = aclmdlCreateDesc();
    if (modelDesc_ == nullptr) {
        ERROR_LOG("create model description failed");
        return FAILED;
    }

    aclError ret = aclmdlGetDesc(modelDesc_, modelId_);
    if (ret != ACL_SUCCESS) {
        ERROR_LOG("get model description failed, modelId is %u", modelId_);
        return FAILED;
    }

    INFO_LOG("create model description success");
    return SUCCESS;
}

// 有了模型描述, 然后才可以创建输出
Result ModelProcess::CreateOutput()
{
    if (modelDesc_ == nullptr) {
        ERROR_LOG("no model description, create ouput failed");
        return FAILED;
    }

    output_ = aclmdlCreateDataset();
    if (output_ == nullptr) {
        ERROR_LOG("can't create dataset, create output failed");
        return FAILED;
    }

	// 从模型描述中获取输出个数
    size_t outputSize = aclmdlGetNumOutputs(modelDesc_);
    for (size_t i = 0; i < outputSize; ++i) {
    	// 获取第i个输出的大小
        size_t buffer_size = aclmdlGetOutputSizeByIndex(modelDesc_, i);

        void *outputBuffer = nullptr;
        aclError ret = aclrtMalloc(&outputBuffer, buffer_size, ACL_MEM_MALLOC_NORMAL_ONLY);
        if (ret != ACL_ERROR_NONE) {
            ERROR_LOG("can't malloc buffer, size is %zu, create output failed", buffer_size);
            return FAILED;
        }
        
		// 创建databuffer
        aclDataBuffer *outputData = aclCreateDataBuffer(outputBuffer, buffer_size);
        if (outputData == nullptr) {
            ERROR_LOG("can't create data buffer, create output failed");
            (void)aclrtFree(outputBuffer);
            return FAILED;
        }
		
		// 将databuffer放入output dataset
        ret = aclmdlAddDatasetBuffer(output_, outputData);
        if (ret != ACL_ERROR_NONE) {
            ERROR_LOG("can't add data buffer, create output failed");
            // 如果失败，释放内存
            (void)aclrtFree(outputBuffer);            
            (void)aclDestroyDataBuffer(outputData);
            return FAILED;
        }
    }

    INFO_LOG("create model output success");

    return SUCCESS;
}

Result ModelProcess::CreateInput(void *inputDataBuffer, size_t bufferSize)
{
    input_ = aclmdlCreateDataset();
    if (input_ == nullptr) {
        ERROR_LOG("can't create dataset, create input failed");
        return FAILED;
    }
	// 创建databuffer
    aclDataBuffer *inputData = aclCreateDataBuffer(inputDataBuffer, bufferSize);
    if (inputData == nullptr) {
        ERROR_LOG("can't create data buffer, create input failed");
        return FAILED;
    }

    aclError ret = aclmdlAddDatasetBuffer(input_, inputData);
    if (ret != ACL_SUCCESS) {
        ERROR_LOG("add input dataset buffer failed");
        (void)aclDestroyDataBuffer(inputData);
        inputData = nullptr;
        return FAILED;
    }
    INFO_LOG("create model input success");

    return SUCCESS;
}

Result ModelProcess::Execute()
{
	// 推理，结果放入output中
    aclError ret = aclmdlExecute(modelId_, input_, output_);
    if (ret != ACL_ERROR_NONE) {
        ERROR_LOG("execute model failed, modelId is %u", modelId_);
        return FAILED;
    }

    INFO_LOG("model execute success");
    return SUCCESS;
}

void ModelProcess::OutputModelResult()
{
    for (size_t i = 0; i < aclmdlGetDatasetNumBuffers(output_); ++i) {
        // 遍历output dataset里的data buffer
        aclDataBuffer* dataBuffer = aclmdlGetDatasetBuffer(output_, i);
        void* data = aclGetDataBufferAddr(dataBuffer);
        // uint32_t len = aclGetDataBufferSizeV2(dataBuffer);
        uint32_t len = aclGetDataBufferSizeV2(dataBuffer);

        void *outHostData = nullptr;
        aclError ret = ACL_ERROR_NONE;
        float *outData = nullptr;
        if (!g_isDevice) { // 如果正在运行在HOST上，就需要把数据从Device上取回到HOST
            aclError ret = aclrtMallocHost(&outHostData, len);
            if (ret != ACL_ERROR_NONE) {
                ERROR_LOG("aclrtMallocHost failed, ret[%d]", ret);
                return;
            }

            // Device -> HOST
            ret = aclrtMemcpy(outHostData, len, data, len, ACL_MEMCPY_DEVICE_TO_HOST);
            if (ret != ACL_ERROR_NONE) {
                ERROR_LOG("aclrtMemcpy failed, ret[%d]", ret);
                (void)aclrtFreeHost(outHostData);
                return;
            }

            outData = reinterpret_cast<float*>(outHostData);
        } else {
            outData = reinterpret_cast<float*>(data);
        }

		// 打印
        map<float, unsigned int, greater<float> > resultMap;
        for (unsigned int j = 0; j < len / sizeof(float); ++j) {
            resultMap[*outData] = j;
            outData++;
        }

        int cnt = 0;
        for (auto it = resultMap.begin(); it != resultMap.end(); ++it) {
            // print top 5
            if (++cnt > 5) {
                break;
            }

            INFO_LOG("top %d: index[%d] value[%lf]", cnt, it->second, it->first);
        }
        if (!g_isDevice) {
            ret = aclrtFreeHost(outHostData);
            if (ret != ACL_ERROR_NONE) {
                ERROR_LOG("aclrtFreeHost failed, ret[%d]", ret);
                return;
            }
        }
    }

    INFO_LOG("output data success");
    return;
}

void ModelProcess::DestroyInput()
{
    if (input_ == nullptr) {
        return;
    }

    for (size_t i = 0; i < aclmdlGetDatasetNumBuffers(input_); ++i) {
    	// 销毁所有的buffer
        aclDataBuffer *dataBuffer = aclmdlGetDatasetBuffer(input_, i);
        (void)aclDestroyDataBuffer(dataBuffer);
    }
    (void)aclmdlDestroyDataset(input_);
    input_ = nullptr;
    INFO_LOG("destroy model input success");
}

void ModelProcess::Unload()
{
    if (!loadFlag_) {
        WARN_LOG("no model had been loaded, unload failed");
        return;
    }

    aclError ret = aclmdlUnload(modelId_);
    if (ret != ACL_ERROR_NONE) {
        ERROR_LOG("unload model failed, modelId is %u", modelId_);
    }
	
	// 销毁模型描述信息
    if (modelDesc_ != nullptr) {
        (void)aclmdlDestroyDesc(modelDesc_);
        modelDesc_ = nullptr;
    }

    if (modelMemPtr_ != nullptr) {
        (void)aclrtFree(modelMemPtr_);
        modelMemPtr_ = nullptr;
        modelMemSize_ = 0;
    }

    if (modelWeightPtr_ != nullptr) {
        (void)aclrtFree(modelWeightPtr_);
        modelWeightPtr_ = nullptr;
        modelWeightSize_ = 0;
    }

    loadFlag_ = false;
    INFO_LOG("unload model success, modelId is %u", modelId_);
}

void ModelProcess::DestroyDesc()
{
    if (modelDesc_ != nullptr) {
        (void)aclmdlDestroyDesc(modelDesc_);
        modelDesc_ = nullptr;
    }
    INFO_LOG("destroy model description success");
}

void ModelProcess::DestroyOutput()
{
    if (output_ == nullptr) {
        return;
    }

    for (size_t i = 0; i < aclmdlGetDatasetNumBuffers(output_); ++i) {
        aclDataBuffer* dataBuffer = aclmdlGetDatasetBuffer(output_, i);
        void* data = aclGetDataBufferAddr(dataBuffer);
        (void)aclrtFree(data);
        (void)aclDestroyDataBuffer(dataBuffer);
    }

    (void)aclmdlDestroyDataset(output_);
    output_ = nullptr;
    INFO_LOG("destroy model output success");
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308

9. utils.cpp

#include "utils.h"
#include <iostream>
#include <fstream>
#include <cstring>
#include "acl/acl.h"
#include <sys/stat.h>

extern bool g_isDevice;

void* Utils::ReadBinFile(std::string fileName, uint32_t &fileSize)
{
    struct stat sBuf;
    int fileStatus = stat(fileName.data(), &sBuf);
    if (fileStatus == -1) {
        ERROR_LOG("failed to get file");
        return nullptr;
    }
    if (S_ISREG(sBuf.st_mode) == 0) {
        ERROR_LOG("%s is not a file, please enter a file", fileName.c_str());
        return nullptr;
    }

    std::ifstream binFile(fileName, std::ifstream::binary);
    if (binFile.is_open() == false) {
        ERROR_LOG("open file %s failed", fileName.c_str());
        return nullptr;
    }

    binFile.seekg(0, binFile.end);
    uint32_t binFileBufferLen = binFile.tellg();
    if (binFileBufferLen == 0) {
        ERROR_LOG("binfile is empty, filename is %s", fileName.c_str());
        binFile.close();
        return nullptr;
    }

    binFile.seekg(0, binFile.beg);

    void* binFileBufferData = nullptr;
    aclError ret = ACL_ERROR_NONE;
    if (!g_isDevice) {
        ret = aclrtMallocHost(&binFileBufferData, binFileBufferLen);
        if (binFileBufferData == nullptr) {
            ERROR_LOG("malloc binFileBufferData failed");
            binFile.close();
            return nullptr;
        }
    } else {
        ret = aclrtMalloc(&binFileBufferData, binFileBufferLen, ACL_MEM_MALLOC_NORMAL_ONLY);
        if (ret != ACL_ERROR_NONE) {
            ERROR_LOG("malloc device buffer failed. size is %u", binFileBufferLen);
            binFile.close();
            return nullptr;
        }
    }
    binFile.read(static_cast<char *>(binFileBufferData), binFileBufferLen);
    binFile.close();
    fileSize = binFileBufferLen;
    return binFileBufferData;
}

void* Utils::GetDeviceBufferOfFile(std::string fileName, uint32_t &fileSize)
{
	// 根据文件名及文件大小 返回模型内存地址
    uint32_t inputHostBuffSize = 0;
    void* inputHostBuff = Utils::ReadBinFile(fileName, inputHostBuffSize);
    if (inputHostBuff == nullptr) {
        return nullptr;
    }
    if (!g_isDevice) {
        void *inBufferDev = nullptr;
        uint32_t inBufferSize = inputHostBuffSize;
        aclError ret = aclrtMalloc(&inBufferDev, inBufferSize, ACL_MEM_MALLOC_NORMAL_ONLY);
        if (ret != ACL_ERROR_NONE) {
            ERROR_LOG("malloc device buffer failed. size is %u", inBufferSize);
            aclrtFreeHost(inputHostBuff);
            return nullptr;
        }

        ret = aclrtMemcpy(inBufferDev, inBufferSize, inputHostBuff, inputHostBuffSize, ACL_MEMCPY_HOST_TO_DEVICE);
        if (ret != ACL_ERROR_NONE) {
            ERROR_LOG("memcpy failed. device buffer size is %u, input host buffer size is %u",
                inBufferSize, inputHostBuffSize);
            aclrtFree(inBufferDev);
            aclrtFreeHost(inputHostBuff);
            return nullptr;
        }
        aclrtFreeHost(inputHostBuff);
        fileSize = inBufferSize;
        return inBufferDev;
    } else {
        fileSize = inputHostBuffSize;
        return inputHostBuff;
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96

1.5.5 编译&运行

下载模型文件.prototxt和.caffemodel
设置环境. /usr/local/Ascend/ascend-toolkit/set_env.sh
转换模型atc --model caffe_model/resnet50.prototxt --weight caffe_model/resnet50.caffemodel --framework 0 --output model/resnet50 --soc_version Ascend310 --input_format NCHW --input_fp16_nodes data --output_type FP32 --out_nodes prob:0
转换图片格式python3 ../scripts/transferPic.py

转到根目录下mkdir -p build/intermediates/host
编译cmake ../../../src/ -DCMAKE_CXX_COMPILER=g++ -DCMAKE_SKIP_RPATH=TRUE
make

转到生成的out目录下 ./main

result

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/小小林熬夜学编程/article/detail/533992

华为CANN训练营笔记——应用开发全流程 [3]（with 代码版）_getdevicebufferoffile

1.5 模型加载与执行

1.5.1 模型推理三大流程

1. 模型加载流程

2. 模型推理整体流程

3. 准备推理输入&输出数据

1.5.2 模型推理资源准备相关接口

1. 自动管理模型

2. 手动管理模型

3. 卸载

4. 模型描述

5. 输入 输出数据

1.5.3 模型推理执行接口

1.5.4 Demo！！！

1. 目录结构

2. utils.h

3. sample_process.h

4. model_process.h

5. transferPic.py

6. main.cpp

7. sample_process.cpp

8. model_process.cpp

9. utils.cpp

1.5.5 编译&运行

5. 输入输出数据