当前位置:   article > 正文

ONNX Runtime使用简单介绍

onnx runtime

前面系列博客中有用tensorRT、OpenVINO加速模型推理

TensorRT加速方法介绍(python pytorch模型)_竹叶青lvye的博客-CSDN博客_tensorrt加速

OpenVINO使用介绍_竹叶青lvye的博客-CSDN博客_openvino resnet在

这边再简单提下ONNX Runtime的使用(微软推出),上面博客中只是将ONNX模型作为一个中间转换模型用,可能不怎么去接触更为原生态的推理框架ONNX Runtime

ONNX Runtime | Homehttps://onnxruntime.ai/从官网介绍看,其也是提供训练用API调用的,可以看到微软也是有一些想和tensorflow,pytorch在深度学习领域分一杯羹的想法的,大公司毕竟是有强有力的资本的,何愁招兵买马呢。

此时cuda、cuddn、python版本同前面博客时的配置

主要过程如下:

这边还是按照官方配置,去推断一张图片,pip安装下如下库

pip install onnxruntime-gpu

 博主这边还是拿tensorflow来示例

pip install tf2onnx

代码如下:

  1. import tensorflow as tf
  2. import numpy as np
  3. from tensorflow.keras.preprocessing import image
  4. from tensorflow.keras.applications import resnet50
  5. from tensorflow.keras.applications.resnet50 import preprocess_input, decode_predictions
  6. from PIL import Image
  7. import time
  8. import tf2onnx
  9. import onnxruntime as rt
  10. import cv2
  11. import time
  12. physical_devices = tf.config.list_physical_devices('GPU')
  13. tf.config.experimental.set_memory_growth(physical_devices[0], True)
  14. #加载预训练模型
  15. model = resnet50.ResNet50(weights='imagenet')
  16. spec = (tf.TensorSpec((None, 224, 224, 3), tf.float32, name="input"),)
  17. output_path = "test.onnx"
  18. model_proto, _ = tf2onnx.convert.from_keras(model, input_signature=spec, opset=13, output_path=output_path)
  19. output_names = [n.name for n in model_proto.graph.output]
  20. providers = ['CPUExecutionProvider']
  21. m = rt.InferenceSession(output_path, providers=providers)
  22. img = cv2.imread('2008_002682.jpg')
  23. img = cv2.resize(img,(224,224))
  24. img_np = np.array(img, dtype=np.float32) / 255.
  25. img_np = np.expand_dims(img_np, axis=0)
  26. print(img_np.shape)
  27. t_model = time.perf_counter()
  28. onnx_pred = m.run(output_names, {"input": img_np})
  29. print(f'do inference cost:{time.perf_counter() - t_model:.8f}s')
  30. print('ONNX Predicted:', decode_predictions(onnx_pred[0], top=3)[0])

 测试图片,还是前面常用的小猫图片

部分执行结果如下:

  1. 2022-05-01 22:41:51.165567: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:1144] Optimization results for grappler item: graph_to_optimize
  2. constant_folding: Graph size after: 556 nodes (-320), 891 edges (-320), time = 253.008ms.
  3. function_optimizer: function_optimizer did nothing. time = 0.565ms.
  4. constant_folding: Graph size after: 556 nodes (0), 891 edges (0), time = 111.51ms.
  5. function_optimizer: function_optimizer did nothing. time = 0.847ms.
  6. (1, 224, 224, 3)
  7. do inference cost:0.01735498s
  8. ONNX Predicted: [('n01930112', 'nematode', 0.13559905), ('n03041632', 'cleaver', 0.04139605), ('n03838899', 'oboe', 0.03445778)]
  9. Process finished with exit code 0

预测结果同OpenVINO下的结果,耗时更少。暂时不再深入了解了,官网上也提供了一些方面资料,后面需要时再用吧。

声明:本文内容由网友自发贡献,不代表【wpsshop博客】立场,版权归原作者所有,本站不承担相应法律责任。如您发现有侵权的内容,请联系我们。转载请注明出处:https://www.wpsshop.cn/w/Cpp五条/article/detail/492710
推荐阅读
相关标签
  

闽ICP备14008679号