【已解决】CLIP 的 textencoder 部分 .pt 转化 onnx 模型没有输入节点 [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid_onnx model has no [input] node

遇到该类问题，可以通过以下方法来确认是否与本文的问题一样：

1. 看模型定义的输入（1）通过 onnx_model.graph.input 来看，（2）通过 ONNX 可视化工具 Netron 来看，均没有 input
2. 通过 onnxruntime 来加载模型用于推理，出现报错 [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid

解决方案：修改 export 时的输入为 random ，而非确定的值，否则会被视为常量、被 onnx 折叠起来。

1.看模型定义的输入

（1）通过 onnx 的 .graph.input 可以找到转化时定义的模型输入，示例代码如下

import onnx
import onnxruntime as ort

# 加载 ONNX 模型
onnx_model_path = "/path/to/clip_txt1.onnx"
onnx_model = onnx.load(onnx_model_path)

# 打印模型输入定义列表
print("ONNX 模型的输入定义:")
for input in onnx_model.graph.input:
    print(f"Name: {input.name}")
    print(f"Type: {input.type}")
    print(f"Shape: {[dim.dim_value for dim in input.type.tensor_type.shape.dim]}")

# 使用 onnxruntime 加载模型并打印输入定义
try:
    ort_session = ort.InferenceSession(onnx_model_path)
    print("\nONNX Runtime 模型的输入定义:")
    for input in ort_session.get_inputs():
        print(f"Name: {input.name}")
        print(f"Type: {input.type}")
        print(f"Shape: {input.shape}")
except Exception as e:
    print(f"Error loading model with onnxruntime: {e}")

# 验证模型以确保其正确性
try:
    onnx.checker.check_model(onnx_model)
    print("The model is valid.")
except onnx.checker.ValidationError as e:
    print(f"The model is invalid: {e}")

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

如果没有输入，可能会得到以下结果

ONNX 模型的输入定义:
2024-05-23 09:51:04.537391508 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/Add_79'
2024-05-23 09:51:04.715214096 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.0/attn/Sqrt'
2024-05-23 09:51:04.715499051 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.1/attn/Sqrt'
2024-05-23 09:51:04.715710237 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.2/attn/Sqrt'
2024-05-23 09:51:04.715895645 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.3/attn/Sqrt'
2024-05-23 09:51:04.716076274 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.4/attn/Sqrt'
2024-05-23 09:51:04.716256863 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.5/attn/Sqrt'
2024-05-23 09:51:04.716439756 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.6/attn/Sqrt'
2024-05-23 09:51:04.716620936 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.7/attn/Sqrt'
2024-05-23 09:51:04.716800192 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.8/attn/Sqrt'
2024-05-23 09:51:04.716978226 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.9/attn/Sqrt'
2024-05-23 09:51:04.717158474 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.10/attn/Sqrt'
2024-05-23 09:51:04.717340426 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.11/attn/Sqrt'
2024-05-23 09:51:04.737839929 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.11/attn/Sqrt'
2024-05-23 09:51:04.737862642 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.10/attn/Sqrt'
2024-05-23 09:51:04.737872220 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.9/attn/Sqrt'
2024-05-23 09:51:04.737881878 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.8/attn/Sqrt'
2024-05-23 09:51:04.737890714 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.7/attn/Sqrt'
2024-05-23 09:51:04.737899811 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.6/attn/Sqrt'
2024-05-23 09:51:04.737908648 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.5/attn/Sqrt'
2024-05-23 09:51:04.737917344 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.4/attn/Sqrt'
2024-05-23 09:51:04.737926081 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.3/attn/Sqrt'
2024-05-23 09:51:04.737934697 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.2/attn/Sqrt'
2024-05-23 09:51:04.737943253 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.1/attn/Sqrt'
2024-05-23 09:51:04.737951839 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.0/attn/Sqrt'
2024-05-23 09:51:04.738515476 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/Add_79'

ONNX Runtime 模型的输入定义:
The model is valid.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

尽管通过了 onnx.checker.check_model 但仍然有问题，即输入不存在，（2）通过 ONNX 可视化工具 Netron 查看时，最上方的是 Add 而非 Input

或者可以看到 Model Properties 中无输入部分（只存在 Outputs），如下图

2.通过 onnxruntime 加载模型并推理

通过 onnxruntime 来推理模型的代码示例如下，也可参考官方示例

import onnxruntime
from PIL import Image
from typing import AnyStr
import numpy as np
import torch
import torch.nn as nn
from typing import Union, List, Tuple
from functools import partial
from torchvision import transforms
import os
import sys
from pytorch_svgrender.painter.clipfont import (imagenet_templates, compose_text_with_templates, Painter,
                                                PainterOptimizer)
import clip

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

# # 第一个输入以及预处理
prompt = 'Green Cthulhu'
template_text = compose_text_with_templates(prompt, imagenet_templates)
tokenize_fn = partial(clip.tokenize, context_length=77)
tokens1 = tokenize_fn(template_text).numpy()

# runtime 实例化
ort_session = onnxruntime.InferenceSession('/path/to/clip_txt1.onnx', providers=["CUDAExecutionProvider"])
# onnxruntime.InferenceSession 用于获取一个 ONNX Runtime 推理器，其参数是用于推理的 ONNX 模型文件。

# 第一个 runtime 输入
ort_inputs = {'input': tokens1}

# 第一次 runtime 推理
text_features = ort_session.run(['output'], ort_inputs)[0]

# 第二个输入以及预处理
source = "A photo"
template_source = compose_text_with_templates(source, imagenet_templates)
tokens2 = tokenize_fn(template_text).numpy()   

# 第二个 runtime 输入         
ort_inputs = {'input': tokens2}

# 第二次 runtime 推理            
text_source = ort_session.run(['output'], ort_inputs)[0]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43

出现以下报错

/path/miniconda3/envs/svgrender/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:69: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
  warnings.warn(
2024-05-23 10:03:56.768496802 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/Add_79'
2024-05-23 10:03:56.943615158 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.0/attn/Sqrt'
2024-05-23 10:03:56.943943475 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.1/attn/Sqrt'
2024-05-23 10:03:56.944162473 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.2/attn/Sqrt'
2024-05-23 10:03:56.944398975 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.3/attn/Sqrt'
2024-05-23 10:03:56.944606811 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.4/attn/Sqrt'
2024-05-23 10:03:56.944810019 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.5/attn/Sqrt'
2024-05-23 10:03:56.945021934 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.6/attn/Sqrt'
2024-05-23 10:03:56.945225472 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.7/attn/Sqrt'
2024-05-23 10:03:56.945441505 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.8/attn/Sqrt'
2024-05-23 10:03:56.945642549 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.9/attn/Sqrt'
2024-05-23 10:03:56.945846608 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.10/attn/Sqrt'
2024-05-23 10:03:56.946049756 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.11/attn/Sqrt'
2024-05-23 10:03:56.962879497 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.11/attn/Sqrt'
2024-05-23 10:03:56.962898172 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.10/attn/Sqrt'
2024-05-23 10:03:56.962914894 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.9/attn/Sqrt'
2024-05-23 10:03:56.962932037 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.8/attn/Sqrt'
2024-05-23 10:03:56.962950051 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.7/attn/Sqrt'
2024-05-23 10:03:56.962967385 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.6/attn/Sqrt'
2024-05-23 10:03:56.962985429 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.5/attn/Sqrt'
2024-05-23 10:03:56.963002401 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.4/attn/Sqrt'
2024-05-23 10:03:56.963019965 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.3/attn/Sqrt'
2024-05-23 10:03:56.963037498 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.2/attn/Sqrt'
2024-05-23 10:03:56.963054170 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.1/attn/Sqrt'
2024-05-23 10:03:56.963072215 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Sqrt node '/transformer/resblocks/resblocks.0/attn/Sqrt'
2024-05-23 10:03:56.963655278 [W:onnxruntime:, constant_folding.cc:269 ApplyImpl] Could not find a CPU kernel and hence can't constant fold Add node '/Add_79'
Traceback (most recent call last):
  File "/path/PyTorch-SVGRender/test_onnx_txt.py", line 32, in <module>
    text_features = ort_session.run(['output'], ort_inputs)[0]
  File "/path/lib/python3.10/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 220, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Invalid input name: input
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/盐析白兔/article/detail/850162