从pb模型到tflite？_tflite pb区别

今天实现了波形输入到输出，模型直接就是对波形处理，其实中间我用tf做了stft，感觉写的挺费劲的差不多比librosa库每帧多200ms，的确tf库挺慢的。但现在我不想顾这么多，能否试试是否能否加速或者再压缩为tflite？所以这篇的任务就是加速和转tflite。

经过我搜索发现，有的说引入ARM指令集加速，地址在此：百度百科指令集，但寡人也没看出来个什么鬼啊？

有的说算法加速，这个在我这里只有两种情况，1-减少tf计算的复杂度，一周时间，2-直接将stft/istft采用cpp或者Java来写，这个估计要花费两周时间，我推荐第2个，因为模型中有些东西是每次都用的常数，都放在模型中每次都要重新计算，不太好。

有的说量化模型，但我的pb模型在转化时已经进行了8位压缩，不知道算不算在内，可以尝试下，tflite的资料，资源在此：TensorFlow官方资料 试试也无妨，诱人的是人家说RNN可以加速3倍以上，不知道是不是吹牛逼，有没有普适性？？

其他可优化的地方：1-模型参数可以少训练几次，但这样需要记录loss，也不麻烦；

2-是否可以512来进行？这样能减少时间吗？

补充一：我特么看了看TensorFlow的contrib下根本没有lite了，所以上面的第二个参考地址无效，官方已经移除了这个玩意。只能先参考bazel build的方法了，试试看了。补充：其实lite直接放在了tf下，所以contrib下没有了。

补充二：其实这个模型的Model定义时是对短时谱进行的处理，只不过我将stft及istft都融入了Model，当然这是在训练完成后融入的，不过在我转成pb时定义的y_out2是没有任何报错的，而且用

for nn in sess.graph.get_operations():#得到所有节点的名字
        print(nn.name)
是能看到我定义的节点名字y_out2，但是转成的pb在转tflite时却告诉我在图中没有我定义的y_out2

2019-09-17 20:34:57.551997: F tensorflow/lite/toco/tooling_util.cc:918] Check failed: GetOpWithOutput(model, output_array) Specified output array "y_out2" is not produced by any op in this graph. Is it a typo? This should not happen. If you trigger this error please send a bug report (with code to reporduce this error), to the TensorFlow Lite team.
Fatal Python error: Aborted

我测试了pb模型，用我定义的y_out2是能够得到正确结果的，这是为啥？

我特么招谁惹谁了，这特么也是醉了，难道非要我用底层编写stft不可？烦。

补充三：为了看看图到底是个什么鬼，我在tensorboard中可视化了我的图，明显这里面有y_out2

明显可见y_out2是输出

WHY ？？？没道理啊。

给个图的全景照吧

补充四：昨天将睡时突然想起来可以试试没有stft的直接将pb转下tflite，如果可行，那么应该就是我后来融入stft的问题（虽说不能百分百肯定，也差不多），如果不可行，那么就是模型转化的问题了。像我这么爱工作的人不多了，有问题就会时时想着。

我特么同样得到的是y_out1没有定义在op，（如果是处理的stft的话，输出的两个短时谱都要保留，不然咋用掩蔽，输出节点必须有两个，y_out1和y_out2,输入和输出都是同样的shape=（2,4,513））

2019-09-18 10:00:37.281096: F tensorflow/lite/toco/tooling_util.cc:918] Check failed: GetOpWithOutput(model, output_array) Specified output array "y_out1" is not produced by any op in this graph. Is it a typo? This should not happen. If you trigger this error please send a bug report (with code to reporduce this error), to the TensorFlow Lite team.
Fatal Python error: Aborted

我特么看看这个图是个啥玩意，为啥老是说没有定义，我去年没有买表啊。温馨提示下：IE和Chrome浏览器都不太好，Edge浏览器可以，看图不错。再特么明显不过的图了，为啥tflite这么难搞啊？？看看下面这个图多么清晰，输入和输出都非常明显。然后告诉我找不到y_out1的定义??尼玛，你是瞎啊。

补充五：孤准备转投bazel了，直接用tf来转似乎存在问题，根据bazel在Ubuntu18上的安装-官方资源中Using Bazel's APT repository 下面的步骤，安装完了，不知道是否能够发挥作用。安装好后告诉我出错了，

pb2tflite$ bazel build tensorflow/contrib/lite/toco:toco
ERROR: The 'build' command is only supported from within a workspace (below a directory having a WORKSPACE file).
See documentation at https://docs.bazel.build/versions/master/build-ref.html#workspace
WARNING: --batch mode is deprecated. Please instead explicitly shut down your Bazel server using the command "bazel shutdown".

这特么啥玩意？？？后来发现要下载TensorFlow源码才能编译。编译时出错，这个就有点烦人了。

补充六：有人说我为啥不放代码，因为我觉得实现这些很简单，并没有难度，所以就没放。

python3.5 Ubuntu1804 下又尝试下之前的lite转化，代码如下，不知道为啥错了，这是训练时的模型，没有任何改变。

path="./my_freeze_graph12.pb"        #pb
inputs=["x_mixed"]               #输入节点名称
outputs=["y_out1","y_out2"]            #输出节点名称
converter = tf.contrib.lite.TocoConverter.from_frozen_graph(path, inputs, outputs)
tflite_model=converter.convert()
open("./my_12.tflite", "wb").write(tflite_model)
 
ValueError: NodeDef mentions attr 'batch_dims' not in Op<name=GatherV2; 
 
signature=params:Tparams, indices:Tindices, axis:Taxis -> output:Tparams; 
 
attr=Tparams:type; attr=Tindices:type,allowed=[DT_INT32, DT_INT64]; 
 
attr=Taxis:type,allowed=[DT_INT32, DT_INT64]>; 
NodeDef: {{node y_hat_src1/Tensordot/GatherV2}} = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_INT32, batch_dims=0]
(y_hat_src1/Tensordot/Shape, y_hat_src1/Tensordot/free, y_hat_src1/Tensordot/GatherV2/axis). 
(Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.)

补充七：放大招，我看都是直接toco就可以了，似乎很吊

在tensorflow1.9以上版本,替换input_file为 graph_def_file,同时去掉--input_format=TENSORFLOW_GRAPHDEF

toco \
  --graph_def_file=./retrained_graph.pb \
  --output_file=./optimized_graph.lite \
  --output_format=TFLITE \
  --input_shape=1,224,224,3 \
  --input_array=input \
  --output_array=final_result \
  --inference_type=FLOAT \
  --inference_input_type=FLOAT

如果在转为pb时用的是8位量化，请用QUANTIZED_UINT8数据类型，否则错误太多了，你改到死都改不完。

如果用了unit8，那么就必须要有4个参数输入 ：标准差、均值、最小值、最大值

--std_dev_values STD_DEV_VALUES      --mean_values MEAN_VALUES

--default_ranges_min DEFAULT_RANGES_MIN           --default_ranges_max DEFAULT_RANGES_MAX

官方量化方法与我的稍微有点不同，我用的是tf.contrib.quantize.create_eval_graph()，对于这一点，一个大佬解释了，只能有一个量化存在。一个大佬的话，并且指明了，如果是量化的，不需要指定后两个

--default_ranges_min DEFAULT_RANGES_MIN           --default_ranges_max DEFAULT_RANGES_MAX

tf官方给出一些参数的解释：--input_arrays=foo,bar --input_shapes=2,3:4,5,6 表示 "foo" 的形状为 [2, 3]， "bar" 的形状为 [4, 5, 6]。

mean_values 与 std_dev_values 的意义如下：量化输入数组中的每个量化值将根据如下公式被解读为一个数学实数（即一个输入激活值），real_value = (quantized_input_value - mean_value) / std_dev_value

当进行量化推断 （--inference_type=QUANTIZED_UINT8） 时，推断代码不会进行反量化。然而，所有数组的量化函数参数，包括输入数组通过 mean_value 和 std_dev_value 指定的量化函数参数，决定了量化推断代码中使用的不动点乘数。mean_value 在进行量化推断时必须是整数。

为啥还要量化和反量化啊，有点懵逼啊！！！

补充八：融入stft的东西有问题，很多东西不知道咋不支持，Op为啥都不支持啊！！FFT是无法支持的，我没法了。

toco \--graph_def_file=./my_freeze_graph_stft.pb 
\--output_file=./my_stft.tflite 
\--output_format=TFLITE 
\--input_shape=1024 
\--input_array=x_mixed 
\--output_array=y_out2 
\--inference_type=QUANTIZED_UINT8 
\--inference_input_type=QUANTIZED_UINT8 
\--std_dev_values=1 
\--mean_values=0
 
#error
 
2019-09-18 16:47:09.876900: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Cos\n2019-09-18 16:47:09.877130: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: ReverseV2\n2019-09-18 16:47:09.877171: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: ReverseV2\n2019-09-18 16:47:09.877295: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: RFFT\n2019-09-18 16:47:09.877309: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1127] Op node missing output type attribute: rfft\n2019-09-18 16:47:09.877337: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: ComplexAbs\n2019-09-18 16:47:09.877355: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Angle\n2019-09-18 16:47:09.877465: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: TensorArrayV3\n2019-09-18 16:47:09.877490: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1127] Op node missing output type attribute: rnn/TensorArray\n2019-09-18 16:47:09.877503: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: TensorArrayV3\n2019-09-18 16:47:09.877515: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1127] Op node missing output type attribute: rnn/TensorArray_1\n2019-09-18 16:47:09.877548: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: TensorArrayScatterV3\n2019-09-18 16:47:09.877565: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1127] Op node missing output type attribute: rnn/TensorArrayUnstack/TensorArrayScatter/TensorArrayScatterV3\n2019-09-18 16:47:09.877585: I tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.877603: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.877616: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.877628: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.877641: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.877653: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.877675: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.877689: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.877704: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: LoopCond\n2019-09-18 16:47:09.877715: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1127] Op node missing output type attribute: rnn/while/LoopCond\n2019-09-18 16:47:09.877751: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.877764: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:189] Unsupported data type in placeholder op: 20\n2019-09-18 16:47:09.877774: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.877786: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: TensorArrayReadV3\n2019-09-18 16:47:09.878281: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.878304: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.878335: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.878353: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.878701: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.878722: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.878752: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.878769: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.879113: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.879134: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.879163: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.879180: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.879206: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Enter\n2019-09-18 16:47:09.879219: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:189] Unsupported data type in placeholder op: 20\n2019-09-18 16:47:09.879230: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: TensorArrayWriteV3\n2019-09-18 16:47:09.879244: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1127] Op node missing output type attribute: rnn/while/TensorArrayWrite/TensorArrayWriteV3\n2019-09-18 16:47:09.879266: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Exit\n2019-09-18 16:47:09.879283: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: TensorArraySizeV3\n2019-09-18 16:47:09.879296: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1127] Op node missing output type attribute: rnn/TensorArrayStack/TensorArraySizeV3\n2019-09-18 16:47:09.879317: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: TensorArrayGatherV3\n2019-09-18 16:47:09.879686: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Abs\n2019-09-18 16:47:09.879706: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Abs\n2019-09-18 16:47:09.879717: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: Abs\n2019-09-18 16:47:09.879743: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:189] Unsupported data type in placeholder op: 8\n2019-09-18 16:47:09.879755: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:189] Unsupported data type in placeholder op: 8\n2019-09-18 16:47:09.879793: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1080] Converting unsupported operation: IRFFT\n2019-09-18 16:47:09.879805: I 
 
tensorflow/contrib/lite/toco/import_tensorflow.cc:1127] Op node missing output type attribute: irfft\n2019-09-18 16:47:09.880015: F 
 
tensorflow/contrib/lite/toco/tooling_util.cc:1377] Check failed: batch == 1 (1024 vs. 1)\nAborted
 

这么多不支持，注定要用底层写stft了。

不过当我用训练的模型时（未融入stft，并且输出为两个）结果仍然出错，不过这时错误不同了，这里的错误应该是两个tf版本不同的问题，因为我训练时采用的服务器tf-gpu 版本为1.12.0 ，而转化为pb时用的win下的1.14.0，不妨试试在服务器上转成pb了

toco --graph_def_file=./my_freeze_graph12.pb --output_file=./my_12.tflite 
--output_format=TFLITE --input_shape=2,4,513 --input_array=x_mixed 
--output_array=y_out1,y_out2 --inference_type=QUANTIZED_UINT8 
--inference_input_type=FLOAT --std_dev_values=1 --mean_values=0
 
#先不考虑--inference_input_type=FLOAT是否正确
#error
 
tensorflow.python.framework.errors_impl.InvalidArgumentError: NodeDef mentions attr 'batch_dims' not in Op<name=GatherV2; 
signature=params:Tparams, indices:Tindices, axis:Taxis -> output:Tparams; 
attr=Tparams:type; attr=Tindices:type,allowed=[DT_INT32, DT_INT64]; attr=Taxis:type,allowed=[DT_INT32, DT_INT64]>; 
NodeDef: {{node y_hat_src1/Tensordot/GatherV2}} = GatherV2[Taxis=DT_INT32, Tindices=DT_INT32, Tparams=DT_INT32, batch_dims=0]
(y_hat_src1/Tensordot/Shape, y_hat_src1/Tensordot/free, y_hat_src1/Tensordot/GatherV2/axis). 
(Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.)
 

另外有相关问题可以加入QQ群讨论，不设微信群

QQ群：868373192 

语音深度学习群