热门标签
热门文章
- 1linux USB驱动分析(四)USB hub集线器驱动分析_usb hub linux驱动
- 2java clone 中的浅复制和深复制
- 3[总结]mac pro 提示“USB设备已停用”导致外接显示器黑屏_usb配件已停用 拔掉耗电量太大的配件电脑进水后
- 4（转）zookeeper报错:NoNodeException: KeeperErrorCode = NoNode for /XXX Node does not exist_zk error: -101 no node
- 5【运动控制】纯跟踪算法_车辆控制中纯跟踪算法的缺点有哪些
- 6机器学习笔记之-数据共线性的问题_样板数量太少 多重共线性
- 7c语言----学生选课系统_学生选课信息系统c语言代码怎么写
- 8参考文献中各字母的含义_参考文献cp
- 9程序员离职原因的最佳回答_面试官：你为什么跳槽？哪些能回答，哪些不宜回答你知道吗？...
- 10业内同行盆友来稿：对libc.so下毒手引发的惨痛血案，围观大型翻车现场...
当前位置: article > 正文
Bert4keras相关环境安装_bert4keras0.4.2
作者：Gausst松鼠会 | 2024-05-29 02:10:22
赞
踩
bert4keras0.4.2
1. 创建虚拟环境，先安装cudatoolkit和cudnn
cudatoolkit版本不对容易报so库找不到的错。
具体安装参考：在conda虚拟环境中安装cuda和cudnn，解决pip install or python setup.py install 将安装包安装到全局环境中的问题（1.2部分）。 - 知乎
2. 安装对应的tensorflow版本与依赖
- (bert4keras-tf22) conda list
- # packages in environment at /home/anaconda3/envs/bert4keras-tf22:
- #
- # Name Version Build Channel
- _libgcc_mutex 0.1 main defaults
- _openmp_mutex 5.1 1_gnu defaults
- _tflow_select 2.1.0 gpu defaults
- absl-py 0.15.0 pyhd3eb1b0_0 defaults
- aiohttp 3.8.1 py38h7f8727e_1 defaults
- aiosignal 1.2.0 pyhd3eb1b0_0 defaults
- astor 0.8.1 py38h06a4308_0 defaults
- asttokens 2.0.5 pyhd3eb1b0_0 defaults
- astunparse 1.6.3 py_0 defaults
- async-timeout 4.0.1 pyhd3eb1b0_0 defaults
- attrs 21.4.0 pyhd3eb1b0_0 defaults
- backcall 0.2.0 pyhd3eb1b0_0 defaults
- blas 1.0 mkl defaults
- blinker 1.4 py38h06a4308_0 defaults
- brotlipy 0.7.0 py38h27cfd23_1003 defaults
- c-ares 1.18.1 h7f8727e_0 defaults
- ca-certificates 2022.10.11 h06a4308_0 defaults
- cachetools 4.2.2 pyhd3eb1b0_0 defaults
- certifi 2022.9.24 py38h06a4308_0 defaults
- cffi 1.15.1 py38h74dc2b5_0 defaults
- charset-normalizer 2.0.4 pyhd3eb1b0_0 defaults
- click 8.0.4 py38h06a4308_0 defaults
- cryptography 37.0.1 py38h9ce1e76_0 defaults
- cudatoolkit 10.1.243 h6bb024c_0 defaults
- cudnn 7.6.5 cuda10.1_0 defaults
- cupti 10.1.168 0 defaults
- dataclasses 0.8 pyh6d0b6a4_7 defaults
- debugpy 1.5.1 py38h295c915_0 defaults
- decorator 5.1.1 pyhd3eb1b0_0 defaults
- entrypoints 0.4 py38h06a4308_0 defaults
- executing 0.8.3 pyhd3eb1b0_0 defaults
- frozenlist 1.2.0 py38h7f8727e_0 defaults
- gast 0.3.3 py_0 defaults
- google-auth 2.6.0 pyhd3eb1b0_0 defaults
- google-auth-oauthlib 0.4.4 pyhd3eb1b0_0 defaults
- google-pasta 0.2.0 pyhd3eb1b0_0 defaults
- grpcio 1.42.0 py38hce63b2e_0 defaults
- h5py 2.10.0 py38hd6299e0_1 defaults
- hdf5 1.10.6 hb1b8bf9_0 defaults
- idna 3.3 pyhd3eb1b0_0 defaults
- importlib-metadata 4.11.3 py38h06a4308_0 defaults
- intel-openmp 2021.4.0 h06a4308_3561 defaults
- ipykernel 6.15.2 py38h06a4308_0 defaults
- ipython 8.4.0 py38h06a4308_0 defaults
- jedi 0.18.1 py38h06a4308_1 defaults
- jupyter_client 7.3.5 py38h06a4308_0 defaults
- jupyter_core 4.11.1 py38h06a4308_0 defaults
- keras 2.3.1 0 defaults
- keras-applications 1.0.8 py_1 defaults
- keras-base 2.3.1 py38_0 defaults
- keras-preprocessing 1.1.2 pyhd3eb1b0_0 defaults
- ld_impl_linux-64 2.38 h1181459_1 defaults
- libffi 3.3 he6710b0_2 defaults
- libgcc-ng 11.2.0 h1234567_1 defaults
- libgfortran-ng 7.5.0 ha8ba4b0_17 defaults
- libgfortran4 7.5.0 ha8ba4b0_17 defaults
- libgomp 11.2.0 h1234567_1 defaults
- libprotobuf 3.20.1 h4ff587b_0 defaults
- libsodium 1.0.18 h7b6447c_0 defaults
- libstdcxx-ng 11.2.0 h1234567_1 defaults
- markdown 3.3.4 py38h06a4308_0 defaults
- matplotlib-inline 0.1.6 py38h06a4308_0 defaults
- mkl 2021.4.0 h06a4308_640 defaults
- mkl-service 2.4.0 py38h7f8727e_0 defaults
- mkl_fft 1.3.1 py38hd3c417c_0 defaults
- mkl_random 1.2.2 py38h51133e4_0 defaults
- multidict 5.2.0 py38h7f8727e_2 defaults
- ncurses 6.3 h5eee18b_3 defaults
- nest-asyncio 1.5.5 py38h06a4308_0 defaults
- numpy 1.21.5 py38h6c91a56_3 defaults
- numpy-base 1.21.5 py38ha15fc14_3 defaults
- oauthlib 3.2.0 pyhd3eb1b0_1 defaults
- openssl 1.1.1s h7f8727e_0 defaults
- opt_einsum 3.3.0 pyhd3eb1b0_1 defaults
- packaging 21.3 pyhd3eb1b0_0 defaults
- parso 0.8.3 pyhd3eb1b0_0 defaults
- pexpect 4.8.0 pyhd3eb1b0_3 defaults
- pickleshare 0.7.5 pyhd3eb1b0_1003 defaults
- pip 22.1.2 py38h06a4308_0 defaults
- prompt-toolkit 3.0.20 pyhd3eb1b0_0 defaults
- protobuf 3.20.1 py38h295c915_0 defaults
- psutil 5.9.0 py38h5eee18b_0 defaults
- ptyprocess 0.7.0 pyhd3eb1b0_2 defaults
- pure_eval 0.2.2 pyhd3eb1b0_0 defaults
- pyasn1 0.4.8 pyhd3eb1b0_0 defaults
- pyasn1-modules 0.2.8 py_0 defaults
- pycparser 2.21 pyhd3eb1b0_0 defaults
- pygments 2.11.2 pyhd3eb1b0_0 defaults
- pyjwt 2.4.0 py38h06a4308_0 defaults
- pyopenssl 22.0.0 pyhd3eb1b0_0 defaults
- pyparsing 3.0.9 py38h06a4308_0 defaults
- pysocks 1.7.1 py38h06a4308_0 defaults
- python 3.8.13 h12debd9_0 defaults
- python-dateutil 2.8.2 pyhd3eb1b0_0 defaults
- pyyaml 6.0 py38h7f8727e_1 defaults
- pyzmq 23.2.0 py38h6a678d5_0 defaults
- readline 8.1.2 h7f8727e_1 defaults
- requests 2.28.1 py38h06a4308_0 defaults
- requests-oauthlib 1.3.0 py_0 defaults
- rsa 4.7.2 pyhd3eb1b0_1 defaults
- scipy 1.7.3 py38hc147768_0 defaults
- setuptools 61.2.0 py38h06a4308_0 defaults
- six 1.16.0 pyhd3eb1b0_1 defaults
- sqlite 3.39.2 h5082296_0 defaults
- stack_data 0.2.0 pyhd3eb1b0_0 defaults
- tensorboard 2.8.0 py38h06a4308_0 defaults
- tensorboard-data-server 0.6.0 py38hca6d32c_0 defaults
- tensorboard-plugin-wit 1.8.1 py38h06a4308_0 defaults
- tensorflow 2.2.0 gpu_py38hb782248_0 defaults
- tensorflow-base 2.2.0 gpu_py38h83e3d50_0 defaults
- tensorflow-estimator 2.6.0 pyh7b7c402_0 defaults
- tensorflow-gpu 2.2.0 h0d30ee6_0 defaults
- termcolor 1.1.0 py38h06a4308_1 defaults
- tk 8.6.12 h1ccaba5_0 defaults
- tornado 6.2 py38h5eee18b_0 defaults
- traitlets 5.1.1 pyhd3eb1b0_0 defaults
- typing-extensions 4.3.0 py38h06a4308_0 defaults
- typing_extensions 4.3.0 py38h06a4308_0 defaults
- urllib3 1.26.11 py38h06a4308_0 defaults
- wcwidth 0.2.5 pyhd3eb1b0_0 defaults
- werkzeug 2.0.3 pyhd3eb1b0_0 defaults
- wheel 0.37.1 pyhd3eb1b0_0 defaults
- wrapt 1.14.1 py38h5eee18b_0 defaults
- xz 5.2.5 h7f8727e_1 defaults
- yaml 0.2.5 h7b6447c_0 defaults
- yarl 1.6.3 py38h27cfd23_0 defaults
- zeromq 4.3.4 h2531618_0 defaults
- zipp 3.8.0 py38h06a4308_0 defaults
- zlib 1.2.12 h7f8727e_2 defaults
3. tensorflow转onnx环境与依赖
- (bert4keras-tf25) conda list
- # packages in environment at /home/anaconda3/envs/bert4keras-tf25:
- #
- # Name Version Build Channel
- _libgcc_mutex 0.1 main defaults
- _openmp_mutex 5.1 1_gnu defaults
- absl-py 0.15.0 pypi_0 pypi
- asttokens 2.0.5 pyhd3eb1b0_0 defaults
- astunparse 1.6.3 pypi_0 pypi
- backcall 0.2.0 pyhd3eb1b0_0 defaults
- bert4keras 0.11.3 pypi_0 pypi
- ca-certificates 2022.10.11 h06a4308_0 defaults
- cachetools 5.2.0 pypi_0 pypi
- certifi 2022.9.24 py38h06a4308_0 defaults
- charset-normalizer 2.1.0 pypi_0 pypi
- coloredlogs 15.0.1 pypi_0 pypi
- debugpy 1.5.1 py38h295c915_0 defaults
- decorator 5.1.1 pyhd3eb1b0_0 defaults
- entrypoints 0.4 py38h06a4308_0 defaults
- executing 0.8.3 pyhd3eb1b0_0 defaults
- flatbuffers 1.12 pypi_0 pypi
- gast 0.4.0 pypi_0 pypi
- google-auth 2.10.0 pypi_0 pypi
- google-auth-oauthlib 0.4.6 pypi_0 pypi
- google-pasta 0.2.0 pypi_0 pypi
- grpcio 1.34.1 pypi_0 pypi
- h5py 3.1.0 pypi_0 pypi
- humanfriendly 10.0 pypi_0 pypi
- idna 3.3 pypi_0 pypi
- importlib-metadata 4.12.0 pypi_0 pypi
- ipykernel 6.15.2 py38h06a4308_0 defaults
- ipython 8.4.0 py38h06a4308_0 defaults
- jedi 0.18.1 py38h06a4308_1 defaults
- jupyter_client 7.3.5 py38h06a4308_0 defaults
- jupyter_core 4.11.1 py38h06a4308_0 defaults
- keras 2.3.1 pypi_0 pypi
- keras-applications 1.0.8 pypi_0 pypi
- keras-nightly 2.5.0.dev2021032900 pypi_0 pypi
- keras-preprocessing 1.1.2 pypi_0 pypi
- ld_impl_linux-64 2.38 h1181459_1 defaults
- libffi 3.3 he6710b0_2 defaults
- libgcc-ng 11.2.0 h1234567_1 defaults
- libgomp 11.2.0 h1234567_1 defaults
- libsodium 1.0.18 h7b6447c_0 defaults
- libstdcxx-ng 11.2.0 h1234567_1 defaults
- markdown 3.4.1 pypi_0 pypi
- markupsafe 2.1.1 pypi_0 pypi
- matplotlib-inline 0.1.6 py38h06a4308_0 defaults
- mpmath 1.2.1 pypi_0 pypi
- ncurses 6.3 h5eee18b_3 defaults
- nest-asyncio 1.5.5 py38h06a4308_0 defaults
- numpy 1.23.2 pypi_0 pypi
- oauthlib 3.2.0 pypi_0 pypi
- onnx 1.12.0 pypi_0 pypi
- onnxruntime 1.12.1 pypi_0 pypi
- openssl 1.1.1s h7f8727e_0 defaults
- opt-einsum 3.3.0 pypi_0 pypi
- packaging 21.3 pyhd3eb1b0_0 defaults
- parso 0.8.3 pyhd3eb1b0_0 defaults
- pexpect 4.8.0 pyhd3eb1b0_3 defaults
- pickleshare 0.7.5 pyhd3eb1b0_1003 defaults
- pip 22.1.2 py38h06a4308_0 defaults
- prompt-toolkit 3.0.20 pyhd3eb1b0_0 defaults
- protobuf 3.19.4 pypi_0 pypi
- psutil 5.9.0 py38h5eee18b_0 defaults
- ptyprocess 0.7.0 pyhd3eb1b0_2 defaults
- pure_eval 0.2.2 pyhd3eb1b0_0 defaults
- pyasn1 0.4.8 pypi_0 pypi
- pyasn1-modules 0.2.8 pypi_0 pypi
- pygments 2.11.2 pyhd3eb1b0_0 defaults
- pyparsing 3.0.9 py38h06a4308_0 defaults
- python 3.8.13 h12debd9_0 defaults
- python-dateutil 2.8.2 pyhd3eb1b0_0 defaults
- pyyaml 6.0 pypi_0 pypi
- pyzmq 23.2.0 py38h6a678d5_0 defaults
- readline 8.1.2 h7f8727e_1 defaults
- requests 2.28.1 pypi_0 pypi
- requests-oauthlib 1.3.1 pypi_0 pypi
- rsa 4.9 pypi_0 pypi
- scipy 1.9.0 pypi_0 pypi
- setuptools 61.2.0 py38h06a4308_0 defaults
- six 1.15.0 pypi_0 pypi
- sqlite 3.39.2 h5082296_0 defaults
- stack_data 0.2.0 pyhd3eb1b0_0 defaults
- sympy 1.10.1 pypi_0 pypi
- tensorboard 2.10.0 pypi_0 pypi
- tensorboard-data-server 0.6.1 pypi_0 pypi
- tensorboard-plugin-wit 1.8.1 pypi_0 pypi
- tensorflow 2.5.0 pypi_0 pypi
- tensorflow-estimator 2.5.0 pypi_0 pypi
- termcolor 1.1.0 pypi_0 pypi
- tf2onnx 1.12.1 pypi_0 pypi
- tk 8.6.12 h1ccaba5_0 defaults
- tornado 6.2 py38h5eee18b_0 defaults
- traitlets 5.1.1 pyhd3eb1b0_0 defaults
- typing-extensions 3.7.4.3 pypi_0 pypi
- urllib3 1.26.11 pypi_0 pypi
- wcwidth 0.2.5 pyhd3eb1b0_0 defaults
- werkzeug 2.2.2 pypi_0 pypi
- wheel 0.37.1 pyhd3eb1b0_0 defaults
- wrapt 1.12.1 pypi_0 pypi
- xz 5.2.5 h7f8727e_1 defaults
- zeromq 4.3.4 h2531618_0 defaults
- zipp 3.8.1 pypi_0 pypi
- zlib 1.2.12 h7f8727e_2 defaults
特别要注意：keras-nightly 2.5.0.dev2021021100 的安装，其版本号有点特别
4. 踩坑指南
(1) so库找不到
2023-11-25 01:10:45.410281: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /path/to/cuda/lib64:/path/to/cudnn/lib64:安装：cudatoolkit 10.1.243
(2) tf转onnx错误
2023-12-07 04:17:32,217 - ERROR - Tensorflow op [StatefulPartitionedCall/model_1/Transformer-0-MultiHeadSelfAttention/BroadcastArgs: BroadcastArgs] is not supported安装：bert4keras 0.11.3
(3) 线程值错误
/site-packages/keras/backend/tensorflow_backend.py", line 73, in symbolic_fn_wrapper
if _SYMBOLIC_SCOPE.value:
AttributeError: '_thread._local' object has no attribute 'value'
将代码段注释掉：
参考：2021-05-21_if _symbolic_scope.value: attributeerror: '_thread-CSDN博客
声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/Gausst松鼠会/article/detail/640139
推荐阅读
相关标签
