devbox
盐析白兔
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

Ubuntu Linux下安装 TensorFlow等开发环境_i_onemkl_p2023.2.0.49

作者：盐析白兔 | 2024-03-03 16:21:55

赞

踩

i_onemkl_p2023.2.0.49

1.  安装基本工具

  1. sudo apt-get install build-essential
  2. sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
  1. sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev libgtkglext1-dev libgtk-3-dev
  2. sudo apt install libcanberra-gtk-module libcanberra-gtk3-module


2. 安装 OPENMP
    

sudo apt install libomp-dev

3. 打开 WayLand    

  编辑 /etc/gdm3/custom.conf，修改配置使得，“WaylandEnable=true”

  1. # edit /etc/gdm3/custom.conf
  2. sudo nano /etc/gdm3/custom.conf
  3. ******************************
  4. + WaylandEnable=true 
  5. ******************************
  6.  
  7. # restart GDM
  8. sudo systemctl restart gdm3

4. [可选] 安装 SCR

    >> tesseract是一个开源的OCR引擎，最初是由惠普公司开发用来作为其平板扫描仪的 OCR引擎，2005年惠普将其开源出来，之后google接手负责维护    

  1. sudo add-apt-repository ppa:alex-p/tesseract-ocr
  2. sudo apt-get update 
  3. sudo apt-get install tesseract-ocr

    >> 字库下载 : tesseract支持60多种语言的识别不同，使用之前需要先下载对应语言的字库，下载地址：https://github.com/tesseract-ocr/tessdata
    latest update : 2021 .    5 years ago generically 
    >> 下载完成之后把.traineddata字库文件放到tessdata目录下，默认路径是/usr/share/tesseract-ocr /4.00/tessdata    

  1. wget https://gitcode.net/mirrors/tesseract-ocr/tessdata/-/archive/4.1.0/tessdata-4.1.0.tar.gz
  2. tar xf tessdata-4.1.0.tar.gz
  3. sudo cp -a tessdata-4.1.0/*.traineddata /usr/share/tesseract-ocr/4.00/tessdata/

5.  安装 Intel@ openCL

    ( https://github.com/intel/compute-runtime/releases )
  

  1.  mkdir neo && cd neo
  2. # Download all *.deb packages
  3. wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.14062.11/intel-igc-core_1.0.14062.11_amd64.deb
  4. wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.14062.11/intel-igc-opencl_1.0.14062.11_amd64.deb
  5. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/intel-level-zero-gpu-dbgsym_1.3.26516.18_amd64.ddeb
  6. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/intel-level-zero-gpu_1.3.26516.18_amd64.deb
  7. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/intel-opencl-icd-dbgsym_23.22.26516.18_amd64.ddeb
  8. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/intel-opencl-icd_23.22.26516.18_amd64.deb
  9. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/libigdgmm12_22.3.0_amd64.deb 
  10. # Verify sha256 sums for packages
  11. wget https://github.com/intel/compute-runtime/releases/download/23.22.26516.18/ww22.sum
  12. sha256sum -c ww22.sum
  13. # Install all packages as root
  14. sudo dpkg -i *.deb
  15. # In case of installation problems, please install required dependencies, for example:
  16. sudo apt install ocl-icd-libopencl1


6.  安装 Intel@ oneAPI MKL (Math Kernel Library)

    6.1 在线安装

    ( https://www.intel.com/content/www/us/en/docs/oneapi/installation-guide-linux/2023-0/apt.html )
        

  1. # download the key to system keyring
  2. wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null
  3. # add signed entry to apt sources and configure the APT client to use Intel repository:
  4. echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list
  5. sudo apt update
  6. sudo apt install intel-basekit 
  7. # Intel® oneAPI HPC Toolkit
  8. sudo apt install intel-hpckit
  9. # Intel® oneAPI IoT Toolkit
  10. sudo apt install intel-iotkit
  11. # Intel® oneAPI DL Framework Developer Toolkit
  12. sudo apt install intel-dlfdkit 
  13. # Intel® AI Analytics Toolkit
  14. sudo apt install intel-aikit
  15. # Intel® oneAPI Rendering Toolkit
  16. sudo apt install intel-renderkit

    6.2 离线安装

  ( https://www.intel.com/content/www/us/en/developer/tools/oneapi/base-toolkit-download.html )

  1. wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/adb8a02c-4ee7-4882-97d6-a524150da358/l_onemkl_p_2023.2.0.49497_offline.sh
  2. sudo sh ./l_onemkl_p_2023.2.0.49497_offline.sh

       +------------------------------------------------------------------------
        | Recommended for host machines with poor or no internet connection
        | Size     913.25 MB
        | Version     2023.2.0
        | Date     July 13, 2023
        | SHA384     f5cc20cdd92ab961693c7649fb0b046937ae8aae92eb1464090a187816e7bad3ccd6ef5bf90924226d5f4d1314fe57ab
        +------------------------------------------------------------------------ 

7. 编译 open-cv 带 opencv_contrib

  1. # Install minimal prerequisites (Ubuntu 18.04 as reference)
  2. sudo apt update && sudo apt install -y cmake g++ wget unzip
  3. # Download and unpack sources
  4. wget -O opencv.zip https://github.com/opencv/opencv/archive/4.x.zip
  5. wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.x.zip
  6. unzip opencv.zip
  7. unzip opencv_contrib.zip
  8. # Create build directory and switch into it
  9. mkdir -p build && cd build
  10. # Configure
  11. cmake -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib-4.x/modules ../opencv-4.x
  12. # or debug mode
  13. cmake -DCMAKE_BUILD_TYPE=Debug -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib-4.x/modules ../opencv-4.x
  14. # Build
  15. cmake --build . -j 8

8. 直接安装预编译好的 TensorFlow

(https://tensorflow.google.cn/install/source?hl=zh-cn)   

pip install tensorflow -i https://pypi.tuna.tsinghua.edu.cn/simple

    或    

pip install tensorflow==2.13 -i https://pypi.tuna.tsinghua.edu.cn/simple

    TensorFlow 将被安装在 ~/.local/lib/python3.8/site-packages/tensorflow 路径下，使用TensorFlow 库的项目将需要包含这个路径 ~/.local/lib/python3.8/site-packages/tensorflow/include as include directory.
    use ~/.local/lib/python3.8/site-packages/tensorflow as lib path
    
      建议添加指向库的软链接，方便部分基于这些库的开源项目的编译。

  1. cd ~/.local/lib/python3.8/site-packages/tensorflow
  2. ln -s libtensorflow_cc.so.2        libtensorflow_cc.so
  3. ln -s libtensorflow_framework.so.2 libtensorflow_framework.so
  4.  
  5. cd ~/.local/lib/python3.8/site-packages/numpy.libs
  6. ln -s libopenblas64_p-r0-15028c96.3.21.so  libopenblas.so
  7. ln -s libquadmath-96973f99.so.0.0.0        libquadmath.so

9.  手动编译 TenserFlow C++ 库

（https://blog.csdn.net/MOU_IT/article/details/87976152)

9.1 install protobuf

  1. wget https://github.com/protocolbuffers/protobuf/releases/download/v3.7.1/protobuf-cpp-3.7.1.tar.gz
  2. tar -xzvf protobuf-cpp-3.7.1.tar.gz
  3. sudo apt-get install automake libtool
  4. ./autogen.sh
  5. ./configure
  6. make
  7. sudo make install
  8. sudo ldconfig
  9. # sudo make uninstall 安装错版本后卸载指令
  10. protoc --version  # 查看protobuf版本

9.2 Install bazel 

        +--------------------------------+--------------------------+
        |    tensorflow 2.15.0 rc     |    Bazel 6.1.0          |
        |    tensorflow 2.13.0         |    Bazel 5.3.0          |
        +--------------------------------+--------------------------+
        bazel是Google开源的一套编译构建工具，广泛应用于Google内部，包括TensorFlow项目。

  1. # prepare tools
  2. sudo apt-get install pkg-config zip g++ zlib1g-dev unzip python
  3. # download bazel 5.3.0
  4. wget https://github.com/bazelbuild/bazel/releases/download/5.3.0/bazel-5.3.0-installer-linux-x86_64.sh
  5. sudo chmod +x bazel-5.3.0-installer-linux-x86_64.sh
  6. # install bazel to $HOME/.bazel/bin
  7. ./bazel-5.3.0-installer-linux-x86_64.sh --user
  8. export PATH="$PATH:$HOME/bin"

9.3 downlaod tenserFlow

  1. git clone --recursive https://github.com/tensorflow/tensorflow.git
  2. cd tensorflow
  3. # switch to the branch you want to build
  4. git checkout r2.13  # r1.9, r1.10, etc. 

      

9.4 build tenserFlow library

(libtensorflow_cc.so & libtensorflow_cc_framework.so)

  1. ./configure
  2. ## tensorflow:libtensorflow_cc.so
  3. bazel build --config=opt  //tensorflow:libtensorflow_cc.so
  4.  
  5. ## tensorflow:libtensorflow_cc.so with cuda
  6. bazel build --config=opt --config=cuda //tensorflow:libtensorflow_cc.so
  7.  
  8. ## tensorflow/tools/pip_package:build_pip_package
  9. bazel build -c opt --copt=-msse3 --copt=-msse4.1 --copt=-msse4.2 --copt=-mavx --copt=-mavx2 --copt=-mfma //tensorflow:libtensorflow_cc.so
  10.  
  11. bazel build
  12.  
  13. ## build install python-package
  14. bazel-bin/tensorflow/tools/pip_package/build_pip_package package/20230912
  15.  
  16. ## remove old version and install new package
  17. pip uninstall  tensorflow
  18. pip install package/20230912/tensorflow-*.whl

9.5  安装 eigen

参考 ： https://eigen.tuxfamily.org/index.php?title=Main_Page

下载&解压：

  1. wget  https://gitlab.com/libeigen/eigen/-/archive/3.4.0/eigen-3.4.0.tar.bz2
  2. tar xf eigen-3.4.0.tar.bz2

编译&安装： 

  1. cd eigen-3.4.0
  2. cmake .. && sudo make install

安装后,头文件安装在/usr/local/include/eigen3/路径下，很多程序中include时 经常使用#include <Eigen/Dense>，而不是使用#include <eigen3/Eigen/Dense>，所以要建立一个软链接。

sudo ln -s /usr/local/include/eigen3/Eigen /usr/local/include/Eigen

9.6 安装 ml_types

     (https://pypi.org/project/ml-dtypes/) (https://github.com/jax-ml/ml_dtypes)
        ml_dtypes is a stand-alone implementation of several NumPy dtype extensions used in machine learning libraries, 

  1. sudo apt install python3-pip
  2. sudo pip install ml_dtypes

 否则将编译 TensorFlow时将报告如下错误： 

     /usr/local/lib/python3.8/dist-packages/third_party/eigen/Eigen
 [ 50%] Building CXX object CMakeFiles/tf_test.dir/src/hello.cpp.o 
  In file included from /home/rd/tensorflow/tensorflow/core/platform/float8.h:19,
   from /home/rd/tensorflow/tensorflow/core/platform/types.h:20,
   from /home/rd/tensorflow/tensorflow/core/platform/env_time.h:20,
   from /home/rd/tensorflow/tensorflow/core/platform/env.h:26,
   from /home/rd/tensorflow-test-prog/src/hello.cpp:1:
   /home/rd/tensorflow/tensorflow/tsl/platform/float8.h:19:10: fatal error: include/float8.h:没有那个文件或目录 
 19 | #include "include/float8.h"  // from @ml_dtypes 
    |          ^~~~~~~~~~~~~~~~~~
 compilation terminated.


9.7 安装 abseil （absl）

  1. wget https://github.com/abseil/abseil-cpp/archive/refs/tags/20230125.3.tar.gz
  2. tar  xf 20230125.3.tar.gz
  3. cd abseil-cpp-20230125.3
  4. mkdir build
  5. cd build
  6. cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local
  7. make -j8
  8. sudo make install

否则将编译 TensorFlow时将报告如下错误：

        | In file included from /home/rd/tensorflow/tensorflow/core/platform/cord.h:19,
        |                  from /home/rd/tensorflow/tensorflow/core/platform/tstring.h:19,
        |                  from /home/rd/tensorflow/tensorflow/core/platform/types.h:22,
        |                  from /home/rd/tensorflow/tensorflow/core/platform/env_time.h:20,
        |                  from /home/rd/tensorflow/tensorflow/core/platform/env.h:26,
        |                  from /home/rd/tensorflow-test-prog/src/hello.cpp:1:
        | /home/rd/tensorflow/tensorflow/tsl/platform/cord.h:21:10: fatal error: absl/strings/cord.h: 没有那个文件或目录
        |    21 | #include "absl/strings/cord.h"  // IWYU pragma: export
        |       |          ^~~~~~~~~~~~~~~~~~~~~
 

9.8 Install protoc & protobuf 

  1. sudo apt-get install autoconf automake libtool curl make g++ unzip
  2. wget https://github.com/protocolbuffers/protobuf/releases/download/v24.2/protobuf-24.2.tar.gz
  3. tar xf protobuf-24.2.tar.gz
  4. cd protobuf-24.2
  5. cp -a abseil-cpp-20230125.3  thirdparty/abseil-cpp
  6. mkdir build && cd build
  7. cmake -Dprotobuf_BUILD_TESTS=OFF ..
  8. make -j8
  9. sudo make install
  10. sudo ldconfig # refresh shared library cache.

或

  1. wget https://github.com/protocolbuffers/protobuf/releases/download/v24.2/protoc-24.2-linux-x86_64.zip
  2. unzip protoc-24.2-linux-x86_64.zip
  3. sudo cp bin/protoc /usr/local/bin/
  4. sudo cp -a include/google /usr/local/include/

否则将编译 TensorFlow时将报告如下错误： 

     In file included from /home/rd/tensorflow/tensorflow/tsl/platform/status.h:39,
                  from /home/rd/tensorflow/tensorflow/core/platform/status.h:23,
                  from /home/rd/tensorflow/tensorflow/core/platform/errors.h:27,
                  from /home/rd/tensorflow/tensorflow/core/platform/env.h:27,
                  from /home/rd/tensorflow-test-prog/src/hello.cpp:1:
 /home/rd/tensorflow/tensorflow/tsl/protobuf/error_codes.pb.h:11:10: fatal error: google/protobuf/port_def.inc: 没有那个文件或目录
    11 | #include "google/protobuf/port_def.inc"
       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 compilation terminated.
 make[2]: *** [CMakeFiles/tf_test.dir/build.make:63：CMakeFiles/tf_test.dir/src/hello.cpp.o] 错误 1
 make[1]: *** [CMakeFiles/Makefile2:76：CMakeFiles/tf_test.dir/all] 错误 2
 make: *** [Makefile:84：all] 错误 2

9.9    Build error_codes.pb.h & error_codes.pb.cc and so on


        cd ~/tensorflow
        ls tensorflow/tsl/protobuf
            bfc_memory_map.proto  coordination_config.proto   distributed_runtime_payloads.proto  error_codes.proto  rpc_options.proto  test_log.proto
            BUILD                 coordination_service.proto  dnn.proto                           histogram.proto    status.proto
        protoc  --cpp_out=.  tensorflow/tsl/protobuf/*.proto
        ls tensorflow/tsl/protobuf/
            bfc_memory_map.pb.cc       coordination_config.proto           distributed_runtime_payloads.proto  error_codes.proto  rpc_options.proto  test_log.proto
            bfc_memory_map.pb.h        coordination_service.pb.cc          dnn.pb.cc                           histogram.pb.cc    status.pb.cc
            bfc_memory_map.proto       coordination_service.pb.h           dnn.pb.h                            histogram.pb.h     status.pb.h
            BUILD                      coordination_service.proto          dnn.proto                           histogram.proto    status.proto
            coordination_config.pb.cc  distributed_runtime_payloads.pb.cc  error_codes.pb.cc                   rpc_options.pb.cc  test_log.pb.cc
            coordination_config.pb.h   distributed_runtime_payloads.pb.h   error_codes.pb.h                    rpc_options.pb.h   test_log.pb.h
            
        protoc  --cpp_out=.  tensorflow/*/*/*.proto
        protoc  --cpp_out=.  tensorflow/*/*/*/*.proto
        +--------------------------------------------------solve compile error ----------------------------------------------+
        |             [ 50%] Building CXX object CMakeFiles/tf_test.dir/src/hello.cpp.o
        | In file included from /home/rd/tensorflow/tensorflow/core/platform/status.h:23,
        |                  from /home/rd/tensorflow/tensorflow/core/platform/errors.h:27,
        |                  from /home/rd/tensorflow/tensorflow/core/platform/env.h:27,
        |                  from /home/rd/tensorflow-test-prog/src/hello.cpp:1:
        | /home/rd/tensorflow/tensorflow/tsl/platform/status.h:39:10: fatal error: tensorflow/tsl/protobuf/error_codes.pb.h: 没有那个文件或目录
        |    39 | #include "tensorflow/tsl/protobuf/error_codes.pb.h"
        |       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        | compilation terminated.
        | make[2]: *** [CMakeFiles/tf_test.dir/build.make:63：CMakeFiles/tf_test.dir/src/hello.cpp.o] 错误 1
        | make[1]: *** [CMakeFiles/Makefile2:76：CMakeFiles/tf_test.dir/all] 错误 2
        | make: *** [Makefile:84：all] 错误 2
        +--------------------------------------------------------------------------------------------------------------------+
        [ 50%] Building CXX object CMakeFiles/tf_test.dir/src/hello.cpp.o
        | In file included from /home/rd/tensorflow-test-prog/src/hello.cpp:2:
        | /home/rd/tensorflow/tensorflow/core/public/session.h:24:10: fatal error: tensorflow/core/framework/device_attributes.pb.h: 没有那个文件或目录
         |   24 | #include "tensorflow/core/framework/device_attributes.pb.h"
        |       |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        | compilation terminated.
        | make[2]: *** [CMakeFiles/tf_test.dir/build.make:63：CMakeFiles/tf_test.dir/src/hello.cpp.o] 错误 1
        | make[1]: *** [CMakeFiles/Makefile2:76：CMakeFiles/tf_test.dir/all] 错误 2
        | make: *** [Makefile:84：all] 错误 2
        +--------------------------------------------------------------------------------------------------------------------+
        

9.10   创建指向 libtensorflow_framework.so.2.15.0 的符号链接

        cd ~/tensorflow/bazel-bin/tensorflow
        ln -s libtensorflow_framework.so.2.15.0 libtensorflow_framework.so
        ln -s libtensorflow_framework.so.2.15.0 libtensorflow_framework.so.2
            lrwxrwxrwx  1 rd rd        33 9月   6 16:43 libtensorflow_framework.so -> libtensorflow_framework.so.2.15.0
            lrwxrwxrwx  1 rd rd        33 9月   6 16:43 libtensorflow_framework.so.2 -> libtensorflow_framework.so.2.15.0
        +--------------------------------------------------solve compile error ----------------------------------------------+
        |     -- Build files have been written to: /home/rd/tensorflow-test-prog
        | [ 50%] Building CXX object CMakeFiles/tf_test.dir/src/hello.cpp.o
        | [100%] Linking CXX executable tf_test
        | /usr/bin/ld: 找不到 -ltensorflow_framework
        | collect2: error: ld returned 1 exit status
        | make[2]: *** [CMakeFiles/tf_test.dir/build.make:84：tf_test] 错误 1
        | make[1]: *** [CMakeFiles/Makefile2:76：CMakeFiles/tf_test.dir/all] 错误 2
        | make: *** [Makefile:84：all] 错误 2s
        +--------------------------------------------------------------------------------------------------------------------+


 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/盐析白兔/article/detail/185098?site
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号