Windows 11 上从零开始基于 wsl-ubuntu 搭建 AI 学习环境及部署多种私有 ChatGPT_部署私有ai

0. 背景和简介

今天(2023/6/2)刚刚换了 1 块 4TB 的硬盘，在 Windows 11 上从零开始，基于 Windows Subsystem for Linux 搭建一套 AI 学习环境。

其中包括，安装 Python、安装 Miniconda3、安装 CUDA Toolkit、安装 git lfs、配置 Hugging Face 的缓存路径、本地部署 ChatGLM-6B、本地部署 VisualGLM-6B等等内容。

1. 安装 wsl-ubuntu

1. 安装 Ubuntu-22.04，默认会安装在 C 盘，

wsl.exe --install -d Ubuntu-22.04
1

2. 导出 Ubuntu-22.04，

wsl --export Ubuntu-22.04 D:\tmp\Ubuntu-22.04.tar
1

3. 删除 Ubuntu-22.04，

wsl --unregister Ubuntu-22.04
1

4. 导入 Ubuntu-22.04，导入到 F:\VirtualPCs\Ubuntu-22.04 目录，

wsl --import Ubuntu-22.04 F:\VirtualPCs\Ubuntu-22.04 D:\tmp\Ubuntu-22.04.tar
1

5. 访问 Ubuntu-22.04，

wsl
1

6. 退出 Ubuntu-22.04，

exit
1

7. (可选)关闭 Ubuntu-22.04，

wsl --shutdown
1

2. (可选)配置清华大学软件源

1. 为了加速软件安装速度，配置清华大学软件源，

mv /etc/apt/sources.list /etc/apt/sources.list.bak
1

cat << EOF > /etc/apt/sources.list
# 默认注释了源码镜像以提高 apt update 速度，如有需要可自行取消注释
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-updates main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-updates main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-backports main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-backports main restricted universe multiverse

# deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-security main restricted universe multiverse
# # deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-security main restricted universe multiverse

deb http://security.ubuntu.com/ubuntu/ jammy-security main restricted universe multiverse
# deb-src http://security.ubuntu.com/ubuntu/ jammy-security main restricted universe multiverse

# 预发布软件源，不建议启用
# deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-proposed main restricted universe multiverse
# # deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ jammy-proposed main restricted universe multiverse
EOF
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

3. 系统更新

sudo apt update && sudo apt upgrade
1

4. 安装 Python

1. 安装必要的构建工具，

sudo apt install build-essential checkinstall
sudo apt install libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev
1
2

2. 下载Python 3.10.6 源代码，

cd /tmp
wget https://www.python.org/ftp/python/3.10.6/Python-3.10.6.tar.xz
tar -xvf Python-3.10.6.tar.xz
cd Python-3.10.6/
1
2
3
4

3. 配置和安装Python 3.10，

./configure --enable-optimizations
sudo make altinstall
1
2

altinstall 选项会安装Python 3.10到/usr/bin/python3.10，这样不会对系统默认的Python产生影响。

4. (可选)配置环境变量，

echo 'export PATH="/usr/bin/python3.10:$PATH"' >> ~/.bashrc 
source ~/.bashrc
1
2

5. 配置软链接，

ln -s /usr/local/bin/python3.10 /usr/local/bin/python3
ln -s /usr/local/bin/python3.10 /usr/local/bin/python
ln -s /usr/local/bin/pip3.10 /usr/local/bin/pip3
ln -s /usr/local/bin/pip3.10 /usr/local/bin/pip
1
2
3
4

6. 验证Python 3.10是否安装成功，

python3 --version

--- 输出
Python 3.10.6
---
1
2
3
4
5

pip3 --version

--- 输出
pip 22.2.1 from /usr/local/lib/python3.10/site-packages/pip (python 3.10)
---
1
2
3
4
5

5. 安装 Miniconda3

1. 下载 Conda 安装脚本，

cd /tmp
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
1
2

2. 运行安装脚本，按提示操作，当提示是否初始化 Conda 时,输入 “yes”，

bash Miniconda3-latest-Linux-x86_64.sh

---
Please, press ENTER to continue
>>> yes

Do you accept the license terms? [yes|no]
[no] >>> yes

Miniconda3 will now be installed into this location:
/root/miniconda3

  - Press ENTER to confirm the location
  - Press CTRL-C to abort the installation
  - Or specify a different location below

[/root/miniconda3] >>> ENTER

Do you wish the installer to initialize Miniconda3
by running conda init? [yes|no]
[no] >>> yes
---
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

3. 安装完成后，激活一下 Conda，

sudo su - root
1

4. 更新 Conda 至最新版本，

conda update conda
1

5. 配置用户登录时是否默认激活 Conda 的 base 环境，

如果想默认激活 Conda 的 base 环境，

conda config --set auto_activate_base true
1

如果不想默认激活 Conda 的 base 环境，

conda config --set auto_activate_base false
1

6. 安装 CUDA Toolkit

refer: https://docs.nvidia.com/cuda/wsl-user-guide/index.html#getting-started-with-cuda-on-wsl

refer: https://developer.nvidia.com/cuda-11-8-0-download-archive?target_os=Linux&target_arch=x86_64&Distribution=WSL-Ubuntu&target_version=2.0&target_type=runfile_local

1. 下载 cuda Toolkit 安装文件，

cd /tmp
wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
1
2

2. 安装 cuda Toolkit，

sudo sh cuda_11.8.0_520.61.05_linux.run
1

输入 accept，

┌──────────────────────────────────────────────────────────────────────────────┐
│  End User License Agreement                                                  │
│  --------------------------                                                  │
│        
1
2
3