赞
踩
ChatGLM3 是智谱AI和清华大学 KEG 实验室联合发布的对话预训练模型。ChatGLM3-6B 是ChatGLM3 系列中的开源模型,在保留了前两代模型对话流畅、部署门槛低等众多优秀特性的基础上,ChatGLM3-6B 有更强大的基础模型,更完整的功能支持,更全面的开源序列
硬件要求(模型推理):
INT4 : RTX3090*1,显存24GB,内存32GB,系统盘200GB
如果你没有 GPU 硬件的话,也可以在 CPU 上进行推理,但是推理速度会更慢。
模型微调硬件要求更高。一般不建议个人用户环境使用
git clone https://hf-mirror.com/THUDM/chatglm3-6b
手动下载以下几个模型(体验时几个模型不一定需全下载)
下载地址:https://hf-mirror.com/THUDM
模型名称: THUDM/chatglm3-6b
本文统一放在模型存档目录:/u01/workspace/models
git clone https://github.com/THUDM/ChatGLM3.git;
cd ChatGLM3
conda create -n chatglm3 python=3.10 -y
source activate chatglm3
pip config set global.index-url http://mirrors.aliyun.com/pypi/simple
pip config set install.trusted-host mirrors.aliyun.com
pip install -r requirements.txt
streamlim run composite_demo/main.py
第一次运行时可能存在部分依赖包没有完整的下载的情况,根据错误提示,分别安装缺失的依赖库:
上面的错误主要是huggingface_hub 的版本问题。指定版本huggingface_hub==0.21.4, 再次执行即可。
(chatglm3) develop@itserver03:/u01/workspace/ChatGLM3$ pip install huggingface_hub==0.21.4 Looking in indexes: http://mirrors.aliyun.com/pypi/simple Collecting huggingface_hub==0.21.4 Downloading http://mirrors.aliyun.com/pypi/packages/ab/28/d4b691840d73126d4c9845f8a22dad033ac872509b6d3a0d93b456eef424/huggingface_hub-0.21.4-py3-none-any.whl (346 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 346.4/346.4 kB 4.6 MB/s eta 0:00:00 Requirement already satisfied: filelock in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (3.14.0) Requirement already satisfied: fsspec>=2023.5.0 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (2024.3.1) Requirement already satisfied: requests in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (2.31.0) Requirement already satisfied: tqdm>=4.42.1 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (4.66.4) Requirement already satisfied: pyyaml>=5.1 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (6.0.1) Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (4.11.0) Requirement already satisfied: packaging>=20.9 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from huggingface_hub==0.21.4) (23.2) Requirement already satisfied: charset-normalizer<4,>=2 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from requests->huggingface_hub==0.21.4) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from requests->huggingface_hub==0.21.4) (3.7) Requirement already satisfied: urllib3<3,>=1.21.1 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from requests->huggingface_hub==0.21.4) (2.2.1) Requirement already satisfied: certifi>=2017.4.17 in /home/develop/anaconda3/envs/chatglm3/lib/python3.10/site-packages (from requests->huggingface_hub==0.21.4) (2024.2.2) Installing collected packages: huggingface_hub Attempting uninstall: huggingface_hub Found existing installation: huggingface-hub 0.23.0 Uninstalling huggingface-hub-0.23.0: Successfully uninstalled huggingface-hub-0.23.0 Successfully installed huggingface_hub-0.21.4 (chatglm3) develop@itserver03:/u01/workspace/ChatGLM3$ streamlit run composite_demo/main.py
启动时模型加载可以离线下载,也可以自动下载,(根据自身网络情况准备)
如果事先已经准备好模型,需要更改模型家在路径,修改文件composite_demo/client.py,把地址改为自己模型的绝对路径即可:
MODEL_PATH = os.environ.get(‘MODEL_PATH’, ‘THUDM/chatglm3-6b’)
更改为:
MODEL_PATH = os.environ.get(‘MODEL_PATH’, ‘/u01/workspace/chatglm3/models/chatglm3-6b’),
/u01/workspace/chatglm3/models/chatglm3-6b
请据实修改
MODEL_PATH = os.environ.get('MODEL_PATH', '/u01/workspace/chatglm3/models/chatglm3-6b')
PT_PATH = os.environ.get('PT_PATH', None)
PRE_SEQ_LEN = int(os.environ.get("PRE_SEQ_LEN", 128))
TOKENIZER_PATH = os.environ.get("TOKENIZER_PATH", MODEL_PATH)
成功加载后可以打开界面
http://localhost:8501/
启动 API 服务,服务的默认端口是 8000:
python openai_api_demo/api_server.py
启动服务端之后,可以通过curl验证
(chatglm3) develop@itserver03:/u01/workspace/chatglm3/ChatGLM3$ curl -X POST "http://127.0.0.1:8000/v1/chat/completions" -H "Content-Type: application/json" -d "{\"model\": \"chatglm3-6b\", \"messages\": [{\"role\": \"system\", \"content\": \"You are ChatGLM3, a large language model trained by Zhipu.AI. Follow the user's instructions carefully. Respond using markdown.\"}, {\"role\": \"user\", \"content\": \"你好,给我讲一个故事,大概100字\"}], \"stream\": false, \"max_tokens\": 100, \"temperature\": 0.8, \"top_p\": 0.8}"
{"model":"chatglm3-6b","id":"","object":"chat.completion","choices":[{"index":0,"message":{"role":"assistant","content":"从前,有一个美丽的村庄,里面住着一个善良的姑娘。她每天都会去河边给河流唱歌,她的歌声感动了河神,于是河神赐予她一片肥沃的土地。她感激不已,用这片土地种植了各种美丽的花草,还养了很多动物。村子的人们都欣赏她的智慧和勤劳,欢迎她成为村子的领袖。她的生活越来越美好,村子也越来越繁荣。","name":null,"function_call":null},"finish_reason":"stop"}],"created":1715245642,"usage":{"prompt_tokens":54,"total_tokens":141,"completion_tokens":87}}
注意
COPY ChatGLM3/ /app/ChatGLM3/
这行执行需要根据世纪ChatGLM3源码下载存放位置。
FROM pytorch/pytorch:2.2.1-cuda12.1-cudnn8-runtime ARG DEBIAN_FRONTEND=noninteractive WORKDIR /app RUN pip config set global.index-url http://mirrors.aliyun.com/pypi/simple RUN pip config set install.trusted-host mirrors.aliyun.com RUN mkdir -p /u01/workspace/models/ COPY ChatGLM3/ /app/ChatGLM3/ WORKDIR /app/ChatGLM3 RUN pip install --verbose --use-pep517 -r requirements.txt RUN pip install huggingface_hub==0.21.4 EXPOSE 8000 8051 CMD [ "streamlit","run", "composite_demo/main.py" ]
本文采用基础镜像pytorch/pytorch:2.2.1-cuda12.1-cudnn8-runtime
自带的 pip 相关版本与源码中的部分版本冲突(torch),所以,下载原名后需要修改requirements.txt文件:
# basic requirements protobuf>=4.25.3 transformers>=4.39.3 tokenizers>=0.15.0 cpm_kernels>=1.0.11 #torch>=2.1.0 gradio>=4.26.0 sentencepiece>=0.2.0 sentence_transformers>=2.4.0 accelerate>=0.29.2 streamlit>=1.33.0 fastapi>=0.110.0 loguru~=0.7.2 mdtex2html>=1.3.0 latex2mathml>=3.77.0 jupyter_client>=8.6.1 # for openai demo openai>=1.17.1 zhipuai>=2.0.1 pydantic>=2.7.0 sse-starlette>=2.0.0 uvicorn>=0.29.0 timm>=0.9.16 tiktoken>=0.6.0 # for langchain demo langchain>=0.1.16 langchainhub>=0.1.15 arxiv>=2.1.0
docker build -t qingcloudtech/chatglm3:v1.0 .
docker run -it --gpus all -p 8501:8501 -v /u01/workspace/models:/u01/workspace/models qingcloudtech/chatglm3:v1.1
docker run -it --gpus all -p 8000:8000 -v /u01/workspace/models:/u01/workspace/models qingcloudtech/chatglm3:v1.1 python openai_api_demo/api_server.py
【Qinghub Studio 】更适合开发人员的低代码开源开发平台
【QingHub企业级应用统一部署】
【QingHub企业级应用开发管理】
【QingHub** 演示】**
【https://qingplus.cn】
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。