赞
踩
写在前面:本次部署版本为Qwen-VL-Chat-Int4(卑微),电脑配置:ubuntu22.04;显卡:RTX4090 24G,以下为搬运工记录
三个参考链接:
(前提:已经安装了anaconda)
1、下载代码:
git clone https://github.com/QwenLM/Qwen-VL.git
2、配置环境:python建议使用3.10,本人尝试过3.12会导致failed to build totenizers从而环境安装失败。
- conda create -name qwenvl python=3.10
- cd Qwen-VL
- pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
- ###建议添加清华镜像提高包的安装时间
3、配置Qwen-VL-Chat-Int4额外依赖库:
- pip install optimum -i https://pypi.tuna.tsinghua.edu.cn/simple
- pip install auto-gptq -i https://pypi.tuna.tsinghua.edu.cn/simple
- pip install modelscope -U -i https://pypi.tuna.tsinghua.edu.cn/simple
4、测试Demo:代码test.py内容如下:执行python test.py即可
- from modelscope import (
- snapshot_download, AutoModelForCausalLM, AutoTokenizer, GenerationConfig
- )
- from auto_gptq import AutoGPTQForCausalLM
-
- import torch
- torch.manual_seed(1234)
-
- ##Qwen-VL-Chat-Int4模型权重文件夹路径,可通过第三个链接下载
- model_dir = "./Qwen-VL-Chat-Int4"
-
- # Note: The default behavior now has injection attack prevention off.
- tokenizer = AutoTokenizer.from_pretrained(model_dir, trust_remote_code=True)
-
- # use cuda device
- model = AutoModelForCausalLM.from_pretrained(model_dir, device_map="cuda", trust_remote_code=True,use_safetensors=True).eval()
-
- # 1st dialogue turn
- query = tokenizer.from_list_format([
- {'image': 'https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-VL/assets/demo.jpeg'},
- {'text': '这是什么'},
- ])
- response, history = model.chat(tokenizer, query=query, history=None)
- print(response)
- # 图中是一名年轻女子在沙滩上和她的狗玩耍,狗的品种可能是拉布拉多。她们坐在沙滩上,狗的前腿抬起来,似乎在和人类击掌。两人之间充满了信任和爱。
-
- # 2nd dialogue turn
- response, history = model.chat(tokenizer, '输出"击掌"动作的检测框', history=history)
- print(response)
-
- image = tokenizer.draw_bbox_on_latest_picture(response, history)
- if image:
- image.save('1.jpg')
- else:
- print("no box")
结果显示:
分割线:
以下内容可忽略,对部署没有贡献但是确实遇到的问题记录
遇到报错:ERROR: Could not build wheels for tokenizers, which is required to install pyproject.toml-based projects
解决方法:安装rust:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
直接安装太慢,需要配置安装源
- curl --proto '=https' --tlsv1.2 https://sh.rustup.rs -sSf > rust.sh && chmod +x rust.sh
- ##设置环境变量
- ##export RUSTUP_DIST_SERVER=https://mirrors.ustc.edu.cn/rust-static
- export RUSTUP_DIST_SERVER=https://mirrors.tuna.tsinghua.edu.cn/rustup
- export RUSTUP_UPDATE_ROOT=https://mirrors.ustc.edu.cn/rust-static/rustup
- ./rust.sh
- ##安装成功
安装成功后依旧解决不了问题,最后将python3.12降低到python3.10后得以解决
Copyright © 2003-2013 www.wpsshop.cn 版权所有,并保留所有权利。