【xinference】（6）：在autodl上，使用xinference部署yi-vl-chat和qwen-vl-chat模型，可以使用openai-client调用成功，进行对比测试，各有特色_qwen-vl 和 yi-vl区别

1，视频地址

https://www.bilibili.com/video/BV19Z421z7cv/

2，关于大模型 vision能力

大模型的vision能力，即视觉能力，主要指的是模型在处理图像和视频数据方面的能力。这种能力主要通过计算机视觉技术实现，其中包括了对图像内容的理解、分析和处理。大模型的vision能力可以广泛应用于多个领域。

图像识别：大模型可以学习到大量的图像特征，从而实现高精度的图像识别。例如，通过训练一个大模型，可以使其能够识别出图像中的物体、场景和人物等。

3，启动Xinference服务

https://gitee.com/fly-llm/xinference-run-llm

2024-02-06 19:55:36,181 xinference.core.supervisor 5029 INFO     Xinference supervisor 0.0.0.0:27232 started
2024-02-06 19:55:36,273 xinference.core.worker 5029 INFO     Starting metrics export server at 0.0.0.0:None
2024-02-06 19:55:36,276 xinference.core.worker 5029 INFO     Checking metrics export server...
2024-02-06 19:55:39,005 xinference.core.worker 5029 INFO     Metrics server is started at: http://0.0.0.0:42785
2024-02-06 19:55:39,007 xinference.core.worker 5029 INFO     Xinference worker 0.0.0.0:27232 started
2024-02-06 19:55:39,008 xinference.core.worker 5029 INFO     Purge cache directory: /root/autodl-tmp/cache
2024-02-06 19:55:44,909 xinference.api.restful_api 5009 INFO     Starting Xinference at endpoint: http://0.0.0.0:9997
1
2
3
4
5
6
7

测试图片识别脚本：

import openai

client = openai.Client(
    api_key="cannot be empty",
    base_url=f"http://127.0.0.1:9997/v1"
)
response = client.chat.completions.create(
    model="qwen-vl-chat", # qwen-vl-chat 或者 yi-vl-chat
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "这个图片是什么？"},
                {
                    "type": "image_url",
                    "image_url": {
                        "url": "https://p3.dcarimg.com/img/motor-mis-img/2465f0c78280efddcc305991bd1f2ea2~2508x0.jpg",
                    },
                },
            ],
        }
    ],
)
print(response.choices[0])
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

图片为汽车：

4，部署yi-vl-chat方法

https://www.modelscope.cn/models/01ai/Yi-VL-6B/summary

先启动 服务，然后执行：

xinference launch --model-name yi-vl-chat --size-in-billions 6 --model-format pytorch
Model uid: yi-vl-chat
root@autodl-container-200a43b416-b4affc4b:~/autodl-tmp/modelscope# nvidia-smi 
Tue Feb  6 20:08:48 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.146.02             Driver Version: 535.146.02   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3080        On  | 00000000:8A:00.0 Off |                  N/A |
|  0%   21C    P8              22W / 320W |  13725MiB / 20480MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
+---------------------------------------------------------------------------------------+
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

测试：

python3 test-vl-chat.py 
Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='该图片显示一辆银色的汽车在建筑物前面,背景有红色的“M”标志。', role='assistant', function_call=None, tool_calls=None))
root@autodl-container-200a43b416-b4affc4b:~/xinference-run-llm# python3 test-vl-chat.py 
Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='该图片是一辆黑色的吉普车在停车场停泊,车前有黄色牌照。', role='assistant', function_call=None, tool_calls=None))
root@autodl-container-200a43b416-b4affc4b:~/xinference-run-llm# python3 test-vl-chat.py 
Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='这个图片是一辆黑色的吉普车在停车场里,停在建筑物前面。', role='assistant', function_call=None, tool_calls=None))
1
2
3
4
5
6

5，部署qwen-vl-chat方法

https://www.modelscope.cn/models/qwen/Qwen-VL-Chat/summary

先启动 服务，然后执行：

root@autodl-container-200a43b416-b4affc4b:~/autodl-tmp/modelscope# xinference launch --model-name qwen-vl-chat --size-in-billions 7 --model-format pytorch
Model uid: qwen-vl-chat
root@autodl-container-200a43b416-b4affc4b:~/autodl-tmp/modelscope# 
root@autodl-container-200a43b416-b4affc4b:~/autodl-tmp/modelscope# 
root@autodl-container-200a43b416-b4affc4b:~/autodl-tmp/modelscope# nvidia-smi 
Tue Feb  6 19:52:25 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.146.02             Driver Version: 535.146.02   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3080        On  | 00000000:8A:00.0 Off |                  N/A |
| 50%   30C    P2              90W / 320W |  18715MiB / 20480MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+
                                                                                         
+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
+---------------------------------------------------------------------------------------+
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

测试：

root@autodl-container-200a43b416-b4affc4b:~/xinference-run-llm#  python3 test-vl-chat.py 
Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='这个图片是一张静态拍摄的图片，展示了一辆灰色的比亚迪秦Pro DM。这辆车停在一个停车场中，周围还有其他车辆。', role='assistant', function_call=None, tool_calls=None))
root@autodl-container-200a43b416-b4affc4b:~/xinference-run-llm# python3 test-vl-chat.py 
Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='这个图片是一张城市街头的静态照片，展示了一辆黑色北京BJ80 SUV停在商业街前。', role='assistant', function_call=None, tool_calls=None))
root@autodl-container-200a43b416-b4affc4b:~/xinference-run-llm# python3 test-vl-chat.py 
Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='这个图片是一张静态照片，拍摄于停车场中的一辆白色北京BJ40。', role='assistant', function_call=None, tool_calls=None))
root@autodl-container-200a43b416-b4affc4b:~/xinference-run-llm# python3 test-vl-chat.py 
Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content='该图片展示了一辆白色的吉普车在停车场,车前有黄色的牌照。', role='assistant', function_call=None, tool_calls=None))
1
2
3
4
5
6
7
8

6，总结

使用xinference部署yi-vl-chat和qwen-vl-chat模型
都可以运行成功，而且各有特色。qwen可以识别具体汽车品牌，但是有些新车是错误的。
yi识别比较保守。