devbox
Monodyee
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

基于modelscope平台搭建百川13b2的对话模型demo_from modelscope import snapshot_download

作者：Monodyee | 2024-03-05 05:02:42

赞

踩

from modelscope import snapshot_download

1、简单粗暴直接上代码

        注释：作者基于下图的环境搭建没什么问题 ，还需要modelscope>=1.9.1   ，gradio直接安装最新的

         

  1. import torch
  2. import gradio as gr
  3. import torch.nn as nn
  4. from modelscope import snapshot_download, Model
  5.  
  6. model_dir = snapshot_download(
  7.     "baichuan-inc/Baichuan2-13B-Chat", revision='v1.0.1')
  8.  
  9.  
  10. model = Model.from_pretrained(
  11.     model_dir, device_map="auto", torch_dtype=torch.float16, trust_remote_code=True)
  12.  
  13.  
  14. def clear_session():
  15.     return []
  16.  
  17.  
  18. def predict(input, history):
  19.     if history is None:
  20.         history = []
  21.     model_input = []
  22.     for chat in history:
  23.         model_input.append({"role": "user", "content": chat[0]})
  24.         model_input.append({"role": "assistant", "content": chat[1]})
  25.     model_input.append({"role": "user", "content": input})
  26.     print(model_input)
  27.     response = model(model_input)["response"]
  28.     history.append((input, response))
  29.     history = history[-20:]
  30.     return '', history
  31.  
  32.  
  33. block = gr.Blocks()
  34.  
  35. with block as demo:
  36.     gr.Markdown("""<h1><center>Baichuan2-13B-Chat</center></h1>
  37.      <center>Baichuan2-13B-Chat为Baichuan2-13B系列模型中对齐后的版本，预训练模型可见Baichuan2-13B-Base</center>
  38.     """)
  39.     chatbot = gr.Chatbot(label='Baichuan2-13B-Chat')
  40.     message = gr.Textbox()
  41.     message.submit(predict,
  42.                    inputs=[message, chatbot],
  43.                    outputs=[message, chatbot])
  44.     with gr.Row():
  45.         clear_history = gr.Button("
    声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/Monodyee/article/detail/189634
    推荐阅读
    相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。