devbox
码创造者
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

ChatGLM2+Langchain 构建本地知识库——Langchain加载_langchain chatglm2

作者：码创造者 | 2024-07-13 04:16:14

赞

踩

langchain chatglm2

参考视频：ChatGLM+Langchain构建本地知识库，只需6G显存，支持实时上传文档_哔哩哔哩_bilibili

一、启动API服务

在ChatGLM2项目下有个 api.py 文件，先将文件内的模型文件改为自己本地的模型文件，同时将host 改为 127.0.0.1 。

  1. if __name__ == '__main__':
  2.     tokenizer = AutoTokenizer.from_pretrained("/home/ChatGLM2-6B/model//chatglm2-6b", trust_remote_code=True)
  3.     model = AutoModel.from_pretrained("/home/ChatGLM2-6B/model/chatglm2-6b", trust_remote_code=True, device='cuda')
  4.     model.eval()
  5.     uvicorn.run(app, host='127.0.0.1', port=8000, workers=1)

启动 api.py

  1. $ python api.py
  2. Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
  3. Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision.
  4. Explicitly passing a `revision` is encouraged when loading a model with custom code to ensure no malicious code has been contributed in a newer revision.
  5. Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:12<00:00,  1.83s/it]
  6. INFO:     Started server process [340455]
  7. INFO:     Waiting for application startup.
  8. INFO:     Application startup complete.
  9. INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

 

二、检验 API 接入

创建 api_access.py

  1. import requests
  2.  
  3. def chat(prompt,history):
  4.     resp = requests.post(
  5.             url = 'http://127.0.0.1:8000',
  6.             json = {"prompt":prompt, "history":history},
  7.             headers = {"Content-Type":"application/json;charset=utf-8"}
  8.          )
  9.     return resp.json()['response'], resp.json()['history']
  10.  
  11. history = []
  12.  
  13. while True:
  14.     response, history = chat(input("Question:"), history)
  15.     print('Answer:', response)

运行 api_access.py

  1. $ python api_access.py
  2. Question:你好
  3. Answer: 你好
    声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/码创造者/article/detail/817544
    推荐阅读
    相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。