devbox
我家自动化
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

开源模型应用落地-chatglm3-6b-集成langchain（十）_128k chain

作者：我家自动化 | 2024-05-31 16:08:23

赞

踩

128k chain

一、前言

    langchain框架调用本地模型，使得用户可以直接提出问题或发送指令，而无需担心具体的步骤或流程。通过LangChain和chatglm3-6b模型的整合，可以更好地处理对话，提供更智能、更准确的响应，从而提高对话系统的性能和用户体验。

二、术语

2.1. ChatGLM3

    是智谱AI和清华大学 KEG 实验室联合发布的对话预训练模型。ChatGLM3-6B 是 ChatGLM3 系列中的开源模型，在保留了前两代模型对话流畅、部署门槛低等众多优秀特性的基础上，ChatGLM3-6B 引入了如下特性：

  1. 更强大的基础模型： ChatGLM3-6B 的基础模型 ChatGLM3-6B-Base 采用了更多样的训练数据、更充分的训练步数和更合理的训练策略。在语义、数学、推理、代码、知识等不同角度的数据集上测评显示，* ChatGLM3-6B-Base 具有在 10B 以下的基础模型中最强的性能*。
  2. 更完整的功能支持： ChatGLM3-6B 采用了全新设计的 Prompt 格式 ，除正常的多轮对话外。同时原生支持工具调用（Function Call）、代码执行（Code Interpreter）和 Agent 任务等复杂场景。
  3. 更全面的开源序列： 除了对话模型 ChatGLM3-6B 外，还开源了基础模型 ChatGLM3-6B-Base 、长文本对话模型 ChatGLM3-6B-32K 和进一步强化了对于长文本理解能力的 ChatGLM3-6B-128K。以上所有权重对学术研究完全开放 ，在填写 问卷 进行登记后亦允许免费商业使用。

2.2.LangChain

    是一个全方位的、基于大语言模型这种预测能力的应用开发工具。LangChain的预构建链功能，就像乐高积木一样，无论你是新手还是经验丰富的开发者，都可以选择适合自己的部分快速构建项目。对于希望进行更深入工作的开发者，LangChain 提供的模块化组件则允许你根据自己的需求定制和创建应用中的功能链条。

    LangChain本质上就是对各种大模型提供的API的套壳，是为了方便我们使用这些 API，搭建起来的一些框架、模块和接口。

    LangChain的主要特性：
        1.可以连接多种数据源，比如网页链接、本地PDF文件、向量数据库等
        2.允许语言模型与其环境交互
        3.封装了Model I/O（输入/输出）、Retrieval（检索器）、Memory（记忆）、Agents（决策和调度）等核心组件
        4.可以使用链的方式组装这些组件，以便最好地完成特定用例。
        5.围绕以上设计原则，LangChain解决了现在开发人工智能应用的一些切实痛点。

三、前提条件 

3.1. 基础环境及前置条件

  1.  操作系统：centos7
  2.  Tesla V100-SXM2-32GB  CUDA Version: 12.2

3.2. 下载chatglm3-6b模型

从huggingface下载：https://huggingface.co/THUDM/chatglm3-6b/tree/main

从魔搭下载：魔搭社区汇聚各领域最先进的机器学习模型，提供模型探索体验、推理、训练、部署和应用的一站式服务。https://www.modelscope.cn/models/ZhipuAI/chatglm3-6b/filesicon-default.png?t=N7T8https://www.modelscope.cn/models/ZhipuAI/chatglm3-6b/files

3.3. 安装虚拟环境

  1. conda create --name langchain python=3.10
  2. conda activate langchain
  3. pip install langchain accelerate

四、技术实现

4.1. 示例一

  1. # -*-  coding = utf-8 -*-
  2.  
  3. from langchain.llms.base import LLM
  4. from langchain import LLMChain, PromptTemplate, ConversationChain
  5. from transformers import AutoTokenizer, AutoModelForCausalLM
  6. from typing import List, Optional
  7.  
  8. modelPath = "/model/chatglm3-6b"
  9.  
  10. class ChatGLM3(LLM):
  11.  
  12.     temperature: float = 0.45
  13.     top_p = 0.8
  14.     repetition_penalty = 1.1
  15.     max_token: int = 8192
  16.     do_sample: bool = True
  17.     tokenizer: object = None
  18.     model: object = None
  19.     history: List = []
  20.  
  21.     def __init__(self):
  22.         super().__init__()
  23.  
  24.     @property
  25.     def _llm_type(self) -> str:
  26.         return "ChatGLM3"
  27.  
  28.     def load_model(self, model_name_or_path=None):
  29.         self.tokenizer = AutoTokenizer.from_pretrained(
  30.             model_name_or_path,
  31.             trust_remote_code=True
  32.         )
  33.         self.model = AutoModelForCausalLM.from_pretrained(
  34.             model_name_or_path, trust_remote_code=True, device_map="auto").cuda()
  35.  
  36.  
  37.     def _call(self, prompt: str, history: List = [], stop: Optional[List[str]] = ["<|user|>"]):
  38.  
  39.         response, self.history = self.model.chat(
  40.             self.tokenizer,
  41.             prompt,
  42.             history=self.history,
  43.             do_sample=self.do_sample,
  44.             max_length=self.max_token,
  45.             temperature=self.temperature,
  46.             top_p = self.top_p,
  47.             repetition_penalty = self.repetition_penalty
  48.         )
  49.         history.append((prompt, response))
  50.         return response
  51.  
  52.  
  53. if __name__ == "__main__":
  54.     llm = ChatGLM3()
  55.     llm.load_model(modelPath)
  56.  
  57.     template = """
  58. 问题: {question}
  59. """
  60.     prompt = PromptTemplate.from_template(template)
  61.  
  62.     llm_chain = LLMChain(prompt=prompt, llm=llm)
  63.  
  64.     question = "广州有什么特色景点?"
  65.     print(llm_chain.run(question))

调用结果：

4.2. 示例二

  1. # -*-  coding = utf-8 -*-
  2.  
  3. from langchain.llms.base import LLM
  4. from langchain import LLMChain, ConversationChain
  5. from langchain.prompts import SystemMessagePromptTemplate, HumanMessagePromptTemplate, ChatPromptTemplate
  6. from transformers import AutoTokenizer, AutoModelForCausalLM
  7. from typing import List, Optional
  8.  
  9. modelPath = "/model/chatglm3-6b"
  10.  
  11. class ChatGLM3(LLM):
  12.  
  13.     temperature: float = 0.45
  14.     top_p = 0.8
  15.     repetition_penalty = 1.1
  16.     max_token: int = 8192
  17.     do_sample: bool = True
  18.     tokenizer: object = None
  19.     model: object = None
  20.     history: List = []
  21.  
  22.     def __init__(self):
  23.         super().__init__()
  24.  
  25.     @property
  26.     def _llm_type(self) -> str:
  27.         return "ChatGLM3"
  28.  
  29.     def load_model(self, model_name_or_path=None):
  30.         self.tokenizer = AutoTokenizer.from_pretrained(
  31.             model_name_or_path,
  32.             trust_remote_code=True
  33.         )
  34.         self.model = AutoModelForCausalLM.from_pretrained(
  35.             model_name_or_path, trust_remote_code=True, device_map="auto").cuda()
  36.  
  37.  
  38.     def _call(self, prompt: str, history: List = [], stop: Optional[List[str]] = ["<|user|>"]):
  39.         # print(f'prompt: {prompt}')
  40.         # print(f'history: {history}')
  41.  
  42.         response, self.history = self.model.chat(
  43.             self.tokenizer,
  44.             prompt,
  45.             history=self.history,
  46.             do_sample=self.do_sample,
  47.             max_length=self.max_token,
  48.             temperature=self.temperature,
  49.             top_p = self.top_p,
  50.             repetition_penalty = self.repetition_penalty
  51.         )
  52.         history.append((prompt, response))
  53.         return response
  54.  
  55.  
  56. if __name__ == "__main__":
  57.     llm = ChatGLM3()
  58.     llm.load_model(modelPath)
  59.  
  60.  
  61.     template = "你是一个数学专家，很擅长解决复杂的逻辑推理问题。"
  62.     system_message_prompt = SystemMessagePromptTemplate.from_template(template)
  63.     human_template = "问题: {question}"
  64.     human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)
  65.     prompt_template = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])
  66.  
  67.  
  68.     llm_chain = LLMChain(prompt=prompt_template, llm=llm)
  69.     print(llm_chain.run(question="若一个三角形的两条边长度分别为3和4，且夹角为直角，最后一条边的长度是多少？"))

调用结果：

五、附带说明

5.1. 示例中ChatGLM3可以扩展，实现更复杂的功能

参见官方示例：

ChatGLM3.py

  1. import ast
  2. import json
  3. from langchain.llms.base import LLM
  4. from transformers import AutoTokenizer, AutoModel, AutoConfig
  5. from typing import List, Optional
  6.  
  7.  
  8. class ChatGLM3(LLM):
  9.     max_token: int = 8192
  10.     do_sample: bool = True
  11.     temperature: float = 0.8
  12.     top_p = 0.8
  13.     tokenizer: object = None
  14.     model: object = None
  15.     history: List = []
  16.     has_search: bool = False
  17.  
  18.     def __init__(self):
  19.         super().__init__()
  20.  
  21.     @property
  22.     def _llm_type(self) -> str:
  23.         return "ChatGLM3"
  24.  
  25.     def load_model(self, model_name_or_path=None):
  26.         model_config = AutoConfig.from_pretrained(
  27.             model_name_or_path,
  28.             trust_remote_code=True
  29.         )
  30.         self.tokenizer = AutoTokenizer.from_pretrained(
  31.             model_name_or_path,
  32.             trust_remote_code=True
  33.         )
  34.         self.model = AutoModel.from_pretrained(
  35.             model_name_or_path, config=model_config, trust_remote_code=True, device_map="auto").eval()
  36.  
  37.     def _tool_history(self, prompt: str):
  38.         ans = []
  39.  
  40.         tool_prompts = prompt.split(
  41.             "You have access to the following tools:\n\n")[1].split("\n\nUse a json blob")[0].split("\n")
  42.         tools_json = []
  43.  
  44.         for tool_desc in tool_prompts:
  45.             name = tool_desc.split(":")[0]
  46.             description = tool_desc.split(", args:")[0].split(":")[1].strip()
  47.             parameters_str = tool_desc.split("args:")[1].strip()
  48.             parameters_dict = ast.literal_eval(parameters_str)
  49.             params_cleaned = {}
  50.             for param, details in parameters_dict.items():
  51.                 params_cleaned[param] = {'description': details['description'], 'type': details['type']}
  52.  
  53.             tools_json.append({
  54.                 "name": name,
  55.                 "description": description,
  56.                 "parameters": params_cleaned
  57.             })
  58.  
  59.         ans.append({
  60.             "role": "system",
  61.             "content": "Answer the following questions as best as you can. You have access to the following tools:",
  62.             "tools": tools_json
  63.         })
  64.  
  65.         dialog_parts = prompt.split("Human: ")
  66.         for part in dialog_parts[1:]:
  67.             if "\nAI: " in part:
  68.                 user_input, ai_response = part.split("\nAI: ")
  69.                 ai_response = ai_response.split("\n")[0]
  70.             else:
  71.                 user_input = part
  72.                 ai_response = None
  73.  
  74.             ans.append({"role": "user", "content": user_input.strip()})
  75.             if ai_response:
  76.                 ans.append({"role": "assistant", "content": ai_response.strip()})
  77.  
  78.         query = dialog_parts[-1].split("\n")[0]
  79.         return ans, query
  80.  
  81.     def _extract_observation(self, prompt: str):
  82.         return_json = prompt.split("Observation: ")[-1].split("\nThought:")[0]
  83.         self.history.append({
  84.             "role": "observation",
  85.             "content": return_json
  86.         })
  87.         return
  88.  
  89.     def _extract_tool(self):
  90.         if len(self.history[-1]["metadata"]) > 0:
  91.             metadata = self.history[-1]["metadata"]
  92.             content = self.history[-1]["content"]
  93.  
  94.             lines = content.split('\n')
  95.             for line in lines:
  96.                 if 'tool_call(' in line and ')' in line and self.has_search is False:
  97.                     # 获取括号内的字符串
  98.                     params_str = line.split('tool_call(')[-1].split(')')[0]
  99.  
  100.                     # 解析参数对
  101.                     params_pairs = [param.split("=") for param in params_str.split(",") if "=" in param]
  102.                     params = {pair[0].strip(): pair[1].strip().strip("'\"") for pair in params_pairs}
  103.                     action_json = {
  104.                         "action": metadata,
  105.                         "action_input": params
  106.                     }
  107.                     self.has_search = True
  108.                     print("*****Action*****")
  109.                     print(action_json)
  110.                     print("*****Answer*****")
  111.                     return f"""
  112. Action: 
  113. ```
  114. {json.dumps(action_json, ensure_ascii=False)}
  115. ```"""
  116.         final_answer_json = {
  117.             "action": "Final Answer",
  118.             "action_input": self.history[-1]["content"]
  119.         }
  120.         self.has_search = False
  121.         return f"""
  122. Action: 
  123. ```
  124. {json.dumps(final_answer_json, ensure_ascii=False)}
  125. ```"""
  126.  
  127.     def _call(self, prompt: str, history: List = [], stop: Optional[List[str]] = ["<|user|>"]):
  128.         if not self.has_search:
  129.             self.history, query = self._tool_history(prompt)
  130.         else:
  131.             self._extract_observation(prompt)
  132.             query = ""
  133.         _, self.history = self.model.chat(
  134.             self.tokenizer,
  135.             query,
  136.             history=self.history,
  137.             do_sample=self.do_sample,
  138.             max_length=self.max_token,
  139.             temperature=self.temperature,
  140.         )
  141.         response = self._extract_tool()
  142.         history.append((prompt, response))
  143.         return response

main.py

  1. """
  2. This script demonstrates the use of the LangChain's StructuredChatAgent and AgentExecutor alongside various tools
  4. The script utilizes the ChatGLM3 model, a large language model for understanding and generating human-like text.
  5. The model is loaded from a specified path and integrated into the chat agent.
  7. Tools:
  8. - Calculator: Performs arithmetic calculations.
  9. - Weather: Provides weather-related information based on input queries.
  10. - DistanceConverter: Converts distances between meters, kilometers, and feet.
  12. The agent operates in three modes:
  13. 1. Single Parameter without History: Uses Calculator to perform simple arithmetic.
  14. 2. Single Parameter with History: Uses Weather tool to answer queries about temperature, considering the
  15. conversation history.
  16. 3. Multiple Parameters without History: Uses DistanceConverter to convert distances between specified units.
  17. 4. Single use Langchain Tool: Uses Arxiv tool to search for scientific articles.
  19. Note:
  20. The model calling tool fails, which may cause some errors or inability to execute. Try to reduce the temperature
  21. parameters of the model, or reduce the number of tools, especially the third function.
  22. The success rate of multi-parameter calling is low. The following errors may occur:
  24. Required fields [type=missing, input_value={'distance': '30', 'unit': 'm', 'to': 'km'}, input_type=dict]
  26. The model illusion in this case generates parameters that do not meet the requirements.
  27. The top_p and temperature parameters of the model should be adjusted to better solve such problems.
  29. Success example:
  31. *****Action*****
  33. {
  34.     'action': 'weather',
  35.     'action_input': {
  36.         'location': '厦门'
  37.         }
  38. }
  40. *****Answer*****
  42. {
  43.     'input': '厦门比北京热吗?',
  44.     'chat_history': [HumanMessage(content='北京温度多少度'), AIMessage(content='北京现在12度')],
  45.     'output': '根据最新的天气数据，厦门今天的气温为18度，天气晴朗。而北京今天的气温为12度。所以，厦门比北京热。'
  46. }
  48. ****************
  50. """
  51.  
  52. import os
  53.  
  54. from langchain import hub
  55. from langchain.agents import AgentExecutor, create_structured_chat_agent, load_tools
  56. from langchain_core.messages import AIMessage, HumanMessage
  57.  
  58. from ChatGLM3 import ChatGLM3
  59. from tools.Calculator import Calculator
  60. from tools.Weather import Weather
  61. from tools.DistanceConversion import DistanceConverter
  62.  
  63. MODEL_PATH = os.environ.get('MODEL_PATH', 'THUDM/chatglm3-6b')
  64.  
  65. if __name__ == "__main__":
  66.     llm = ChatGLM3()
  67.     llm.load_model(MODEL_PATH)
  68.     prompt = hub.pull("hwchase17/structured-chat-agent")
  69.  
  70.     # for single parameter without history
  71.  
  72.     tools = [Calculator()]
  73.     agent = create_structured_chat_agent(llm=llm, tools=tools, prompt=prompt)
  74.     agent_executor = AgentExecutor(agent=agent, tools=tools)
  75.     ans = agent_executor.invoke({"input": "34 * 34"})
  76.     print(ans)
  77.  
  78.     # for singe parameter with history
  79.  
  80.     tools = [Weather()]
  81.     agent = create_structured_chat_agent(llm=llm, tools=tools, prompt=prompt)
  82.     agent_executor = AgentExecutor(agent=agent, tools=tools)
  83.     ans = agent_executor.invoke(
  84.         {
  85.             "input": "厦门比北京热吗?",
  86.             "chat_history": [
  87.                 HumanMessage(content="北京温度多少度"),
  88.                 AIMessage(content="北京现在12度"),
  89.             ],
  90.         }
  91.     )
  92.     print(ans)
  93.  
  94.     # for multiple parameters without history
  95.  
  96.     tools = [DistanceConverter()]
  97.     agent = create_structured_chat_agent(llm=llm, tools=tools, prompt=prompt)
  98.     agent_executor = AgentExecutor(agent=agent, tools=tools)
  99.     ans = agent_executor.invoke({"input": "how many meters in 30 km?"})
  100.  
  101.     print(ans)
  102.  
  103.     # for using langchain tools
  104.  
  105.     tools = load_tools(["arxiv"], llm=llm)
  106.     agent = create_structured_chat_agent(llm=llm, tools=tools, prompt=prompt)
  107.     agent_executor = AgentExecutor(agent=agent, tools=tools)
  108.     ans = agent_executor.invoke({"input": "Describe the paper about GLM 130B"})
  109.  
  110.     print(ans)

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/我家自动化/article/detail/653136
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号