本地搭建【文档助手】大模型版（LangChain+llama+Streamlit）_llama-cpp-python文档

概述

本文的文档助手就是：我们上传一个文档，然后在对话框中输入问题，大模型会把问题的答案返回。

安装步骤

1. 先下载代码到本地

LangChain调用llama模型的示例代码：https://github.com/afaqueumer/DocQA（代码不是本人写的，尊重原创）

git clone https://github.com/afaqueumer/DocQA.git
1

2. 环境安装

双击 setup_env.bat
1

• 如果没反应可能是缺少环境，打开控制台手动执行一下，缺python或者pip的自己根据报错下载一下

如果llama-cpp-python安装报错
（1）需要下载Visual Studio
（2）打开Visual Studio，工具，获取工具和功能

（3）等待下载完，重新运行setup_env.bat

如果还有报错【error C2061: 语法错误:】，那么可能是Visual Studio的版本太低了，我一开始用的是2019版本，后来换成了2022

更新为2022之后重复上面操作

3. 下载一个靠谱的模型
   https://huggingface.co/TheBloke/Llama-2-7B-GGUF
   本文用的是：TheBloke/Llama-2-7B-GGUF中的最精简版
   
4. 进入DocQA，修改app.py

原始

llm = LlamaCpp(model_path="./models/llama-7b.ggmlv3.q4_0.bin")
embeddings = LlamaCppEmbeddings(model_path="models/llama-7b.ggmlv3.q4_0.bin")
1
2

这里改成自己下载的模型地址，比如：llama-2-7b.Q2_K.gguf

llm = LlamaCpp(model_path="../llama.cpp/models/7B/llama-2-7b.Q2_K.gguf")
embeddings = LlamaCppEmbeddings(model_path="../llama.cpp/models/7B/llama-2-7b.Q2_K.gguf")
1
2

5. 运行

双击run_app.bat
1

6. 测试

准备好一个txt文档

As of October this year, there were nearly 2,500 geographical indication products in China
The reporter learned from the State Intellectual Property Office that in recent years, the quantity and quality of China's geographical indication products have risen rapidly. As of October this year, China has approved a total of 2,495 geographical indication products, and approved 7,013 geographical indications to be registered as collective trademarks and certification trademarks. In 2021, the direct output value of GI products exceeded 700 billion yuan.
In recent years, the State Intellectual Property Office has conscientiously implemented the decisions and arrangements of the CPC Central Committee and the State Council, actively and steadily promoted the unified acceptance channels, unified special signs, unified announcements, unified protection and supervision, unified foreign cooperation and other work, and further improved the system of protection, management and application of geographical indications.
In terms of institutional construction, the State Intellectual Property Office issued the "14th Five-Year Plan for the Protection and Use of Geographical Indications", formulated and issued a unified special indication for geographical indications, revised and issued the "Measures for the Protection of Foreign Geographical Indication Products", and launched the legislative work on geographical indications; In the year, a total of 1,416 cases of infringement of geographical indications were investigated and dealt with across the country, involving an amount of 9.28 million yuan and a fine of 13.023 million yuan
1
2
3
4

上传到页面中

没有GPU的痛苦，运行太慢了

注意：别用中文问，这个模型好像不支持中文，换一个支持中文的模型就行了