Llama 3 基于知识库应用实践（一）_llama3-8b 中文 知识库

一、概述

Llama 3 是Meta最新推出的开源大语言模型，其8B和13B参数的模型的性能与之前的Llama 2相比实现了质的飞跃。以下是官方给出的模型性能评测对比结果（引自：https://ai.meta.com/blog/meta-llama-3/），如Llama 3 的8B模型在这个级别上获得了新的SOTA：

在架构方面，Llama 3 采用了一个相对标准的decoder-only的transformer架构，相较于Llama 2，Llama 3做了以下几个方面的改进：

       -Llama 3 使用了基于128K tokens词库的tokenizer，能更有效地对语言进行编码

       -采用分组查询注意力机制（GQA）来改进模型推理效率

       -基于8192 tokens的上下文长度来训练模型，并且使用掩码（mask）来确保自注意力不跨越文档边界

二、Llama 3 模型加载

Ollama是一个免费开源的应用，通过它可以方便地在本地以相对较低的硬件需求来运行各种大语言模型，它也提供了package manager来让用户以命令行的方式快速有效地下载和运行LLMs。

首先运行以下命令加载Llama 3：

这里加载的模型是Meta-Llama-3-8B-Instruct：

Ollama加载Llama 3完成后打印这行信息，表示可以正常调用模型了：

[GIN] 2024/04/28 - 02:53:44 | 200 | 13.435240965s |       127.0.0.1 | POST     "/api/generate"

三、Llama 3模型调用测试

参照Ollama提供的API，执行以下命令：

curl -X POST http://localhost:11434/api/generate -d '{"model": "llama3","prompt":"Why is the sky blue?"}'

打印信息如下：

可以看出以上命令执行的是流式处理，即按行依次输出Llama 3 针对输入prompt生成的每个单词，也可以改用下面的命令来一次性输出Llama 3生成的response：

curl -X POST http://localhost:11434/api/generate -d '{"model": "llama3","prompt":"Why is the sky blue?", "stream":false}'

生成的response如下：

{"model":"llama3","created_at":"2024-04-28T03:12:47.652533109Z","response":"What a great question!\n\nThe short answer: The sky appears blue because of the way that light from the sun interacts with the Earth's atmosphere.\n\nHere's a more detailed explanation:\n\n1. **Sunlight**: When the sun emits light, it contains all the colors of the visible spectrum (red, orange, yellow, green, blue, indigo, and violet).\n2. **Atmosphere**: The Earth's atmosphere is made up of different gases, including nitrogen (N2) and oxygen (O2). These gases scatter shorter wavelengths of light more than longer wavelengths.\n3. **Scattering**: When sunlight enters the Earth's atmosphere, it encounters these gas molecules and scatters in all directions. The shorter blue wavelengths are scattered more than the longer red wavelengths because they have a higher energy and interact more with the smaller gas molecules.\n4. **Blue dominance**: As a result of this scattering, the blue light is distributed throughout the sky, making it appear blue to our eyes. This is known as Rayleigh scattering, named after the British physicist Lord Rayleigh, who first described the phenomenon in the late 19th century.\n\n**Other factors that influence the color of the sky:**\n\n1. **Dust and water vapor**: Tiny particles in the air, like dust, smoke, and water vapor, can also scatter light and change its apparent color.\n2. **Clouds**: Clouds can reflect or absorb certain wavelengths of light, making the sky appear whiter, grayer, or even darker.\n3. **Time of day**: The angle of the sun in relation to the observer's position on Earth affects the color of the sky. During sunrise and sunset, the sun's rays have to travel longer distances through the atmosphere, scattering blue light more and making the sky appear more red or orange.\n\nSo, to summarize: the sky appears blue because of the combined effects of sunlight scattering by the Earth's atmosphere and other factors that influence the color of the sky."

下面测试一下当使用中文提问时，Llama 3如何给出回答，输入命令如下：

curl -X POST http://localhost:11434/api/generate -d '{"model": "llama3","prompt":"为何天空是蓝色的？", "stream":false}'

生成的response如下：

{"model":"llama3","created_at":"2024-04-28T04:32:20.650993108Z","response":"A question that has puzzled humans for centuries!