Llama Factory 笔记_valueerror: cannot merge adapters to a quantized m

作者：IT小白 | 2024-07-24 11:19:19

踩

valueerror: cannot merge adapters to a quantized model.

本地环境：cuda 11.7 torch2.1.0

项目文件结构：

1. 项目文件结构：

如果利用Llama Factory 进行微调主要会用到 LLama-Factory/src 中的文件

2. src 下的目录结构

本地推理的demo

通过api.py 进行 LLaMa-Factory 项目文件下运行，会有一个 web的demo

(可能需要修改 gradio 下面一个包的权限，创建一个公共的端口就可以)

CUDA_VISIBLE_DEVICES=1  python src/api.py     --model_name_or_path  LLama/Llama3-8B-Chinese-Chat     --template llama3

我运行之后打不开网址所以根据之前的为了简单起见还是用 cli_demo.py 放在 src 路径下


from llamafactory.chat import ChatModel
from llamafactory.extras.misc import torch_gc
 
try:
    import platform
 
    if platform.system() != "Windows":
        import readline  # noqa: F401
except ImportError:
    print("Install `readline` for a better experience.")
 
 
def main():
    chat_model = ChatModel()
    messages = []
    print("Welcome to the CLI application, use `clear` to remove the history, use `exit` to exit the application.")
 
    while True:
        try:
            query = input("\nUser: ")
        except UnicodeDecodeError:
            print("Detected decoding error at the inputs, please set the terminal encoding to utf-8.")
            continue
        except Exception:
            raise
 
        if query.strip() == "exit":
            break
 
        if query.strip() == "clear":
            messages = []
            torch_gc()
            print("History has been removed.")
            continue
 
        messages.append({"role": "user", "content": query})
        print("Assistant: ", end="", flush=True)
 
        response = ""
        for new_text in chat_model.stream_chat(messages):
            print(new_text, end="", flush=True)
            response += new_text
        print()
        messages.append({"role": "assistant", "content": response})
 
 
if __name__ == "__main__":
    main()

CUDA_VISIBLE_DEVICES=0  python src/cli_demo.py     --model_name_or_path  自己模型地址    --template 和模型有关（看github 的 readme）

遇到的问题：如果torch的版本低会有一个 BFloat16 的问题(开始是 2.0.1 报错了)

升级成 2.1.0 就好了

pytorch 官网 2.1.0 应该最低是cuda11.8 的直接升级成这个就行 conda install 速度会快一些

可以在命令行进行展示：效果如下：

============= 以上是 2024.05.29 的最新 LLaMa Factory 版本 =====================

本地微调：

再进行微调的时，主要就是运行train.py 这个文件，但是需要指定一些参数比如模型路径数据集微调方式等

train.py 内容


from llamafactory.train.tuner import run_exp
 
def main():
    run_exp()
 
def _mp_fn(index):
    # For xla_spawn (TPUs)
    run_exp()
 
if __name__ == "__main__":
    main()

可以看到 train.py 就是用到了 llamafactory.train.tuner ，所以进一步看一下 llamafactory 文件的目录结构

llamafactory/train 的结构：

tuner.py 内容如下：python 相对导入：python 相对导入-CSDN博客


from typing import TYPE_CHECKING, Any, Dict, List, Optional
 
import torch
from transformers import PreTrainedModel
 
from ..data import get_template_and_fix_tokenizer
from ..extras.callbacks import LogCallback
from ..extras.logging import get_logger
from ..hparams import get_infer_args, get_train_args
from ..model import load_model, load_tokenizer
from .dpo import run_dpo
from .kto import run_kto
from .ppo import run_ppo
from .pt import run_pt
from .rm import run_rm
from .sft import run_sft
 
 
if TYPE_CHECKING:
    from transformers import TrainerCallback
 
 
logger = get_logger(__name__)
 
 
def run_exp(args: Optional[Dict[str, Any]] = None, callbacks: List["TrainerCallback"] = []) -> None:
    model_args, data_args, training_args, finetuning_args, generating_args = get_train_args(args)
    callbacks.append(LogCallback(training_args.output_dir))
 
    if finetuning_args.stage == "pt":
        run_pt(model_args, data_args, training_args, finetuning_args, callbacks)
    elif finetuning_args.stage == "sft":
        run_sft(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
    elif finetuning_args.stage == "rm":
        run_rm(model_args, data_args, training_args, finetuning_args, callbacks)
    elif finetuning_args.stage == "ppo":
        run_ppo(model_args, data_args, training_args, finetuning_args, generating_args, callbacks)
    elif finetuning_args.stage == "dpo":
        run_dpo(model_args, data_args, training_args, finetuning_args, callbacks)
    elif finetuning_args.stage == "kto":
        run_kto(model_args, data_args, training_args, finetuning_args, callbacks)
    else:
        raise ValueError("Unknown task.")
 
 
def export_model(args: Optional[Dict[str, Any]] = None) -> None:
    model_args, data_args, finetuning_args, _ = get_infer_args(args)
 
    if model_args.export_dir is None:
        raise ValueError("Please specify `export_dir` to save model.")
 
    if model_args.adapter_name_or_path is not None and model_args.export_quantization_bit is not None:
        raise ValueError("Please merge adapters before quantizing the model.")
 
    tokenizer_module = load_tokenizer(model_args)
    tokenizer = tokenizer_module["tokenizer"]
    processor = tokenizer_module["processor"]
    get_template_and_fix_tokenizer(tokenizer, data_args.template)
    model = load_model(tokenizer, model_args, finetuning_args)  # must after fixing tokenizer to resize vocab
 
    if getattr(model, "quantization_method", None) and model_args.adapter_name_or_path is not None:
        raise ValueError("Cannot merge adapters to a quantized model.")
 
    if not isinstance(model, PreTrainedModel):
        raise ValueError("The model is not a `PreTrainedModel`, export aborted.")
 
    if getattr(model, "quantization_method", None) is None:  # cannot convert dtype of a quantized model
        output_dtype = getattr(model.config, "torch_dtype", torch.float16)
        setattr(model.config, "torch_dtype", output_dtype)
        model = model.to(output_dtype)
    else:
        setattr(model.config, "torch_dtype", torch.float16)
 
    model.save_pretrained(
        save_directory=model_args.export_dir,
        max_shard_size="{}GB".format(model_args.export_size),
        safe_serialization=(not model_args.export_legacy_format),
    )
    if model_args.export_hub_model_id is not None:
        model.push_to_hub(
            model_args.export_hub_model_id,
            token=model_args.hf_hub_token,
            max_shard_size="{}GB".format(model_args.export_size),
            safe_serialization=(not model_args.export_legacy_format),
        )
 
    try:
        tokenizer.padding_side = "left"  # restore padding side
        tokenizer.init_kwargs["padding_side"] = "left"
        tokenizer.save_pretrained(model_args.export_dir)
        if model_args.export_hub_model_id is not None:
            tokenizer.push_to_hub(model_args.export_hub_model_id, token=model_args.hf_hub_token)
 
        if model_args.visual_inputs and processor is not None:
            getattr(processor, "image_processor").save_pretrained(model_args.export_dir)
            if model_args.export_hub_model_id is not None:
                getattr(processor, "image_processor").push_to_hub(
                    model_args.export_hub_model_id, token=model_args.hf_hub_token
                )
 
    except Exception:
        logger.warning("Cannot save tokenizer, please copy the files manually.")

可以看到包含两个函数:

1. run_exp() 根据传入参数的不同选择不同的方式

2. export_model: 将原来的模型和微调之后的checkpoint 进行合并

到这里就基本上完成了流程上的梳理具体的微调方法需要到每个函数内部自行查看

======================= 以上 2024/05/27 ========================

怎么finetuning起来？

写一个脚本 train.sh ，放在 llama-factory 根目录下：终端运行 bash train.sh 即可


CUDA_VISIBLE_DEVICES=0 python src/train.py \
    --stage sft \
    --do_train True \
    --model_name_or_path 自己模型的路径\
    --finetuning_type lora \
    --template default \
    --flash_attn auto \
    --dataset_dir data \
    --dataset 自己的数据集\
    --cutoff_len 1024 \
    --learning_rate 5e-05 \
    --num_train_epochs 1.0 \
    --max_samples 100000 \
    --per_device_train_batch_size 2 \
    --gradient_accumulation_steps 8 \
    --lr_scheduler_type cosine \
    --max_grad_norm 1.0 \
    --logging_steps 5 \
    --save_steps 100 \
    --warmup_steps 0 \
    --optim adamw_torch \
    --report_to none \
    --output_dir 模型微调完成之后adapter的输出位置 \
    --fp16 True \
    --lora_rank 8 \
    --lora_alpha 16 \
    --lora_dropout 0 \
    --lora_target q_proj,v_proj \
    --plot_loss True

具体的参数 batch_size ，lora_rank 需自行确定

推理：


CUDA_VISIBLE_DEVICES=0 python src/cli_demo.py
--model_name_or_path  模型地址
--adapter_name_or_path  训练出来的适配器的位置
--template  提示词模版和模型相关

即可成功！

注：暂时没用 vllm 框架，用的话可能问题较多

本文内容由网友自发贡献，转载请注明出处：【wpsshop博客】