devbox
小丑西瓜9
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

预训练模型transformers综合总结（一）_transformers.js

作者：小丑西瓜9 | 2024-02-22 10:39:17

赞

踩

transformers.js

这是我对transformers库查看了原始文档后，进行的学习总结。

第一部分是将如何调用加载本地模型，使用模型，修改模型，保存模型

之后还会更新如何使用自定义的数据集训练以及对模型进行微调，感觉这样这个库基本就能玩熟了。

# 加载本地模型须知

* 1.使用transformers库加载预训练模型，99%的时间都是用于模型的下载。
为此，我直接从清华大学软件（"https://mirrors.tuna.tsinghua.edu.cn/hugging-face-models/"）把模型放在了我的本地目录地址："H:\\code\\Model\\"下，这里可以进行修改。
* 2.下载的模型通常会是"模型名称-"+"config.json"的格式例如(bert-base-cased-finetuned-mrpc-config.json)，但如果使用transformers库加载本地模型，需要的是模型路径中是config.json、vocab.txt、pytorch_model.bin、tf_model.h5、tokenizer.json等形式，为此在加载前，需要将把文件前面的模型名称，才能加载成功

我自己写的处理代码如下：

  1. #coding=utf-8
  2. import os
  3. import os.path
  4. # 模型存放路径
  5. rootdir = r"H:\code\Model\bert-large-uncased-whole-word-masking-finetuned-squad"# 指明被遍历的文件夹
  6.  
  7. for parent,dirnames,filenames in os.walk(rootdir):#三个参数：分别返回1.父目录 2.所有文件夹名字（不含路径） 3.所有文件名字
  8.     for filename in filenames:#文件名
  9. #         nameList=filename.split('.')
  10. #         print(nameList)
  11.         print(filename)
  12. #         filenew=nameList[0]+'.jpg'
  13. #         print(filenew)
  14.         #模型的名称
  15.         newName=filename.replace('bert-large-uncased-whole-word-masking-finetuned-squad-','')
  16.         os.rename(os.path.join(parent,filename),os.path.join(parent,newName))#重命名

处理完后就可以使用transformers库进行代码加载了。

模型使用

序列分类（以情感分类为例）

1.使用管道

  1. model_path="H:\\code\\Model\\bert-base-cased-finetuned-mrpc\\"
  2.  
  3. from transformers import pipeline
  4. #使用当前模型+使用Tensorflow框架，默认应该是使用PYTORCH框架
  5. nlp = pipeline("sentiment-analysis",model=model_path, tokenizer=model_path, framework="tf")
  6. result = nlp("I hate you")[0]
  7. print(f"label: {result['label']}, with score: {round(result['score'], 4)}")
  8. result = nlp("I love you")[0]
  9. print(f"label: {result['label']}, with score: {round(result['score'], 4)}")

2.直接使用模型

  1. model_path="H:\\code\\Model\\bert-base-cased-finetuned-mrpc\\"
  2. #pytorch框架
  3.  
  4. from transformers import AutoTokenizer, AutoModelForSequenceClassification
  5. import torch
  6. tokenizer = AutoTokenizer.from_pretrained(model_path)
  7. model = AutoModelForSequenceClassification.from_pretrained(model_path)
  8. classes = ["not paraphrase", "is paraphrase"]
  9. sequence_0 = "The company HuggingFace is based in New York City"
  10. sequence_1 = "Apples are especially bad for your health"
  11. sequence_2 = "HuggingFace's headquarters are situated in Manhattan"
  12. paraphrase = tokenizer(sequence_0, sequence_2, return_tensors="pt")
  13. not_paraphrase = tokenizer(sequence_0, sequence_1, return_tensors="pt")
  14. paraphrase_classification_logits = model(**paraphrase).logits
  15. not_paraphrase_classification_logits = model(**not_paraphrase).logits
  16. paraphrase_results = torch.softmax(paraphrase_classification_logits, dim=1).tolist()[0]
  17. not_paraphrase_results = torch.softmax(not_paraphrase_classification_logits, dim=1).tolist()[0]
  18. # Should be paraphrase
  19. for i in range(len(classes)):
  20.     print(f"{classes[i]}: {int(round(paraphrase_results[i] * 100))}%")
  21. # Should not be paraphrase
  22. for i in range(len(classes)):
  23.     print(f"{classes[i]}: {int(round(not_paraphrase_results[i] * 100))}%")
  24.  
  25. #tensorflow框架
  26. from transformers import AutoTokenizer, TFAutoModelForSequenceClassification
  27. import tensorflow as tf
  28. tokenizer = AutoTokenizer.from_pretrained(model_path)
  29. model = TFAutoModelForSequenceClassification.from_pretrained(model_path)
  30. classes = ["not paraphrase", "is paraphrase"]
  31. sequence_0 = "The company HuggingFace is based in New York City"
  32. sequence_1 = "Apples are especially bad for your health"
  33. sequence_2 = "HuggingFace's headquarters are situated in Manhattan"
  34. paraphrase = tokenizer(sequence_0, sequence_2, return_tensors="tf")
  35. not_paraphrase = tokenizer(sequence_0, sequence_1, return_tensors="tf")
  36. paraphrase_classification_logits = model(paraphrase)[0]
  37. not_paraphrase_classification_logits = model(not_paraphrase)[0]
  38. paraphrase_results = tf.nn.softmax(paraphrase_classification_logits, axis=1).numpy()[0]
  39. not_paraphrase_results = tf.nn.softmax(not_paraphrase_classification_logits, axis=1).numpy()[0]
  40. # Should be paraphrase
  41. for i in range(len(classes)):
  42.     print(f"{classes[i]}: {int(round(paraphrase_results[i] * 100))}%")
  43. # Should not be paraphrase
  44. for i in range(len(classes)):
  45.     print(f"{classes[i]}: {int(round(not_paraphrase_results[i] * 100))}%")

提取式问答

1.使用管道

  1. model_path="H:\\code\\Model\\bert-large-uncased-whole-word-masking-finetuned-squad\\"
  2.  
  3. from transformers import pipeline
  4. nlp = pipeline("question-answering",model=model_path, tokenizer=model_path)
  5. context = r"""
  6. Extractive Question Answering is the task of extracting an answer from a text given a question. An example of a
  7. question answering dataset is the SQuAD dataset, which is entirely based on that task. If you would like to fine-tune
  8. a model on a SQuAD task, you may leverage the examples/question-answering/run_squad.py script.
  9. """
  10. result = nlp(question="What is extractive question answering?", context=context)
  11. print(f"Answer: '{result['answer']}', score: {round(result['score'], 4)}, start: {result['start']}, end: {result['end']}")
  12. result = nlp(question="What is a good example of a question answering dataset?", context=context)
  13. print(f"Answer: '{result['answer']}', score: {round(result['score'], 4)}, start: {result['start']}, end: {result['end']}")

2.直接使用模型

  1. model_path="H:\\code\\Model\\bert-large-uncased-whole-word-masking-finetuned-squad\\"
  2. #使用pytorch框架
  3. from transformers import AutoTokenizer, AutoModelForQuestionAnswering
  4. import torch
  5. tokenizer = AutoTokenizer.from_pretrained(model_path)
  6. model = AutoModelForQuestionAnswering.from_pretrained(model_path)
  7. text = r"""
  8. 声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/小丑西瓜9/article/detail/129761
    推荐阅读
    相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。