Transformers 库在NLP任务上的入门与使用_transformers库官网

作者：Cpp五条 | 2024-04-06 08:26:43

踩

transformers库官网

0 库

	网址
库的 GitHub 地址	https://github.com/huggingface/transformers
官方开发文档	https://huggingface.co/docs/transformers/index
预训练模型下载地址	https://huggingface.co/models

1 pipeline

1.1 介绍

现有的模型和checkpoint，可以直接拿来用，处理某个任务，如情感分类、文本生成、命名实体识别、问答等。

支持的任务

Task	Description	Modality	Pipeline identifier
Text classification	assign a label to a given sequence of text	NLP	pipeline(task=“sentiment-analysis”)
Text generation	generate text that follows a given prompt	NLP	pipeline(task=“text-generation”)
Name entity recognition	assign a label to each token in a sequence (people, organization, location, etc.)	NLP	pipeline(task=“ner”)
Question answering	extract an answer from the text given some context and a question	NLP	pipeline(task=“question-answering”)
Fill-mask	predict the correct masked token in a sequence	NLP	pipeline(task=“fill-mask”)
Summarization	generate a summary of a sequence of text or document	NLP	pipeline(task=“summarization”)
Translation	translate text from one language into another	NLP	pipeline(task=“translation”)
Image classification	assign a label to an image	Computer vision	pipeline(task=“image-classification”)
Image segmentation	assign a label to each individual pixel of an image (supports semantic, panoptic, and instance segmentation)	Computer vision	pipeline(task=“image-segmentation”)
Object detection	predict the bounding boxes and classes of objects in an image	Computer vision	pipeline(task=“object-detection”)
Audio classification	assign a label to an audio file	Audio	pipeline(task=“audio-classification”)
Automatic speech recognition	extract speech from an audio file into text	Audio	pipeline(task=“automatic-speech-recognition”)
Visual question answering	given an image and a question, correctly answer a question about the image	Multimodal	pipeline(task=“vqa”)

1.2 用于某个任务

1.2.1 情感分析

默认模型

from transformers import pipeline

classifier = pipeline("sentiment-analysis")
1
2
3

测试 “We are very happy to show you the

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/Cpp五条/article/detail/370819