【动手写BERT】tokenizer初探_no model was supplied defaulted to

import transformers
from transformers import pipeline
1
2

task_name = 'sentiment-analysis'
1

自动下载的模型就是"distilbert-base-uncased-finetuned-sst-2-english"

pipeline(task_name)
1

No model was supplied, defaulted to distilbert-base-uncased-finetuned-sst-2-english and revision af0f99b (https://huggingface.co/distilbert-base-uncased-finetuned-sst-2-english).
Using a pipeline without specifying a model name and revision in production is not recommended.
Downloading model.safetensors:  63%|████████████████████████████████████████▍                       | 169M/268M [00:35<00:20, 4.79MB/s]
Downloading model.safetensors: 100%|████████████████████████████████████████████████████████████████| 268M/268M [00:51<00:00, 5.18MB/s]





<transformers.pipelines.text_classification.TextClassificationPipeline at 0x14543df30>
1
2
3
4
5
6
7
8
9
10

tokenizer, 构造输入

• tokenizer, model:相匹配。tokenizer output => model input;在tokenizer部分模型已经做了大部分的工作

test_sentence = ['today is not that bad', 'today is so bad']
model_name = "distilbert-base-uncased-finetuned-sst-2-english"
1
2

from transformers import AutoTokenizer, AutoModelForSequenceClassification
1

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
1
2

batch_input = tokenizer(test_sentence, return_tensors='pt', padding=True)
1

batch_input
1

{'input_ids': tensor([[ 101, 2651, 2003, 2025, 2008, 2919,  102],
        [ 101, 2651, 2003, 2061, 2919,  102,    0]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1],
        [1, 1, 1, 1, 1, 1, 0]])}
1
2
3

model, 调用模型

import torch
import torch.nn.functional as F
1
2

output = model(**batch_input)
1

output
1

SequenceClassifierOutput(loss=None, logits=tensor([[-3.4620,  3.6118],
        [ 4.7508, -3.7899]], grad_fn=<AddmmBackward0>), hidden_states=None, attentions=None)
1
2

model.config
1

DistilBertConfig {
  "_name_or_path": "distilbert-base-uncased-finetuned-sst-2-english",
  "activation": "gelu",
  "architectures": [
    "DistilBertForSequenceClassification"
  ],
  "attention_dropout": 0.1,
  "dim": 768,
  "dropout": 0.1,
  "finetuning_task": "sst-2",
  "hidden_dim": 3072,
  "id2label": {
    "0": "NEGATIVE",
    "1": "POSITIVE"
  },
  "initializer_range": 0.02,
  "label2id": {
    "NEGATIVE": 0,
    "POSITIVE": 1
  },
  "max_position_embeddings": 512,
  "model_type": "distilbert",
  "n_heads": 12,
  "n_layers": 6,
  "output_past": true,
  "pad_token_id": 0,
  "qa_dropout": 0.1,
  "seq_classif_dropout": 0.2,
  "sinusoidal_pos_embds": false,
  "tie_weights_": true,
  "transformers_version": "4.34.0",
  "vocab_size": 30522
}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

with torch.no_grad():
    output = model(**batch_input)
    print(output)
    scores = F.softmax(output.logits, dim=-1)
    print(scores)
    labels = scores.argmax(dim=-1)
    print(labels)
    labels = [model.config.id2label[id] for id in labels.tolist()]
    print(labels)
1
2
3
4
5
6
7
8
9

SequenceClassifierOutput(loss=None, logits=tensor([[-3.4620,  3.6118],
        [ 4.7508, -3.7899]]), hidden_states=None, attentions=None)
tensor([[8.4631e-04, 9.9915e-01],
        [9.9980e-01, 1.9531e-04]])
tensor([1, 0])
['POSITIVE', 'NEGATIVE']
1
2
3
4
5
6

更详细内容请查看网站how to use huggingface