devbox
小惠珠哦
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

基于 BERT+BILSTM 实现情感分析分类（附源码）

作者：小惠珠哦 | 2024-08-01 11:38:21

赞

踩

bert+bilstm

目录

一、数据集

二、数据清洗和划分

2.1 安装依赖

2.2 清洗和划分

三、下载 Bert 模型

四、训练和测试模型

本文主要基于 Bert 和 BiLSTM 实现情感分类，其中参考了多个博客，具体见参考链接。

源码已上传Gitee : bert-bilstm-in-Sentiment-classification

一、数据集

数据集采用疫情期间微博评论数据集，一共10万条，可从这里下载Weibo nCoV Data。

二、数据清洗和划分

2.1 安装依赖

主要安装如下依赖，pip 更换国内源。

  1. pandas
  2. scikit-learn
  3. matplotlib
  4. seaborn
  5. torch
  6. torchvision
  7. datasets
  8. transformers
  9. pytorch_lightning

2.2 清洗和划分

实现代码如下所示。

  1. import pandas as pd
  2. from sklearn.model_selection import train_test_split
  3. import pandas as pd
  4. import matplotlib.pyplot as plt
  5. import seaborn as sns
  6.  
  7. # Read Data
  8. df = pd.read_csv('data/nCoV_100k_train.labled.csv')
  9.  
  10. # Only need text and labels
  11. df = df[['微博中文内容', '情感倾向']]
  12. df = df.rename(columns={'微博中文内容': 'text', '情感倾向': 'label'})
  13. print(df)
  14.  
  15. # Observing data balance
  16. print(df.label.value_counts())
  17. print(df.label.value_counts() / df.shape[0] * 100)
  18. plt.figure(figsize=(8, 4))
  19. sns.countplot(x='label', data=df)
  20. plt.show()
  21. # print(df_train[df_train.label > 5.0])
  22. # print(df_train[(df_train.label < -1.1)])
  23. # # discarding outliers
  24. # df_train.drop(df_train[(df_train.label < -1.1) | (df_train.label > 5)].index, inplace=True, axis=0)
  25. # df_train.reset_index(inplace=True, drop=True)
  26. # print(df_train.label.value_counts())
  27. # sns.countplot(x='label', data=df_train)
  28. # plt.show()
  29.  
  30. df.drop(df[(df.label == '4') |
  31.            (df.label == '-') |
  32.            (df.label == '·') |
  33.           (df.label == '-2') |
  34.           (df.label == '10') |
  35.            (df.label == '9')].index, inplace=True, axis=0)
  36. df.reset_index(inplace=True, drop=True)
  37. print(df.value_counts())
  38. sns.countplot(x='label', data=df)
  39. plt.show()
  40.  
  41.  
  42. # checking for empty rows
  43. print(df.isnull().sum())
  44. # deleting empty row data
  45. df.dropna(axis=0, how='any', inplace=True)
  46. df.reset_index(inplace=True, drop=True)
  47. print(df.isnull().sum())
  48.  
  49. # examining duplicate data
  50. print(df.duplicated().sum())
  51. print(df[df.duplicated()==True])
  52. # deleting duplicate data
  53. index = df[df.duplicated() == True].index
  54. df.drop(index, axis=0, inplace=True)
  55. df.reset_index(inplace=True, drop=True)
  56. print(df.duplicated().sum())
  57.  
  58. # We also need to address duplicate data where the text is the same but the label is different
  59. print(df['text'].duplicated().sum())
  60. print(df[df['text'].duplicated() == True])
  61.  
  62.  
  63. # viewing examples
  64. print(df[df['text'] == df.iloc[1473]['text']])
  65. print(df[df['text'] == df.iloc[1814]['text']])
  66.  
  67. # removing data where the text is the same but the label is different
  68. index = df[df['text'].duplicated() == True].index
  69. df.drop(index, axis=0, inplace=True)
  70. df.reset_index(inplace=True, drop=True)
  71.  
  72. # checking
  73. print(df['text'].duplicated().sum())  # 0
  74. print(df)
  75.  
  76.  
  77. # inspecting shapes and indices
  78. print("======data-clean======")
  79. print(df.tail())
  80. print(df.shape)
  81.  
  82. # viewing the maximum length of text
  83. print(df['text'].str.len().sort_values())
  84.  
  85.  
  86. # Split dataset. 0.6/0.2/0.2
  87. train, test = train_test_split(df, test_size=0.2)
  88. train, val = train_test_split(train, test_size=0.25)
  89. print(train.shape)
  90. print(test.shape)
  91. print(val.shape)
  92.  
  93. train.to_csv('./data/clean/train.csv', index=None)
  94. val.to_csv('./data/clean/val.csv', index=None)
  95. test.to_csv('./data/clean/test.csv', index=None)

三、下载 Bert 模型

在 Hugging Face google-bert/bert-base-chinese 下载对应的模型文件，魔搭社区下载的运行可能有如下问题。

safetensors_rust.SafetensorError: Error while deserializing header: HeaderTooLarge

 下载对应的模型文件。

四、训练和测试模型

训练和测试代码如下所示。

  1. # https://www.kaggle.com/code/isseyice/sentiment-classification-based-on-bert-and-lstm
  2. # https://github.com/iceissey/issey_Kaggle/blob/main/Bert_BiLSTM/BiLSTM_lighting.py#L36
  3. # https://www.cnblogs.com/chuanzhang053/p/17653381.html
  4. import torch
  5. import datasets
  6. import pandas as pd
  7. from datasets import load_dataset  # hugging-face dataset
  8. from torch.utils.data import Dataset
  9. from torch.utils.data import DataLoader
  10. import torch.nn as nn
  11. from transformers import BertTokenizer, BertModel
  12. import torch.optim as optim
  13. from torch.nn.functional import one_hot
  14. import pytorch_lightning as pl
  15. from pytorch_lightning import Trainer
  16. from torchmetrics.functional import accuracy, recall, precision, f1_score  # lightning中的评估
  17. from pytorch_lightning.callbacks.early_stopping import EarlyStopping
  18. from pytorch_lightning.callbacks import ModelCheckpoint
  19.  
  20.  
  21. # todo: 定义超参数
  22. batch_size = 16
  23. epochs = 5
  24. dropout = 0.1
  25. rnn_hidden = 768
  26. rnn_layer = 1
  27. class_num = 3
  28. lr = 0.001
  29. PATH = './model'  # model checkpoint path
  30. # 分词器
  31. token = BertTokenizer.from_pretrained('./model/bert-base-chinese')
  32.  
  33. # Customize dataset 自定义数据集
  34. class MydataSet(Dataset):
  35.     def __init__(self, path, split):
  36.         # self.dataset = load_dataset('csv', data_files=path, split=split)  # TypeError: read_csv() got an unexpected keyword argument 'mangle_dupe_cols'.
  37.         self.df = pd.read_csv(path)
  38.         self.dataset = datasets.Dataset.from_pandas(self.df)
  39.  
  40.     def __getitem__(self, item):
  41.         text = self.dataset[item]['text']
  42.         label = self.dataset[item]['label']
  43.         return text, label
  44.  
  45.     def __len__(self):
  46.         return len(self.dataset)
  47.  
  48.  
  49. # todo: 定义批处理函数
  50. def collate_fn(data):
  51.     sents = [i[0] for i in data]
  52.     labels = [i[1] for i in data]
  53.  
  54.     # 分词并编码
  55.     data = token.batch_encode_plus(
  56.         batch_text_or_text_pairs=sents,  # 单个句子参与编码
  57.         truncation=True,  # 当句子长度大于max_length时,截断
  58.         padding='max_length',  # 一律补pad到max_length长度
  59.         max_length=300,
  60.         return_tensors='pt',  # 以pytorch的形式返回，可取值tf,pt,np,默认为返回list
  61.         return_length=True,
  62.     )
  63.  
  64.     # input_ids: 编码之后的数字
  65.     # attention_mask: 是补零的位置是0,其他位置是1
  66.     input_ids = data['input_ids']   # input_ids 就是编码后的词
  67.     attention_mask = data['attention_mask']  # pad的位置是0,其他位置是1
  68.     token_type_ids = data['token_type_ids']  # (如果是一对句子)第一个句子和特殊符号的位置是0,第二个句子的位置是1
  69.     labels = torch.LongTensor(labels)  # 该批次的labels
  70.  
  71.     # print(data['length'], data['length'].max())
  72.     return input_ids, attention_mask, token_type_ids, labels
  73.  
  74.  
  75. # 定义模型，上游使用bert预训练，下游任务选择双向LSTM模型，最后加一个全连接层
  76. class BiLSTMClassifier(nn.Module):
  77.     def __init__(self, drop, hidden_dim, output_dim):
  78.         super(BiLSTMClassifier, self).__init__()
  79.         self.drop = drop
  80.         self.hidden_dim = hidden_dim
  81.         self.output_dim = output_dim
  82.  
  83.         # 加载bert中文模型
  84.         self.embedding = BertModel.from_pretrained('./model/bert-base-chinese')
  85.  
  86.         # 冻结上游模型参数(不进行预训练模型参数学习)
  87.         for param in self.embedding.parameters():
  88.             param.requires_grad_(False)
  89.         
  90.         #生成下游RNN层以及全连接层
  91.         self.lstm = nn.LSTM(input_size=768, hidden_size=self.hidden_dim, num_layers=2, batch_first=True,
  92.                             bidirectional=True, dropout=self.drop)
  93.         self.fc = nn.Linear(self.hidden_dim * 2, self.output_dim)
  94.         # 使用CrossEntropyLoss作为损失函数时，不需要激活。因为实际上CrossEntropyLoss将softmax-log-NLLLoss一并实现的。
  95.  
  96.     def forward(self, input_ids, attention_mask, token_type_ids):
  97.         embedded = self.embedding(input_ids=input_ids, attention_mask=attention_mask, token_type_ids=token_type_ids)
  98.         # 第0维才是我们需要的embedding,embedding.last_hidden_state = embedding[0]
  99.         embedded = embedded.last_hidden_state  # The embedding we need is in the 0th dimension. So, to extract it, we use: embedding.last_hidden_state = embedding[0].
  100.         out, (h_n, c_n) = self.lstm(embedded)
  101.         output = torch.cat((h_n[-2, :, :], h_n[-1, :, :]), dim=1)  # This part represents the output of the bidirectional LSTM. For detailed explanations, please refer to the article.
  102.         output = self.fc(output)
  103.         return output
  104.  
  105.  
  106. # Define PyTorch Lightning
  107. class BiLSTMLighting(pl.LightningModule):
  108.     def __init__(self, drop, hidden_dim, output_dim):
  109.         super(BiLSTMLighting, self).__init__()
  110.         self.model = BiLSTMClassifier(drop, hidden_dim, output_dim)  # Set up model
  111.         self.criterion = nn.CrossEntropyLoss()  # Set up loss function
  112.         self.train_dataset = MydataSet('./data/clean/train.csv', 'train')
  113.         self.val_dataset = MydataSet('./data/clean/val.csv', 'train')
  114.         self.test_dataset = MydataSet('./data/clean/test.csv', 'train')
  115.  
  116.     def configure_optimizers(self):
  117.         optimizer = optim.AdamW(self.parameters(), lr=lr)
  118.         return optimizer
  119.  
  120.     def forward(self, input_ids, attention_mask, token_type_ids):  # forward(self,x)
  121.         return self.model(input_ids, attention_mask, token_type_ids)
  122.  
  123.     def train_dataloader(self):
  124.         train_loader = DataLoader(dataset=self.train_dataset, batch_size=batch_size, collate_fn=collate_fn,
  125.                                   shuffle=True, num_workers=3)
  126.         return train_loader
  127.  
  128.     def training_step(self, batch, batch_idx):
  129.         input_ids, attention_mask, token_type_ids, labels = batch  # x, y = batch
  130.         y = one_hot(labels + 1, num_classes=3)
  131.         # 将one_hot_labels类型转换成float
  132.         y = y.to(dtype=torch.float)
  133.         # forward pass
  134.         y_hat = self.model(input_ids, attention_mask, token_type_ids)
  135.         # y_hat = y_hat.squeeze()  # 将[128, 1, 3]挤压为[128,3]
  136.         loss = self.criterion(y_hat, y)  # criterion(input, target)
  137.         self.log('train_loss', loss, prog_bar=True, logger=True, on_step=True, on_epoch=True)  # 将loss输出在控制台
  138.         return loss  # 必须把log返回回去才有用
  139.  
  140.     def val_dataloader(self):
  141.         val_loader = DataLoader(dataset=self.val_dataset, batch_size=batch_size, collate_fn=collate_fn, shuffle=False, num_workers=3)
  142.         return val_loader
  143.  
  144.     def validation_step(self, batch, batch_idx):
  145.         input_ids, attention_mask, token_type_ids, labels = batch
  146.         y = one_hot(labels + 1, num_classes=3)
  147.         y = y.to(dtype=torch.float)
  148.         # forward pass
  149.         y_hat = self.model(input_ids, attention_mask, token_type_ids)
  150.         # y_hat = y_hat.squeeze()
  151.         loss = self.criterion(y_hat, y)
  152.         self.log('val_loss', loss, prog_bar=False, logger=True, on_step=True, on_epoch=True)
  153.         return loss
  154.  
  155.     def test_dataloader(self):
  156.         test_loader = DataLoader(dataset=self.test_dataset, batch_size=batch_size, collate_fn=collate_fn, shuffle=False, num_workers=3)
  157.         return test_loader
  158.  
  159.     def test_step(self, batch, batch_idx):
  160.         input_ids, attention_mask, token_type_ids, labels = batch
  161.         target = labels + 1  # Used for calculating accuracy and F1-score later on
  162.         y = one_hot(target, num_classes=3)
  163.         y = y.to(dtype=torch.float)
  164.         # forward pass
  165.         y_hat = self.model(input_ids, attention_mask, token_type_ids)
  166.         # y_hat = y_hat.squeeze()
  167.         pred = torch.argmax(y_hat, dim=1)
  168.         acc = (pred == target).float().mean()
  169.  
  170.         loss = self.criterion(y_hat, y)
  171.         self.log('loss', loss)
  172.         # task: Literal["binary", "multiclass", "multilabel"], corresponding to [binary classification, multiclass classification, multilabel classification]
  173.         # average=None: outputs scores for each class separately; without it, the scores are averaged.
  174.         re = recall(pred, target, task="multiclass", num_classes=class_num, average=None)
  175.         pre = precision(pred, target, task="multiclass", num_classes=class_num, average=None)
  176.         f1 = f1_score(pred, target, task="multiclass", num_classes=class_num, average=None)
  177.  
  178.         def log_score(name, scores):
  179.             for i, score_class in enumerate(scores):
  180.                 self.log(f"{name}_class{i}", score_class)
  181.  
  182.         log_score("recall", re)
  183.         log_score("precision", pre)
  184.         log_score("f1", f1)
  185.         self.log('acc', accuracy(pred, target, task="multiclass", num_classes=class_num))
  186.         self.log('avg_recall', recall(pred, target, task="multiclass", num_classes=class_num, average="weighted"))
  187.         self.log('avg_precision', precision(pred, target, task="multiclass", num_classes=class_num, average="weighted"))
  188.         self.log('avg_f1', f1_score(pred, target, task="multiclass", num_classes=class_num, average="weighted"))
  189.  
  190. # 训练
  191. def train():
  192.     # 增加回调最优模型，这个比较好用
  193.     checkpoint_callback = ModelCheckpoint(
  194.         monitor='val_loss',  # Monitor object: 'val_loss'
  195.         dirpath='./model/checkpoints/',  # Path to save the model
  196.         filename='model-{epoch:02d}-{val_loss:.2f}',  # Name of the best model
  197.         save_top_k=1,  # Save only the best one
  198.         mode='min'  # When the monitored metric is at its lowest.
  199.     )
  200.  
  201.     # Trainer可以帮助调试，比如快速运行、只使用一小部分数据进行测试、完整性检查等，
  202.     # 详情请见官方文档https://lightning.ai/docs/pytorch/latest/debug/debugging_basic.html
  203.     # auto自适应gpu数量
  204.     trainer = Trainer(max_epochs=epochs, log_every_n_steps=10, accelerator='gpu', devices="auto", fast_dev_run=False, callbacks=[checkpoint_callback])
  205.     model = BiLSTMLighting(drop = dropout, hidden_dim = rnn_hidden, output_dim = class_num)
  206.     trainer.fit(model)
  207.     
  208.     return model
  209.  
  210. # 测试
  211. def test(model = None):
  212.     # Load the parameters of the previously trained best model.
  213.     if model is None:
  214.         model = BiLSTMLighting.load_from_checkpoint(checkpoint_path=PATH,
  215.                                                     drop=dropout, hidden_dim=rnn_hidden, output_dim=class_num)
  216.     trainer = Trainer(fast_dev_run=False)
  217.     result = trainer.test(model)
  218.     print(result)
  219.  
  220. if __name__ == '__main__':
  221.  
  222.     model = train()
  223.     test(model)

执行如下命令。

python main.py

运行结果：

  1. root@dsw-398300-bf64cb7b7-f28cl:/mnt/workspace/bert-bilstm-in-sentiment-classification# python main.py 
  2. 2024-07-13 13:18:25.494250: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
  3. 2024-07-13 13:18:25.933228: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
  4. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
  5. 2024-07-13 13:18:27.151707: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
  6. GPU available: True (cuda), used: True
  7. TPU available: False, using: 0 TPU cores
  8. IPU available: False, using: 0 IPUs
  9. HPU available: False, using: 0 HPUs
  10. Some weights of the model checkpoint at ./model/bert-base-chinese were not used when initializing BertModel: ['cls.seq_relationship.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.seq_relationship.weight', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']
  11. - This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
  12. - This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  13. Missing logger folder: /mnt/workspace/bert-bilstm-in-sentiment-classification/lightning_logs
  14. LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
  15.  
  16.   | Name      | Type             | Params
  17. -----------------------------------------------
  18. 0 | model     | BiLSTMClassifier | 125 M 
  19. 1 | criterion | CrossEntropyLoss | 0     
  20. -----------------------------------------------
  21. 23.6 M    Trainable params
  22. 102 M     Non-trainable params
  23. 125 M     Total params
  24. 503.559   Total estimated model params size (MB)
  25. Epoch 4: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4506/4506 [13:41<00:00,  5.48it/s, loss=0.52, v_num=0, train_loss_step=0.654, train_loss_epoch=0.566]`Trainer.fit` stopped: `max_epochs=5` reached.                                                                                                                                                                                                                                              
  26. Epoch 4: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4506/4506 [13:42<00:00,  5.48it/s, loss=0.52, v_num=0, train_loss_step=0.654, train_loss_epoch=0.566]
  27. root@dsw-398300-bf64cb7b7-f28cl:/mnt/workspace/bert-bilstm-in-sentiment-classification# python main.py 
  28. 2024-07-13 15:10:59.831283: I tensorflow/core/util/port.cc:110] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
  29. 2024-07-13 15:10:59.868774: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
  30. To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
  31. 2024-07-13 15:11:00.420343: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
  32. GPU available: True (cuda), used: True
  33. TPU available: False, using: 0 TPU cores
  34. IPU available: False, using: 0 IPUs
  35. HPU available: False, using: 0 HPUs
  36. Some weights of the model checkpoint at ./model/bert-base-chinese were not used when initializing BertModel: ['cls.predictions.transform.dense.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.bias', 'cls.seq_relationship.weight', 'cls.predictions.bias']
  37. - This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
  38. - This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  39. LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]
  40.  
  41.   | Name      | Type             | Params
  42. -----------------------------------------------
  43. 0 | model     | BiLSTMClassifier | 125 M 
  44. 1 | criterion | CrossEntropyLoss | 0     
  45. -----------------------------------------------
  46. 23.6 M    Trainable params
  47. 102 M     Non-trainable params
  48. 125 M     Total params
  49. 503.559   Total estimated model params size (MB)
  50. Epoch 4: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4506/4506 [13:41<00:00,  5.48it/s, loss=0.544, v_num=1, train_loss_step=0.435, train_loss_epoch=0.568]`Trainer.fit` stopped: `max_epochs=5` reached.                                                                                                                                                                                                                                              
  51. Epoch 4: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4506/4506 [13:41<00:00,  5.48it/s, loss=0.544, v_num=1, train_loss_step=0.435, train_loss_epoch=0.568]
  52. GPU available: True (cuda), used: False
  53. TPU available: False, using: 0 TPU cores
  54. IPU available: False, using: 0 IPUs
  55. HPU available: False, using: 0 HPUs
  56. /opt/conda/lib/python3.8/site-packages/pytorch_lightning/trainer/trainer.py:1764: PossibleUserWarning: GPU available but not used. Set `accelerator` and `devices` using `Trainer(accelerator='gpu', devices=1)`.
  57.   rank_zero_warn(
  58. Testing DataLoader 0:  61%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▉                                                                                      | 690/1127 [46:08<29:13,  4.01s/it]
  59. Testing DataLoader 0: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1127/1127 [1:17:28<00:00,  4.12s/it]
  60. ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
  61. ┃        Test metric        ┃       DataLoader 0        ┃
  62. ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
  63. │            acc            │    0.7424814105033875     │
  64. │          avg_f1           │    0.7335338592529297     │
  65. │       avg_precision       │    0.7671085596084595     │
  66. │        avg_recall         │    0.7424814105033875     │
  67. │         f1_class0         │    0.5128293633460999     │
  68. │         f1_class1         │    0.7843108177185059     │
  69. │         f1_class2         │    0.6716415286064148     │
  70. │           loss            │    0.5851244330406189     │
  71. │     precision_class0      │     0.637470543384552     │
  72. │     precision_class1      │    0.7558891773223877     │
  73. │     precision_class2      │    0.7077022790908813     │
  74. │       recall_class0       │    0.4741062521934509     │
  75. │       recall_class1       │    0.8350556492805481     │
  76. │       recall_class2       │    0.6949878334999084     │
  77. └───────────────────────────┴───────────────────────────┘
  78. [{'loss': 0.5851244330406189, 'recall_class0': 0.4741062521934509, 'recall_class1': 0.8350556492805481, 'recall_class2': 0.6949878334999084, 'precision_class0': 0.637470543384552, 'precision_class1': 0.7558891773223877, 'precision_class2': 0.7077022790908813, 'f1_class0': 0.5128293633460999, 'f1_class1': 0.7843108177185059, 'f1_class2': 0.6716415286064148, 'acc': 0.7424814105033875, 'avg_recall': 0.7424814105033875, 'avg_precision': 0.7671085596084595, 'avg_f1': 0.7335338592529297}]
  79. root@dsw-398300-bf64cb7b7-f28cl:/mnt/workspace/bert-bilstm-in-sentiment-classification#

参考链接：

[1] 使用huggingface实现BERT+BILSTM情感3分类（附数据集源代码）_bert和bilstm中间需要加入drop层么-CSDN博客

[2] https://huggingface.co/google-bert/bert-base-chinese/tree/main

[3] 【NLP实战】基于Bert和双向LSTM的情感分类【上篇】_bert+lstm-CSDN博客

[4] 【NLP实战】基于Bert和双向LSTM的情感分类【中篇】_bert+lstm-CSDN博客

[5] https://github.com/iceissey/issey_Kaggle/tree/main/Bert_BiLSTM

[6] https://www.kaggle.com/code/isseyice/sentiment-classification-based-on-bert-and-lstm#Part-2:-Training-and-Evaluating-the-Model [7]https://www.kaggle.com/datasets/liangqingyuan/chinese-text-multi-classification?resource=download

[8]千言（LUGE）| 全面的中文开源数据集合 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/小惠珠哦/article/detail/913877
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号