bert textcnn用于文本分类_bert-textrnn

bert 

1. load bert 方式

tf bert源代码 <br/>
tf keras_bert <br/>
tf tensorflow_hub <br/>
tf bert-as-service <br/>
torch tf-->torch(用transformers库把tf-bert模型转成torch) pytorch_pretrained_bert <br/>

input_ids <br/>
input_mask <br/>
segment_ids <br/>

2.preprocess our data <br/>
1. lowercase our text (for:EN) <br/>
2. Tokenize it(i.e. "sally says hi" -> ["sally", "says", "hi"]) <br/>
3. break words into wordpieces <br/>
4. map our words to indexes using a vocab file that bert provides <br/>
5. add special 'CLS' and 'SEP' token <br/>
6. append 'index' and 'segment' tokens to each input (see the [BERT paper](https://arxiv.org/pdf/1810.04805.pdf))<br/>

bert fine-tune <br/>
bert 得到特征进行下游任务 <br/>
一、当作文本特征提取的工具，类似Word2vec模型一样 <br/>
二、作为一个可训练的层，后面可接入客制化的网络，做迁移学习 <br/>

使用keras_bert来加载构建bert模型

import codecs
import pandas as pd
import numpy as np
from keras.utils import to_categorical, multi_gpu_model
from keras.preprocessing.text import Tokenizer
from ker