fakeryfx/bert-pytorch-chinese-textclassification

Pytorch Bert Finetune in Chinese Text Classification

Bert-Pytorch-Chinese-TextClassification

Pytorch Bert Finetune in Chinese Text Classification

Step 1

Download the pretrained TensorFlow model:chinese_L-12_H-768_A-12

Step 2

Change the TensorFlow Pretrained Model into Pytorch

cd  convert_tf_to_pytorch

export BERT_BASE_DIR=/workspace/mnt/group/ocr/xieyufei/bert-tf-chinese/chinese_L-12_H-768_A-12

python3 convert_tf_checkpoint_to_pytorch.py \
  --tf_checkpoint_path $BERT_BASE_DIR/bert_model.ckpt \
  --bert_config_file $BERT_BASE_DIR/bert_config.json \
  --pytorch_dump_path $BERT_BASE_DIR/pytorch_model.bin

Step 3

Download the Chinese News DataSet:Train for 5w and Dev for 5k

Step 4

Just Train and Test

cd src

export GLUE_DIR=/workspace/mnt/group/ocr/xieyufei/bert-tf-chinese/glue_data
export BERT_BASE_DIR=/workspace/mnt/group/ocr/xieyufei/bert-tf-chinese/chinese_L-12_H-768_A-12/
export BERT_PYTORCH_DIR=/workspace/mnt/group/ocr/xieyufei/bert-tf-chinese/chinese_L-12_H-768_A-12/

python3 run_classifier_word.py \
  --task_name NEWS \
  --do_train \
  --do_eval \
  --data_dir $GLUE_DIR/SouGou/ \
  --vocab_file $BERT_BASE_DIR/vocab.txt \
  --bert_config_file $BERT_BASE_DIR/bert_config.json \
  --init_checkpoint $BERT_PYTORCH_DIR/pytorch_model.bin \
  --max_seq_length 256 \
  --train_batch_size 24 \
  --learning_rate 2e-5 \
  --num_train_epochs 50.0 \
  --output_dir ./newsAll_output/ \
  --local_rank 3

1个Epoch的结果如下：

eval_accuracy = 0.9742
eval_loss = 0.10202122390270234
global_step = 2084
loss = 0.15899521649851786

bert-pytorch-chinese-textclassification's Issues

关于test的问题

您好，您的代码让我受益匪浅，我也成功完成了训练阶段，我想请教您一下关于test的测试阶段的代码您方便提供我参考学习吗，万分感谢

when I run the shell,hava a CUDA error to me. it will runing on a right status if i use cpu.

python3 run_classifier_word.py --task_name NEWS --do_train --do_eval --data_dir $GLUE_DIR/ --vocab_file $BERT_BASE_DIR/vocab.txt --bert_config_file $BERT_BASE_DIR/bert_config.json --init_checkpoint $BERT_BASE_DIR/pytorch_model.bin --max_seq_length 256 --train_batch_size 24 --learning_rate 2e-5 --num_train_epochs 50.0 --output_dir ./newsAll_output/ --local_rank 3
04/16/2019 16:33:35 - INFO - main - device cuda:3 n_gpu 1 distributed training True
04/16/2019 16:33:35 - INFO - main - LOOKING AT /home/gpu0/Litao/Bert/Bert-Pytorch-Chinese-TextClassification/Corpus/train.tsv
label_list.size:10

Traceback (most recent call last):
File "run_classifier_word.py", line 704, in
main()
File "run_classifier_word.py", line 559, in main
model.to(device)
File "/opt/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 381, in to
return self._apply(convert)
File "/opt/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 187, in _apply
module._apply(fn)
File "/opt/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 187, in _apply
module._apply(fn)
File "/opt/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 187, in _apply
module._apply(fn)
File "/opt/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 193, in _apply
param.data = fn(param.data)
File "/opt/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 379, in convert
return t.to(device, dtype if t.is_floating_point() else None, non_blocking)
RuntimeError: CUDA error: invalid device ordinal

不好意思 請問有新聞的data可以分享一下嗎 我看網盤的連結好像過期了><