friedrichor/nlp-huggingface-tutorial

NLP-HuggingFace-Tutorial

使用 Hugging Face 以及 PyTorch 做一些 NLP 任务的 tutorial

代码相对简洁易懂，适合新手入门

如有任何问题，可以在 issues 中提出，也可以在 CSDN/friedrichor 私信我。

PyTorch 环境配置 / 服务器

Windows: Windows 系统从零配置 Python 环境，安装CUDA、CUDNN、PyTorch 详细教程
Linux服务器: 远程服务器配置 Anaconda 并安装 PyTorch 详细教程
本地-服务器文件传输: 将本地项目/文件上传到远程服务器中详细教程（vscode，sftp）

对话生成

备注：对话生成的代码同样适用于文本生成、文本摘要等 text-to-text 任务。

• DialoGPT
  • paper: DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation (ACL 2020)
  • fine-tune DialoGPT for dialogue generation: code
• T5
  • paper: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (JMLR 2020)
  • fine-tune T5 for dialogue generation: code

文本分类

备注：情感分析、主题分类、意图识别等任务均属于文本分类。

• BERT
  • paper: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (NAACL 2019)
  • fine-tune BERT for text classification: code
  • prompt-tuning BERT for text classification: code
• RoBERTa
  • paper: RoBERTa: A Robustly Optimized BERT Pretraining Approach (arXiv 2019)
  • fine-tune RoBERTa for dialogue generation: code
• T5
  • paper: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer (JMLR 2020)
  • fine-tune T5 for text classification: code
  • fine-tune T5 Encoder for text classification: code

更新中...

nlp-huggingface-tutorial's Issues

[BUG] RuntimeError: Unexpected error from cudaGetDeviceCount()

(base) root@I12f01b1f1300b01732:/hy-tmp/code/NLP-HuggingFace-Tutorial/text_classification/T5# !53
python train.py --device='cuda'
/usr/local/miniconda3/lib/python3.8/site-packages/torch/cuda/init.py:88: UserWarning: CUDA initialization: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.)
return torch._C._cuda_getDeviceCount() > 0
{'batch_size': 16,
'classes_map_dir': '/hy-tmp/code/NLP-HuggingFace-Tutorial/text_classification/T5/classes_map.json',
'data_dir': '/hy-tmp/code/NLP-HuggingFace-Tutorial/text_classification/T5/dataset',
'device': 'cuda',
'learning_rate': 0.0001,
'lr_warmup_steps': 0,
'num_train_epochs': 10,
'num_workers': 8,
'prefix_text': 'tweet_eval emotion sentence: ',
'pretrained_model_name_or_path': 't5-base',
'save_weights_path': '/hy-tmp/code/NLP-HuggingFace-Tutorial/text_classification/T5/weights',
'use_Adafactor': True,
'use_AdafactorSchedule': True,
'use_weighted_random_sampler': False,
'weight_decay': 0}
Traceback (most recent call last):
File "train.py", line 187, in
main(args)
File "train.py", line 93, in main
model.to(args.device)
File "/usr/local/miniconda3/lib/python3.8/site-packages/transformers/modeling_utils.py", line 1811, in to
return super().to(*args, **kwargs)
File "/usr/local/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 989, in to
return self._apply(convert)
File "/usr/local/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 641, in _apply
module._apply(fn)
File "/usr/local/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 664, in _apply
param_applied = fn(param)
File "/usr/local/miniconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 987, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
File "/usr/local/miniconda3/lib/python3.8/site-packages/torch/cuda/init.py", line 229, in _lazy_init
torch._C._cuda_init()
RuntimeError: Unexpected error from cudaGetDeviceCount(). Did you run some cuda functions before calling NumCudaDevices() that might have already set an error? Error 804: forward compatibility was attempted on non supported HW

你好，在CPU上可以正常训练，在linux的GPU上有报错

输入长度

当我输入长文本的时候，error:Token indices sequence length is longer than the specified maximum sequence length for this model (2965 > 512)