Comments (5)
This error can actually be due to different reasons. It is recommended to debug CUDA errors by running the code on the CPU, if possible. If that’s not possible, try to execute the script via:
CUDA_LAUNCH_BLOCKING=1 python [YOUR_PROGRAM]
from guwenbert.
This error can actually be due to different reasons. It is recommended to debug CUDA errors by running the code on the CPU, if possible. If that’s not possible, try to execute the script via:
CUDA_LAUNCH_BLOCKING=1 python [YOUR_PROGRAM]
禁用了gpu之后的报错是这这样的:
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
<ipython-input-20-ce6b8456fe47> in <module>
32 time.sleep(0.2)
33
---> 34 train_loss, train_acc = train_model(model, train_loader)
35 val_loss, val_acc = test_model(model, val_loader)
36
<ipython-input-19-7219d0502762> in train_model(model, train_loader)
30
31 with autocast(): #使用半精度训练
---> 32 output = model(input_ids, attention_mask, token_type_ids).logits
33
34 loss = criterion(output, y) / CFG['accum_iter']
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725 result = self._slow_forward(*input, **kwargs)
726 else:
--> 727 result = self.forward(*input, **kwargs)
728 for hook in itertools.chain(
729 _global_forward_hooks.values(),
/opt/conda/lib/python3.8/site-packages/transformers/models/roberta/modeling_roberta.py in forward(self, input_ids, token_type_ids, attention_mask, labels, position_ids, head_mask, inputs_embeds, output_attentions, output_hidden_states, return_dict)
1243 )
1244
-> 1245 outputs = self.roberta(
1246 flat_input_ids,
1247 position_ids=flat_position_ids,
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725 result = self._slow_forward(*input, **kwargs)
726 else:
--> 727 result = self.forward(*input, **kwargs)
728 for hook in itertools.chain(
729 _global_forward_hooks.values(),
/opt/conda/lib/python3.8/site-packages/transformers/models/roberta/modeling_roberta.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, past_key_values, use_cache, output_attentions, output_hidden_states, return_dict)
803 head_mask = self.get_head_mask(head_mask, self.config.num_hidden_layers)
804
--> 805 embedding_output = self.embeddings(
806 input_ids=input_ids,
807 position_ids=position_ids,
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725 result = self._slow_forward(*input, **kwargs)
726 else:
--> 727 result = self.forward(*input, **kwargs)
728 for hook in itertools.chain(
729 _global_forward_hooks.values(),
/opt/conda/lib/python3.8/site-packages/transformers/models/roberta/modeling_roberta.py in forward(self, input_ids, token_type_ids, position_ids, inputs_embeds, past_key_values_length)
116 if inputs_embeds is None:
117 inputs_embeds = self.word_embeddings(input_ids)
--> 118 token_type_embeddings = self.token_type_embeddings(token_type_ids)
119
120 embeddings = inputs_embeds + token_type_embeddings
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
725 result = self._slow_forward(*input, **kwargs)
726 else:
--> 727 result = self.forward(*input, **kwargs)
728 for hook in itertools.chain(
729 _global_forward_hooks.values(),
/opt/conda/lib/python3.8/site-packages/torch/nn/modules/sparse.py in forward(self, input)
122
123 def forward(self, input: Tensor) -> Tensor:
--> 124 return F.embedding(
125 input, self.weight, self.padding_idx, self.max_norm,
126 self.norm_type, self.scale_grad_by_freq, self.sparse)
/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
1850 # remove once script supports set_grad_enabled
1851 _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> 1852 return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
1853
1854
IndexError: index out of range in self
我已经让token_type_ids = torch.tensor(token_type_ids)全部为0了,但依然不清楚哪里有错,读取数据的代码如下
def collate_fn(data)
input_ids, attention_mask, token_type_ids = [], [], []
for x in data:
text = tokenizer(x[1], text_pair=x[0], padding='max_length', truncation=True, max_length=CFG['max_len'], return_tensors='pt')
input_ids.append(text['input_ids'].tolist())
attention_mask.append(text['attention_mask'].tolist())
token_type_ids.append(text['token_type_ids'].tolist())
input_ids = torch.tensor(input_ids)
attention_mask = torch.tensor(attention_mask)
token_type_ids = torch.tensor(token_type_ids)
token_type_ids = torch.zeros_like(token_type_ids)
label = torch.tensor([x[-1] for x in data])
# print(token_type_ids)
return input_ids, attention_mask, token_type_ids, label
请问这样需要怎么修改?
from guwenbert.
This error can actually be due to different reasons. It is recommended to debug CUDA errors by running the code on the CPU, if possible. If that’s not possible, try to execute the script via:
CUDA_LAUNCH_BLOCKING=1 python [YOUR_PROGRAM]
运行了这一句话之后的报错是
RuntimeError: CUDA error: device-side assert triggered
from guwenbert.
我在colab测试了一下,是没问题的。你可以再看看你的代码,如果还不行,在colab创建一个可以复现问题的最小样例,分享出来,以便更好的定位和解决问题。
https://colab.research.google.com/drive/1rvMqxEz3ayA6b583dGEXNL0Gu1ZgeL6G?usp=sharing
from guwenbert.
Duplicate of #11
from guwenbert.
Related Issues (17)
- 关于新建词表 HOT 2
- 输入两个句子导致索引越界 HOT 26
- 请问能否公开一下古联杯的训练数据呢 HOT 1
- 预训练载入方式咨询 HOT 5
- 预训练代码 HOT 3
- Mask值 HOT 5
- 请问从huggingface下载的模型怎么使用 HOT 6
- 预训练模型的问题 HOT 5
- 感觉效果不太好 HOT 3
- 你好,“古联杯”评测数据集可以公开吗? HOT 1
- 能建一个交流讨论群么 HOT 1
- 关于未来的工作计划 HOT 6
- 预训练语料 HOT 1
- 想请教一下模型结构的问题,为什么模型权重从pooler层之后就没有了 HOT 4
- 请问古文的训练数据方便公开吗? HOT 1
- 请问评估指标是如何确定的呢? HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from guwenbert.