jasoncao11 / nlp-notebook Goto Github PK
View Code? Open in Web Editor NEWNLP 领域常见任务的实现,包括新词发现、以及基于pytorch的词向量、中文文本分类、实体识别、摘要文本生成、句子相似度判断、三元组抽取、预训练模型等。
License: MIT License
NLP 领域常见任务的实现,包括新词发现、以及基于pytorch的词向量、中文文本分类、实体识别、摘要文本生成、句子相似度判断、三元组抽取、预训练模型等。
License: MIT License
您好,能告诉一下在4-1.Seq2seq项目中的环境吗?
File "/workspace/nlp-notebook/4-3.Transformer/model.py", line 256, in forward
enc_src = self.encoder(src, src_mask)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/workspace/nlp-notebook/4-3.Transformer/model.py", line 35, in forward
src = self.dropout((self.tok_embedding(src) * self.scale) + self.pos_embedding(pos))
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/torch/nn/modules/sparse.py", line 156, in forward
return F.embedding(
File "/opt/conda/lib/python3.8/site-packages/torch/nn/functional.py", line 1916, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self
生成模型Transformer有以上错误,排除了软件包版本问题,S2S+attention也有bug,具体和之前一个issue一样,tuple和tensor的问题
我目前是在校学生,只能用自己的笔记本进行实验,显卡是GTX2060 6GB的,但是显然是不够的,希望知情的人可以告知,不胜感激
(pytorch18) z@z:~/code/nlp-notebook-master/3-2.Bert-CRF$ python demo_train.py
Some weights of the model checkpoint at ./bert-base-chinese were not used when initializing BertForNER: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']
This IS expected if you are initializing BertForNER from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
This IS NOT expected if you are initializing BertForNER from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of BertForNER were not initialized from the model checkpoint at ./bert-base-chinese and are newly initialized: ['transitions', 'hidden2label.weight', 'hidden2label.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
[Train Epoch 0]: 0%| | 0/1584 [00:00<?, ?it/s]
Traceback (most recent call last):
File "demo_train.py", line 66, in
run()
File "demo_train.py", line 53, in run
loss = model.neg_log_likelihood(input_ids, attention_mask, label_ids, real_lens)
File "/home/z/code/nlp-notebook-master/3-2.Bert-CRF/model.py", line 137, in neg_log_likelihood
feats = self.get_features(input_ids, attention_mask)
File "/home/z/code/nlp-notebook-master/3-2.Bert-CRF/model.py", line 53, in get_features
sequence_output, pooled_output = x.last_hidden_state, x.pooler_output
AttributeError: 'tuple' object has no attribute 'last_hidden_state'
输出如上,尝试修改model.from_pretrained(model_path,output_hidden_states = True)也不行
请问是哪里出了问题?环境配置是一样的
请问在Seq2Seq模型中,把trg改为tensor类型后,出现以下报错:
Traceback (most recent call last):
File "E:\nlp-notebook-master\4-1.Seq2seq\train_eval.py", line 54, in
trg, src = trg.to(device), src.to(device)
AttributeError: 'NoneType' object has no attribute 'to'
是怎么回事呢?
您好,跑您p-tuning代码,遇见一个RuntimeError: _th_ceil_out not supported on CUDAType for Long,猜测是不是mlm_pytorch包的版本问题,想问一下您的版本和环境~谢谢!
Traceback (most recent call last):
File "G:/download/yg/nlp-notebook-master/5.PaperwithCode/3.P-tuning/train.py", line 30, in
loss = model(batch_data[0], batch_data[1])
File "C:\Users\Adam-CVTeam\Anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 547, in call
result = self.forward(*input, **kwargs)
File "G:\download\yg\nlp-notebook-master\5.PaperwithCode\3.P-tuning\model.py", line 77, in forward
inputs_embeds = self.embed_input(queries) #[batch size, spell_length + x, hidden_size]
File "G:\download\yg\nlp-notebook-master\5.PaperwithCode\3.P-tuning\model.py", line 45, in embed_input
replace_embeds = self.prompt_encoder() #[spell_length, hidden_size]
File "C:\Users\Adam-CVTeam\Anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 547, in call
result = self.forward(*input, **kwargs)
File "G:\download\yg\nlp-notebook-master\5.PaperwithCode\3.P-tuning\prompt_encoder.py", line 53, in forward
output_embeds = self.mlm_head(input_embeds)[0].squeeze() # [9(sum(template)), hidden_size]
File "C:\Users\Adam-CVTeam\Anaconda\envs\pytorch\lib\site-packages\torch\nn\modules\module.py", line 547, in call
result = self.forward(*input, **kwargs)
File "C:\Users\Adam-CVTeam\Anaconda\envs\pytorch\lib\site-packages\mlm_pytorch\mlm_pytorch.py", line 67, in forward
mask = get_mask_subset_with_prob(~no_mask, self.mask_prob)
File "C:\Users\Adam-CVTeam\Anaconda\envs\pytorch\lib\site-packages\mlm_pytorch\mlm_pytorch.py", line 23, in get_mask_subset_with_prob
mask_excess = (mask.cumsum(dim=-1) > (num_tokens * prob).ceil())
RuntimeError: _th_ceil_out not supported on CUDAType for Long
Process finished with exit code 1
在4-1.Seq2seq中:
for trg, src in pbar:
trg, src = trg.to(device), src.to(device)
trg和src是tuple类型的
hi作者你好,今天发现你这个仓库下的Lattice-LSTM不见了,可否再上传一下呢
提取码不正确
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.