Comments (16)
E.g. if a word (北大) is not recognized as an organisation, could we add this word to let the model know this word?
of course, you can add 北/B_org 大/E_org into the train set.
from chinesener.
@buppt thanks 有个疑问 tensorflow那个是 先python train.py 然后再python train.py pretrained 吗
from chinesener.
@buppt thanks 有个疑问 tensorflow那个是 先python train.py 然后再python train.py pretrained 吗
不用,python train.py是不使用预训练词向量的训练,python train.py pretrained 是使用预训练的词向量训练。
from chinesener.
恩 O(∩_∩)O谢谢 还有个疑问 这个怎么增量数量的 每次的新句子都要加在前面那个训练集吗 然后重新跑一次train吗
from chinesener.
@buppt 这个train.py 会被执行吗
elif len(sys.argv)==3: 看了很久都没有看到过有输入3个参数的 冗余代码吗 谢谢
from chinesener.
恩 O(∩_∩)O谢谢 还有个疑问 这个怎么增量数量的 每次的新句子都要加在前面那个训练集吗 然后重新跑一次train吗
什么意思,是想自己加一些实体的例句?放训练集里或者在训练好的模型基础上继续训练都可以。
三个参数不是文件名那个么,readme里有。
from chinesener.
@buppt 恩 原来是我看漏了 原来还有个文件批处理的 谢谢 , 有比较详细的步骤 ,现在我已经跑完了train.py 如果要加新的语料训练 在现在模型基础 继续训练 要执行那个命令呢 谢谢
from chinesener.
谁能提供一下TensorFlow训练的模型
from chinesener.
你训练不了吗 用显卡大概3个小时
from chinesener.
说一下我遇到的问题哈,
cd data/renMinRiBao/
python data_renmin_word.py
然后 cd tensorflow/
python train.py pretrained
然后报错如下:
train len: 24271
test len: 7585
word2id len 3917
Creating the data generator ...
Finished creating the data generator.
use pretrained embedding
begin to train...
Traceback (most recent call last):
File "train.py", line 107, in
model = Model(config,embedding_pre,dropout_keep=0.5)
File "/home/liyang22/github/ChineseNER/tensorflow/bilstm_crf.py", line 20, in init
self._build_net()
File "/home/liyang22/github/ChineseNER/tensorflow/bilstm_crf.py", line 56, in _build_net
self.viterbi_sequence, viterbi_score = tf.contrib.crf.crf_decode(bilstm_out, self.transition_params,tf.tile(np.array([self.sen_len]),np.array([self.batch_size])))
File "/home/liyang22/tensorflow/local/lib/python2.7/site-packages/tensorflow/contrib/crf/python/ops/crf.py", line 537, in crf_decode
false_fn=_multi_seq_fn)
File "/home/liyang22/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/layers/utils.py", line 206, in smart_cond
pred, true_fn=true_fn, false_fn=false_fn, name=name)
File "/home/liyang22/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/smart_cond.py", line 56, in smart_cond
return false_fn()
File "/home/liyang22/tensorflow/local/lib/python2.7/site-packages/tensorflow/contrib/crf/python/ops/crf.py", line 501, in _multi_seq_fn
sequence_length_less_one = math_ops.maximum(0, sequence_length - 1)
File "/home/liyang22/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_math_ops.py", line 4602, in maximum
"Maximum", x=x, y=y, name=name)
File "/home/liyang22/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 546, in _apply_op_helper
inferred_from[input_arg.type_attr]))
TypeError: Input 'y' of 'Maximum' Op has type int64 that does not match type int32 of argument 'x'.
from chinesener.
@bobkentt 你看一下你的 语料是不是有问题 是你自己编写的吗
from chinesener.
@bobkentt 你看一下你的 语料是不是有问题 是你自己编写的吗
就是把项目直接clone下去啊,没用自己的语料,难到是我TensorFlow版本的问题?你是啥版本的啊?我这俩虚拟机安装的tf环境,版本分别是:1.10.0 1.12.0 都不行
from chinesener.
train.py 中改成int64也不行,同时也试了把数据label强转成int32
from chinesener.
你重新训练前 有将前面训练好的模型文件删掉吗 我用的是tensorflow-gpu==1.10.0
from chinesener.
from chinesener.
@bobkentt 类型转为int32就可以了
self.viterbi_sequence, viterbi_score = tf.contrib.crf.crf_decode(tf.cast(bilstm_out, dtype=tf.int32), tf.cast(self.transition_params, dtype=tf.int32), tf.cast(sequence_length, dtype=tf.int32))
from chinesener.
Related Issues (20)
- pytorch中运行train.py的问题 HOT 6
- “这篇文章” HOT 2
- 如果用來做part of the speech tagger 可以嗎?
- 关于pytorch BiLSTM_CRF模型中EMBEDDING_DIM和HIDDEN_DIM的设置 HOT 1
- 关于Pytorch版本中加入Batch
- 您好
- 博主您好
- 博主您好,想问您关于编码格式运行报错的问题,按照网上的方法改了很多次,还是出错。 HOT 1
- 博主您好,在eval.py中有一小句代码(tag='0' if tag=='O' else tag)看不太懂,博主您是否能解释一下呢。这里都把tag的值赋为0了为什么还要检测是不是为'O'呢?
- 关于 train2.pkl 中from compiler.ast import flatten的问题 HOT 1
- IOError: [Errno 2] No such file or directory: './model/model0.pkl' HOT 2
- 并发情况下,响应时间很慢
- 关系抽取 HOT 1
- pytorch版本模型测试代码 HOT 3
- 大佬谁能给一下tensorflow版本的运行步骤
- tensorflow版本处理数据问题
- 这种数据处理方法算是什么名字呢?
- pytorch pkl编码报错 HOT 3
- 更换Boson数据集pkl
- pytorch 和 tensorflow都是这行word2id = pickle.load(inp)报错
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chinesener.