fanhuaandluomu/bilstm_cnn_crf_cws

BiLstm+CNN+CRF 法律文档（合同类案件）领域分词（100篇标注样本）

BiLstm_CNN_CRF_CWS

BiLstm+CNN+CRF 在线演示

note: 实验基于

anaconda py3.5

tensorflow==1.0.1

keras==2.0.6

keras_contrib==2.0.8 pip install git+https://www.github.com/keras-team/keras-contrib.git

cuda==v8.0

gpu==GTX750Ti

简要介绍

1. 使用 bilstm+cnn+crf训练seq2seq模型
2. 预训练词向量 gensim
3. 段落有的太长，按简单标点切分为句子，maxlen控制在100+,不足maxlen前面补0
4. 测试也是按句子测试，最后还原成段落
5. 有机会写个blog，先准备过年~新年快乐！_

step1: 法律文档+conll2012分词语料 训练word embedding

python embedding_model.py

step2: 预处理+训练+测试

1. 随机抽80篇训练（10%用于验证集），20篇用于测试   实验10次，平均f-score=0.953,详见prf_result_max_epoch_50_em.txt

2. 随机抽50篇训练（10%用于验证集），50篇用于测试   实验10次，平均f-score=0.933,详见prf_result_max_epoch_50_law.txt

3. 用conll2012中6个领域的分词训练语料+法律文档训练语料（20篇）-> 法律80篇测试   时间问题，只测一次：f-score:0.943

4. 用conll2012中6个领域的分词训练语料-> 法律100篇测试   时间问题，只测一次：f-score:0.757

New 拖了好久，终于在毕业论文交（3.26）后写了一篇分词blog（虽然也没啥技术含量，写着玩..）

   基于BiLSTM-CNN-CRF的中文分词

simple_bilstm_model

程序写的太繁琐，简化了一下  只关心 pip install keras==2.0.6  深度学习分词算法的可以只看这个文件夹下的bilstm_cnn_crf.py程序 keras_contrib==2.0.8 pip install git+https://www.github.com/keras-team/keras-contrib.git   pip install gensim
如缺少其它模块，看报错自行安装  

## note
# 把你的语料放到corpus文件夹下  我的corpus中的语料压缩了，如使用可以解压
# 1. python embedding_model.py  -> model_conll_law.m  生成词向量文件
# 2. python bilstm_cnn_crf.py    // is_train==1
# 会得到 train_model.hdf5  lexicon.pkl
# 3. 可以在之前的基础上train_model.hdf5，继续训练
# 4. 训练完成，测试  is_train==0
# python bilstm_cnn_crf.py  按句测试或按文件测试

# my_weights 中存放的是我的权值 

关于simple_bilstm_model程序的运行，写了个讲解，详见

BiLSTM_CNN_CRF分词程序—运行讲解-简书
BiLSTM_CNN_CRF分词程序—运行讲解-知乎

simple_bilstm_model 百度网盘下载：链接：https://pan.baidu.com/s/1b0WRe16aVVILYGEBmhB9lg 密码：9tiv

不想下载全部项目的可以只下载网盘的内容。

bilstm_cnn_crf_cws's Issues

AttributeError: 'CRF' object has no attribute '_inbound_nodes

2018-04-17 14:11:48.703574: I tensorflow/core/platform/cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
WARNING:tensorflow:From /usr/local/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py:1188: calling reduce_sum (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
bilstm_cnn_crf.py:59: UserWarning: Update your Conv1D call to the Keras 2 API: Conv1D(padding="valid", kernel_size=5, filters=50)
conv=Conv1D(nb_filter=50,filter_length=2*half_window_size+1,padding='valid')(padding_layer)
WARNING:tensorflow:From /usr/local/lib/python3.5/site-packages/tensorflow/python/util/deprecation.py:497: calling conv1d (from tensorflow.python.ops.nn_ops) with data_format=NHWC is deprecated and will be removed in a future version.
Instructions for updating:
NHWC for data_format is deprecated, use NWC instead
bilstm_cnn_crf.py:64: UserWarning: The merge function is deprecated and will be removed after 08/2017. Use instead layers from keras.layers.merge, e.g. add, concatenate, etc.
rnn_cnn_merge=merge([bilstm_d,dense_conv],mode='concat',concat_axis=2)
/usr/local/lib/python3.5/site-packages/keras/legacy/layers.py:458: UserWarning: The Merge layer is deprecated and will be removed after 08/2017. Use instead layers from keras.layers.merge, e.g. add, concatenate, etc.
name=name)
WARNING:tensorflow:From /usr/local/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py:1171: calling reduce_min (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
bilstm_cnn_crf.py:72: UserWarning: Update your Model call to the Keras 2 API: Model(inputs=[<tf.Tenso..., outputs=[<tf.Tenso...)
model = Model(output=[crf_output], input=[word_input])
Traceback (most recent call last):
File "bilstm_cnn_crf.py", line 458, in
main()
File "bilstm_cnn_crf.py", line 436, in main
process_train(corpus_path,nb_epoch,base_model_weight)
File "bilstm_cnn_crf.py", line 248, in process_train
model=Bilstm_CNN_Crf(max_len,len(lexicon),len(label_2_index),embedding_weights)
File "bilstm_cnn_crf.py", line 74, in Bilstm_CNN_Crf
model.compile(loss=crf.loss_function,optimizer='adam',metrics=[crf.accuracy])
File "/usr/local/lib/python3.5/site-packages/keras/engine/training.py", line 850, in compile
sample_weight, mask)
File "/usr/local/lib/python3.5/site-packages/keras/engine/training.py", line 450, in weighted
score_array = fn(y_true, y_pred)
File "/usr/local/lib/python3.5/site-packages/keras_contrib/layers/crf.py", line 287, in loss
assert self._inbound_nodes, 'CRF has not connected to any layer.'

您好，请问conll2012分词语料 在哪下载？

为什么会出现没有过的转移

做序列标注 用的OBIE 为什么会出现I O的这种情况 数据里面没有这种情况 这个CRF的实现确定没问题吗？

ImportError: cannot import name normalize_data_format

ub16hp@UB16HP:/ub16_prj/BiLstm_CNN_CRF_CWS/simple_bilstm_model$ python bilstm__cnn_crf.py
Using TensorFlow backend.
Traceback (most recent call last):
File "bilstm__cnn_crf.py", line 27, in
from keras_contrib.layers import CRF
File "/usr/local/lib/python2.7/dist-packages/keras_contrib/init.py", line 4, in
from . import layers
File "/usr/local/lib/python2.7/dist-packages/keras_contrib/layers/init.py", line 3, in
from .convolutional import *
File "/usr/local/lib/python2.7/dist-packages/keras_contrib/layers/convolutional.py", line 15, in
from keras.backend.common import normalize_data_format
ImportError: cannot import name normalize_data_format
ub16hp@UB16HP:/ub16_prj/BiLstm_CNN_CRF_CWS/simple_bilstm_model$

不知道如何解决。求指点

Traceback (most recent call last):
File "bilstm__cnn_crf.py", line 459, in
main()
File "bilstm__cnn_crf.py", line 444, in main
model=Bilstm_CNN_Crf(max_len,len(lexicon),len(index_2_label),is_train=False)
File "bilstm__cnn_crf.py", line 53, in Bilstm_CNN_Crf
bilstm=Bidirectional(LSTM(64,return_sequences=True))(word_emb)
File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/layers/wrappers.py", line 427, in call
return super(Bidirectional, self).call(inputs, **kwargs)
File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/engine/base_layer.py", line 457, in call
output = self.call(inputs, **kwargs)
File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/layers/wrappers.py", line 522, in call
y = self.forward_layer.call(inputs, **kwargs)
File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/layers/recurrent.py", line 2194, in call
initial_state=initial_state)
File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/layers/recurrent.py", line 649, in call
input_length=timesteps)
File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 3011, in rnn
maximum_iterations=input_length)
TypeError: while_loop() got an unexpected keyword argument 'maximum_iterations'
ERROR:tensorflow:==================================
Object was never used (type <class 'tensorflow.python.ops.tensor_array_ops.TensorArray'>):
<tensorflow.python.ops.tensor_array_ops.TensorArray object at 0x7fc6741a0f28>
If you want to mark it as used call its "mark_used()" method.
It was originally created here:
['File "bilstm__cnn_crf.py", line 459, in \n main()', 'File "bilstm__cnn_crf.py", line 444, in main\n model=Bilstm_CNN_Crf(max_len,len(lexicon),len(index_2_label),is_train=False)', 'File "bilstm__cnn_crf.py", line 53, in Bilstm_CNN_Crf\n bilstm=Bidirectional(LSTM(64,return_sequences=True))(word_emb)', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/layers/wrappers.py", line 427, in call\n return super(Bidirectional, self).call(inputs, **kwargs)', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/engine/base_layer.py", line 457, in call\n output = self.call(inputs, **kwargs)', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/layers/wrappers.py", line 522, in call\n y = self.forward_layer.call(inputs, **kwargs)', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/layers/recurrent.py", line 2194, in call\n initial_state=initial_state)', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/layers/recurrent.py", line 649, in call\n input_length=timesteps)', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 2931, in rnn\n input_ta = input_ta.unstack(inputs)', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py", line 175, in wrapped\n return _add_should_use_warning(fn(*args, **kwargs))', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/tensorflow/python/ops/tensor_array_ops.py", line 413, in unstack\n indices=math_ops.range(0, num_elements), value=value, name=name)', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py", line 175, in wrapped\n return _add_should_use_warning(fn(*args, **kwargs))', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py", line 144, in _add_should_use_warning\n wrapped = TFShouldUseWarningWrapper(x)', 'File "/home/wll/anaconda3/lib/python3.5/site-packages/tensorflow/python/util/tf_should_use.py", line 101, in init\n stack = [s.strip() for s in traceback.format_stack()]']

embedding_model的疑问

同学，你好。关于embedding_model.py代码我有个小疑问。代码中用genism做中文词向量的。但是你代码好像将语料库中本来已空格分好的词又replace了，最终输出的是字的向量。不知道是不是我理解有误。

for dirname in self.dirname_list:
	for fname in os.listdir(dirname):
		for line in codecs.open(dirname+os.sep+fname,'r','utf-8'):
			pieces=line.strip().replace(' ','')
			words=[w for w in pieces]
			yield words

pieces这块你把空格给replace了。

FileNotFoundError: [Errno 2] No such file or directory: 'corpus'

mldl@mldlUB1604:/ub16_prj/BiLstm_CNN_CRF_CWS$ python3 embedding_model.py
Using TensorFlow backend.
Traceback (most recent call last):
File "embedding_model.py", line 20, in
sentences=MySentences([corpus_path+os.sep+fname for fname in os.listdir(corpus_path)])
FileNotFoundError: [Errno 2] No such file or directory: 'corpus'
mldl@mldlUB1604:/ub16_prj/BiLstm_CNN_CRF_CWS$

请教模型结构？

请问Bilstm_CNN_Crf()中LSTM的输出64是如何确定的？
LSTM(64, return_sequences=True)

请问CRF层是出了什么问题吗？ 感谢

InvalidArgumentError (see above for traceback): Matrix size-incompatible: In[0]: [3168,1], In[1]: [4,4]
[[Node: loss/crf_1_loss/MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/cpu:0"](loss/crf_1_loss/Reshape_3, loss/crf_1_loss/Reshape_4)]]

请问CRF层是出了什么问题吗？ 感谢

报ImportError: No module named 'lss_fenci_api' 错误，请问没有这个文件么？

Traceback (most recent call last):
File "E:/IDEA/IDEA/python/BiLstm_CNN_CRF_CWS-master/BiLstm_CNN_CRF_CWS-master/wechat_search.py", line 7, in
from fenci_server import word_seg
File "E:\IDEA\IDEA\python\BiLstm_CNN_CRF_CWS-master\BiLstm_CNN_CRF_CWS-master\fenci_server.py", line 391, in
from lss_fenci_api import samme_cws
ImportError: No module named 'lss_fenci_api'

大佬你切好的文书语料哪儿找的

论文

您好，您这个实验对应论文题目叫什么，想拜读一下，谢谢

为什么会出现没有过的转移

做序列标注 用的OBIE 为什么会出现I O的这种情况 数据里面没有这种情况 这个CRF的实现确定没问题吗？