woooooody/neu

Neu nlp toolkit

Neu

中文自然语言处理工具包--东北大学自然语言处理实验室(http://www.nlplab.com)

Neu是东北大学自然语言处理实验室在2014开发的NiuParser的Python版本.

NiuParser(http://www.niuparser.com), NiuParser支持所有的中文自然语言处理底层技术.

Neu初步预计有四个模块,分别是分词,词性标注,实体识别,句法分析.

安装方便,不依赖于繁多的包,敬请期待.!

Installation

Using pip::

pip3 install Neu (目前只支持Python3)

Using git::

clone https://github.com/Woooooody/Neu (解压后放在python的site package目录)

Useage

License

		Neu SDK   					                    

Copyright (c) 2017-2018 NiuParser All rights reserved.

		www.nlplab.com        

Authors and Contributors

Original authors are Wu Jinhang [email protected] and Huxiao [email protected].

Method

For word segment we use the maximum forward match algorithm and HMM(Hidden Markov Model) Using viterbi algorithm to calculate the maximum rotate.

For sequence label model, we use the character level inputs for LSTM(Long Short Time Memory) and CRF(Conditional Random Field) model .

neu's Issues

UnicodeDecodeError: 'gbk' codec can't decode byte 0xad in position 8: illegal multibyte sequence

   ERROR: Command errored out with exit status 1:
     command: 'c:\program files (x86)\microsoft visual studio\shared\python37_64\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\16413\\AppData\\Local\\Temp\\pip-install-gnj_vr4b\\Neu\\setup.py'"'"'; __file__='"'"'C:\\Users\\16413\\AppData\\Local\\Temp\\pip-install-gnj_vr4b\\Neu\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: C:\Users\16413\AppData\Local\Temp\pip-install-gnj_vr4b\Neu\
    Complete output (5 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\16413\AppData\Local\Temp\pip-install-gnj_vr4b\Neu\setup.py", line 10, in <module>
        long_description=open('README.rst').read(),
    UnicodeDecodeError: 'gbk' codec can't decode byte 0xad in position 8: illegal multibyte sequence
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.