index_ansj分词疑惑

[nlpchina]

index_ansj分词疑惑

可能你配置的ansj插件没有生效

[adonis2014]

@hrdxwandg 我也发现了这个问题。index_ansj 感觉包含了单字分词，和作者测试例子确实不一样，不知道什么问题。

{"tokens":[{"token":"六味地黄","start_offset":0,"end_offset":4,"type":"word","position":0},{"token":"六味","start_offset":0,"end_offset":2,"type":"word","position":1},{"token":"六","start_offset":0,"end_offset":1,"type":"word","position":2},{"token":"味","start_offset":1,"end_offset":2,"type":"word","position":3},{"token":"地黄丸","start_offset":2,"end_offset":5,"type":"word","position":4},{"token":"地黄","start_offset":2,"end_offset":4,"type":"word","position":5},{"token":"地","start_offset":2,"end_offset":3,"type":"word","position":6},{"token":"黄丸软","start_offset":3,"end_offset":6,"type":"word","position":7},{"token":"黄","start_offset":3,"end_offset":4,"type":"word","position":8},{"token":"丸","start_offset":4,"end_offset":5,"type":"word","position":9},{"token":"软胶囊","start_offset":5,"end_offset":8,"type":"word","position":10},{"token":"软胶","start_offset":5,"end_offset":7,"type":"word","position":11},{"token":"软","start_offset":5,"end_offset":6,"type":"word","position":12},{"token":"胶囊","start_offset":6,"end_offset":8,"type":"word","position":13},{"token":"胶","start_offset":6,"end_offset":7,"type":"word","position":14},{"token":"囊","start_offset":7,"end_offset":8,"type":"word","position":15}]}

[ansjsun]

你们打开自定义词典..就是 default.dic 词典.把里面的单字删掉..就可以了... 是因为index分词方式是基于..词典来决定显示哪些词语的

…

On 7 Dec 2016, at 4:00 PM, adonis2014 ***@***.*** ***@***.***>> wrote: @hrdxwandg <https://github.com/hrdxwandg> 我也发现了这个问题。index_ansj 感觉包含了单字分词，和作者测试例子确实不一样，不知道什么问题。 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#42 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABKlOzqwUKU09XAwX227Kp4JpYvAXZYSks5rFmeRgaJpZM4LGHbr>.

[adonis2014]

谢谢ansjsun回复。将配置修改成

enabled_stop_filter: false
enable_skip_user_define: true
就可以了。

[hrdxwandg]

@adonis2014
你的这个方法对“六味地黄丸软胶囊” OK。
但是如果对“中华人民共和国”进行分词，只能分出“中华人民共和国”和“人民共和国”两个了。

index_ansj的粒度应该是合适范围内尽量小。

[ansjsun]

@hrdxwandg enable_skip_user_define: true
这个去掉...

[hrdxwandg]

@ansjsun 去掉enable_skip_user_define: true也不行。

[tanyf003]

5.0.2 版本 同样的问题， 也不行，该怎么配置呢

[ansjsun]

吧用户自定义词典的单字去掉..应该在default.dic里面...

去掉单字.