ttys3/gojieba-bleve

GoJieba Bleve support

GoJieba Bleve support

Intro

GoJieba bleve support mod

this repo exists because the original mod has
removed bleve support

Get the mod

go get github.com/ttys3/gojieba-bleve/v2

Usage

import it like this to register the gojieba Tokenizer and Analyzer

_ "github.com/ttys3/gojieba-bleve/v2"

please see bleve_test.go

gojieba-bleve's Issues

安装报错

In file included from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/Unicode.hpp:9,
from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/DictTrie.hpp:15,
from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/QuerySegment.hpp:8,
from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/Jieba.hpp:4,
from jieba.cpp:5:
....\pkg\mod\github.com\yanyiwu\[email protected]\deps/limonp/LocalVector.hpp: In instantiation of 'void limonp::LocalVector::reserve(size_t) [with T = std::pair<long long unsigned int, const cppjieba::DictUnit*>; size_t = long long unsigned int]':
....\pkg\mod\github.com\yanyiwu\[email protected]\deps/limonp/LocalVector.hpp:83:7: required from 'void limonp::LocalVector::push_back(const T&) [with T = std::pair<long long unsigned int, const cppjieba::DictUnit*>]'
....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/Trie.hpp:99:81: required from here
....\pkg\mod\github.com\yanyiwu\[email protected]\deps/limonp/LocalVector.hpp:95:11: warning: 'void* memcpy(void*, const void*, size_t)' writing to an object of type 'struct std::pair<long long unsigned int, const cppjieba::DictUnit*>' with no trivial copy-assignment; use copy-assignment or copy-initialization instead [-Wclass-memaccess]
memcpy(ptr_, old, sizeof(T) * capacity_);
~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/utility:70,
from C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/algorithm:60,
from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/QuerySegment.hpp:4,
from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/Jieba.hpp:4,
from jieba.cpp:5:
C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/bits/stl_pair.h:198:12: note: 'struct std::pair<long long unsigned int, const cppjieba::DictUnit*>' declared here
struct pair
^~~~
In file included from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/Unicode.hpp:9,
from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/DictTrie.hpp:15,
from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/QuerySegment.hpp:8,
from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/Jieba.hpp:4,
from jieba.cpp:5:
....\pkg\mod\github.com\yanyiwu\[email protected]\deps/limonp/LocalVector.hpp: In instantiation of 'limonp::LocalVector& limonp::LocalVector::operator=(const limonp::LocalVector&) [with T = std::pair<long long unsigned int, const cppjieba::DictUnit*>]':
....\pkg\mod\github.com\yanyiwu\[email protected]\deps/limonp/LocalVector.hpp:33:11: required from 'limonp::LocalVector::LocalVector(const limonp::LocalVector&) [with T = std::pair<long long unsigned int, const cppjieba::DictUnit*>]'
....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/Trie.hpp:28:8: required from 'void std::_Construct(_T1*, _Args&& ...) [with _T1 = cppjieba::Dag; _Args = {const cppjieba::Dag&}]'
C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/bits/stl_uninitialized.h:83:18: required from 'static _ForwardIterator std::__uninitialized_copy<_TrivialValueTypes>::__uninit_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = const cppjieba::Dag*; _ForwardIterator = cppjieba::Dag*; bool _TrivialValueTypes = false]'
C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/bits/stl_uninitialized.h:134:15: required from '_ForwardIterator std::uninitialized_copy(_InputIterator, _InputIterator, _ForwardIterator) [with _InputIterator = const cppjieba::Dag*; _ForwardIterator = cppjieba::Dag*]'
C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/bits/stl_uninitialized.h:289:37: required from '_ForwardIterator std::__uninitialized_copy_a(_InputIterator, _InputIterator, _ForwardIterator, std::allocator<_Tp>&) [with _InputIterator = const cppjieba::Dag*; _ForwardIterator = cppjieba::Dag*; _Tp = cppjieba::Dag]'
C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/bits/stl_uninitialized.h:311:2: required from '_ForwardIterator std::__uninitialized_move_if_noexcept_a(_InputIterator, _InputIterator, _ForwardIterator, _Allocator&) [with _InputIterator = cppjieba::Dag*; _ForwardIterator = cppjieba::Dag*; _Allocator = std::allocatorcppjieba::Dag]'
C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/bits/vector.tcc:611:7: required from 'void std::vector<_Tp, _Alloc>::_M_default_append(std::vector<_Tp, _Alloc>::size_type) [with _Tp = cppjieba::Dag; _Alloc = std::allocatorcppjieba::Dag; std::vector<_Tp, _Alloc>::size_type = long long unsigned int]'
C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/bits/stl_vector.h:827:4: required from 'void std::vector<_Tp, _Alloc>::resize(std::vector<_Tp, _Alloc>::size_type) [with _Tp = cppjieba::Dag; _Alloc = std::allocatorcppjieba::Dag; std::vector<Tp, Alloc>::size_type = long long unsigned int]'
....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/Trie.hpp:86:27: required from here
....\pkg\mod\github.com\yanyiwu\[email protected]\deps/limonp/LocalVector.hpp:63:13: warning: 'void* memcpy(void*, const void*, size_t)' writing to an object of type 'struct std::pair<long long unsigned int, const cppjieba::DictUnit*>' with no trivial copy-assignment; use copy-assignment or copy-initialization instead [-Wclass-memaccess]
memcpy(ptr, vec.ptr, vec.size() * sizeof(T));
~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/utility:70,
from C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/algorithm:60,
from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/QuerySegment.hpp:4,
from ....\pkg\mod\github.com\yanyiwu\[email protected]\deps/cppjieba/Jieba.hpp:4,
from jieba.cpp:5:
C:/mingw32/lib/gcc/i686-w64-mingw32/8.1.0/include/c++/bits/stl_pair.h:198:12: note: 'struct std::pair<long long unsigned int, const cppjieba::DictUnit*>' declared here
struct pair

支持一下bleve2.3.5

rt

Can u update the support to bleve/v2?

Thanks a lot.

blevetest.go:93:73: undefined: JiebaTokenizer

I am the go rookie so I follow the test go test
But it's hard for me countinue go through
Am I miss some thing ?

the command go test blevetest.go