hightman/xunsearch

免费开源的中文搜索引擎，采用 C/C++ 编写 (基于 xapian 和 scws)，提供 PHP 的开发接口和丰富文档

xunsearch's Issues

hightman 看能不能出个YII2的版本呢

如题 现在好多都转向YII2了

安装出错

git clone 安装

follow this link:
http://www.xunsearch.com/doc/php/guide/start.installation

configure: error: event2/event.h NOT FOUND. please check your libevent install directory. >= 2.0.0

我用的是 anaconda,
libevent 2.0.22 0 anaconda

不知为何

建议加入in查询

现在已经有了区间搜索，但是对于in查询( field in (a,b,c) ) 的查询还是不方便，好像只能通过多个OR来实现，略微麻烦

xapian-swcs 补丁版本更新

目前版本是基于 xapian 1.2.22 的。但是1.2分支的xapian最新版本是1.2.25。应该尽快更新。

我发现这个patch主要的问题是，它基于一个打包准备过程中生成的文件 queryparser_internal.cc，其实该文件的原始文件应该是 queryparser.lemony，通过 lemon 生成的cc文件。应该另开一个与上游同步的xapian分支，以便随时进行更新。

谢谢。

xunsearch 不支持groupby吗？

php util/Indexer.php --project=course --rebuild --source=mysql://root:*******@127.0.0.1:3306/***** --sql="select c.id,subject_id,c.detail,course_name,c.cover,price,edu_fund_enable,c.hot,c.appraise_nums,c.appraise_score,c.student_nums,o.organ_name,f.name fitage_name,s.name subject_name from ue_course c left join ue_organ o on c.organ_id=o.id left join ue_fitage f on c.fitage_id=f.id left join ue_subject s on c.subject_id=s.id left join ue_course_mark cm on c.id=cm.course_id left join ue_mark m on m.id=cm.mark_id where cm.course_id=c.id group by cm.id"  

以上sql在mysql上没问题可以查询到记录，但是创建索引时显示只导入一条记录

vendor\hightman\xunsearch\lib\XS.class.php 构造函数缺少异常处理

public function __construct($file)
    {
        if (strlen($file) < 255 && !is_file($file)) {
            $appRoot = getenv('XS_APP_ROOT');
            if ($appRoot === false) {
                $appRoot = defined('XS_APP_ROOT') ? XS_APP_ROOT : XS_LIB_ROOT . '/../app';
            }
            $file2 = $appRoot . '/' . $file . '.ini';
            if (is_file($file2)) {
                $file = $file2;
            } else {
                throw new \Exception(sprintf('File [%s] is not exist!', $file)); 
            }
        }
        $this->loadIniFile($file);
        self::$_lastXS = $this;
    }

我修改了类名，忘了修改配置名，结果查了好久才找到原因。这里判断配置不存在的情况，抛出异常。

有没有计划增加nodejs的SDK？

如题

官网下载下来还是 1.4.9 的，github下载的1.4.11release包只有几百k？

dockerfile建议

多个RUN可以用&&合并，每个RUN会多一层，pull的时候会层数过多.

RUN echo "#!/bin/sh" > bin/xs-docker.sh
RUN echo "rm -f tmp/pid.*" >> bin/xs-docker.sh
RUN echo "echo -n > tmp/docker.log" >> bin/xs-docker.sh

RUN echo "#!/bin/sh" > bin/xs-docker.sh \
    && echo "rm -f tmp/pid.*" >> bin/xs-docker.sh \
    && echo "echo -n > tmp/docker.log" >> bin/xs-docker.sh

建议highlight传递html tag，不要hard code在库文件中

对于用户来说方便控制highlight效果，不要让用户去直接动库文件，库升级后也不受影响。

怎样做全站搜索？

站内有文章、问答等多个模块
怎样做全站的搜索？

还是针对每个模块都建立自己的project， 在全站搜索的时候自己做处理，把分块的搜索数据进行处理汇总展示？
望指教

xunsearch是否支持同一个字段的两次range搜索

xunsearch是否支持同一个字段的两次range搜索？比如，有个字段是age，我能否这样搜索：10 < age < 15 and 20 < age < 25?

xunsearch可以支持类似mysql的order by a+b的排序方式吗？setMultiSort方法貌似不行

安装过程时间很长，然后无响应，也不报错

在CentOS 6.6 x32上安装，默认配置，很长时间后还卡在Compiling & installing xapian-core-scws，也不报错，CPU，内存占用正常，不知道什么原因

[code]
[root@centos-web xunsearch-full-1.4.9]# sudo ./setup.sh

+==========================================+
| Welcome to setup xunsearch(full) |
| 欢迎使用 xunsearch (完整版) 安装程序 |
+------------------------------------------+
| Follow the on-screen instructions please |
| 请按照屏幕上的提示操作以完成安装 |
+==========================================+

Please specify the installation directory
请指定安装目录 (默认为中括号内的值)
[/usr/local/xunsearch]:

Confirm the installation directory
请确认安装目录：/usr/local/xunsearch [Y/n]

Checking scws ... 1.2.3-dev
Checking scws dict ... ok
Checking libuuid ... yes: /usr/local/xunsearch
Checking xapian-core-scws ... no
Installing xapian-core-scws (1.2.20) ...
Extracting xapian-core-scws package ...
Configuring xapian-core-scws ...
Compiling & installing xapian-core-scws ...
[/code]

1.4.10使用php脚本建索引会有内存溢出的问题。

PHP Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 2840240 bytes) in /data/wwwroot/www.hongren.com/ThinkPHP/Library/Vendor/Xunsearch/XS.php on line 1379

希望有一篇文档用于描述传输协议。

用于C语言sdk开发。

stopwords.txt没有生效，还会把 “的” 作为搜索词，请问怎么结局

想实现类似 MySQL 的 like 查询功能，不知怎么处理

刚使用这个工具，不知应该如何处理，想直接查询不用分词。描述不清楚的见下面引用中 solr 的解决方案。

If you want to make a search like %asd%, you dont need ngram filters... Basically solr uses ~ for single char and * for multiple chars in their searches...
e.g name:asd~ will match asda asdb asdc and so on while name:asd* will matchasdasadad asdfgkhjfgkl and so on so you can use name: asd to match basdf or easdk and name:asd for aldkaasdsdfsd or oippopasdoppm

stackoverflow

自定义分词报错

Fatal error: Uncaught [XSException] ../../ThinkPHP/Library/Vendor/Xunsearch/XS.php(2439): Task is canceled due to timeout/error(S#514)
thrown in /data/wwwroot/www.hongren.com/ThinkPHP/Library/Vendor/Xunsearch/XS.php on line 2439

在配置文件中配置了
[nickname]
type = title
tokenizer = duality

在XS中实现了方法。把父类的private变量改成了protected了。
class XSTokenizerDuality extends XSTokenizerScws {

public function __construct($arg = null)
{
	parent::__construct($arg);
}

public function getTokens($value, XSDocument $doc = null)
{
	$tokens = array();
	$this->setIgnore(true);
	$this->setDuality(false);
	$_charset = self::$_charset;
	self::$_charset = 'UTF-8';
	$words = $this->getResult($value);
	foreach ($words as $word) {
		$tokens[] = $word['word'];
	}
	self::$_charset = $_charset;
	return $tokens;
}

}

莫名其妙的挂了，不能搜索

xunsearch 1.4.10最新的 centos7.3x64
日志如下：

log.zip

删除数据，重新导入数据好了，稳定性让人崩溃

如何使用in？

所做的项目中，有较多的针对某字段（如cate_id）的in查询，请问除了使用cate_id:1 OR cate_id:2 OR cate_id:3这种“笨方法”外，还有没有更便捷的查询语句？

xs-import -Q -m3 好久好久。。

480w数据
xs-import -Q -m3 data/order/db.re tmp/order_db.snd
导入之后 就一致在运行 这个进程的速度有什么办法可以提高吗？
提高计算机的什么配置 可以提高这个步骤的速度呢
主要是什么制约了这个进程的速度呢？

setLimit 最大只能输出100个结果 无法再多了。

setLimit 最大只能输出100个结果 无法再多了。
XS.php貌似没有限制 是服务本身限制的吗？怎么不限制呢？

XS类有支持in array(a,b,c)的方法吗，文档没有找到。

能否指定返回的字段？

有时候仅仅只是需要获取一两个字段的值即可，如果把整个文档都返回过来就浪费web服务器和搜索服务器之间的带宽

使用 PKGCONFIG 来检测第三方库

libevent和 xapian 都支持使用pkgconfig来进行检测，这样会使得config变得简单。
补丁地址： https://build.opensuse.org/package/view_file/home:fundawang/xunsearch/xunsearch-1.4.11-pkgconfig.patch

setSort排序失效，不起作用

直接所有查可以，加了addQueryTerm，或者关键字查询，setSort不起作用了

php sdk能否改为适应composer安装

rt

小白问问题

您的PHP是怎么调用C的啊？我看了半天没看出来？
是直接调用C的，还是请求的C写的服务器呢？
能给解释一下吗？

变量为n##_var_gl是什么意思呢？

define G_LOCK(n) mm_lock##n(mm_global)

define G_VAR(n) *n##_var_gl

define G_VAR_DECL(n, type) type *n##_var_gl

define G_VAR_DECL_EX(n, type) extern type *n##_var_gl

define G_VAR_PTR(n) n##_var_gl

这些都是什么意思啊？
为什么会有##的符号？
C里面允许有这样的变量名？
不懂。。。

缓存检测

检测程序 不支持php内置的 Zend OPcache
实际运行是不是也不支持呢
// check cache
function check_cache()
{
$rec = array();
if (function_exists('apc_fetch')) {
$rec[] = 'apc';
}
if (function_exists('xcache_get')) {
$rec[] = 'xcache';
}
if (function_exists('eaccelerator_get')) {
$rec[] = 'eAccelerator';
}
if (count($rec) === 0) {
return 'WARNING';
}
return current($rec);
}

安装了xunsearch能否直接用scws分词功能？

我试验过

        $scws = new \XSTokenizerScws();
        $scws->setCharset('utf8');
//         $scws->getResult("what are you doing?");
        $scws->getResult("据悉，2014年吸毒事件后，柯震东参加了不少公益校园活动，希望能透过自己的亲身经历带给更多人正能量。今日，他被曝出与当年一起吸毒被捕的房祖名两人在信义区夜店，与一票小模玩到早上7点才纷纷离开。");
        var_export($scws);

貌似都没有效果？是否要单独安装scws才可以？

不知道是否能支持libevent-2.1.8-stable

系统的最新的BIO结构体不一样了,不能安装libevent-2.0.21.

strtolower导致西欧字符乱码

https://github.com/hightman/xunsearch/blob/master/sdk/php/lib/XSSearch.class.php#L950

一共有三处strtolower，可以修改为mb_strtolower($str,'UTF-8');

如果没有mb扩展，则可以简单的替换特殊字符

比较常见的非标准拉丁字符如下：

德语字母
Ää Öö ß Üü
Aa Oo s Uu

法语字母
Àà Ââ Ææ Çç Éé Èè Êê Ëë Îî Ïï Ôô Œœ Ùù Ûû Üü Ÿÿ
Aa Aa Aa Cc Ee Ee Ee Ee Ii Ii Oo Oo Uu Uu Uu Yy

西班牙语
Áá Éé Íí Ññ Óó Úú Üü
Aa Ee Ii Nn Oo Uu Uu

意大利语
Àà Èè Éé Ìì Íí Îî Òò Óó Ùù Úú
Aa Ee Ee Ii Ii Ii Oo Oo Uu Uu

在什么地方可以调整建立索引的速度

现在用xunsearch，发现建立索引到能搜索，快接近1分钟了，请问如何调整建立索引的速度？

安装时出错ERROR: failed to compile libevent, see 'setup.log' for more detail

Checking xapian-core-scws ... 1.2.22
Checking libevent ... no
Installing libevent (2.0.21-stable) ...
Extracting libevent package ...
Configuring libevent ...
Compiling & installing libevent ...

static BIO_METHOD methods_bufferevent = {
^~~~~~~~~~~~~~~~~~~
Makefile:793: recipe for target 'bufferevent_openssl.lo' failed
make[2]: *** [bufferevent_openssl.lo] Error 1
make[2]: Leaving directory '/home/xq/workspace/xunsearch-full-1.4.10/libevent-2.0.21-stable'
Makefile:857: recipe for target 'install-recursive' failed
make[1]: *** [install-recursive] Error 1
make[1]: Leaving directory '/home/xq/workspace/xunsearch-full-1.4.10/libevent-2.0.21-stable'
Makefile:1182: recipe for target 'install' failed
make: *** [install] Error 2

ERROR: failed to compile libevent, see 'setup.log' for more detail

如何获取一个关键词的搜索次数

官方提供的搜索指数的方法getHotQuery ，最多只能获取到50个词，请问如何查询某一个词的搜索次数呢。另外论坛收不到邮件

Yii2-SDK中的ActiveRecord是否可以将components key设定为变量，不要写死

xunsearch/sdk/php/wrapper/yii2-ext/ActiveRecord.php

Line 57 in 28e3f31

return Yii::$app->get('xunsearch')->getDatabase(static::projectName());

Yii2-SDK中的ActiveRecord是否可以将components name设定为变量，不要写死，这样方便定制的时候不用xunsearch做key的小伙伴。

解决xunsearch/util/Indexer.php --rebuild 导入数据超过200万速度变慢问题

@hightman
经过测试600万数据，越是执行到后面，查询速度越慢，我执行了5天才构造完成，通过后台查询得知，
mysql查询数据，当OFFSET 很大的时候，查询数据会非常慢，每次查询会消耗大量的IO，数据库会根据索引，依次排除前面的2000000条数据，最后得到1000条。
例如：select id,title,content from article limit 1000 offset 2000000 返回数据几十秒
所以，导入速度越来越慢的根本原因是查询数据。
如果把上面的sql改成：select id,title,content from article where id>100000 order by id asc limit 1000 offset 0，
执行速度每次都很快，经过测试，600万数据，15分钟就导入完成。
修改方法：
https://www.cjblog.org/blog/1521431254596
修改后的文件：
https://raw.githubusercontent.com/wcj343169893/xs-sdk-php/master/util/XSDataSource.class.php
希望能有点用处

导入数据库数据时无法使用UNION ALL 或者有什么可替代的方法

导入数据库数据时无法使用UNION ALL 或者有什么可替代的方法
util/Indexer.php --rebuild --source=mysql://root:@localhost/exampletwo --sql="SELECT id, title, content FROM docs UNION ALL SELECT id, title, content FROM documents" --project=demo2
以上是我的cmd，但导入的数据量并不会，我想询问下原因

$search->getLastCount() 总数不正确。

$search->setQuery(‘关键字')->addRange('addtime', $from, $to)->setLimit(100,100)->search();
最后同一页的时候的总数数对的，其他页面 返回的getLastCount是addRange无效的数据总数。

getHotQuery总是没有数据

文档上说要运行一段时间，我已经运行了将近1个月了，一直有调用搜索接口。
但是getHotQuery总是返回空。

opensuse install failed checking for scws_new in -lscws... no

system os information
uname -a
Linux home 3.11.10-17-desktop #1 SMP PREEMPT Mon Jun 16 15:28:13 UTC 2014 (fba7c1f) x86_64 x86_64 x86_64 GNU/Linux
cat /proc/version
Linux version 3.11.10-17-desktop (geeko@buildhost) (gcc version 4.8.1 20130909 [gcc-4_8-branch revision 202388](SUSE Linux) ) #1 SMP PREEMPT Mon Jun 16 15:28:13 UTC 2014 (fba7c1f)

setup.log
configure: loading site script /usr/share/site/x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /usr/bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking how to create a ustar tar archive... gnutar
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking how to print strings... printf
checking for style of include used by make... GNU
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for fgrep... /usr/bin/grep -F
checking for ld used by gcc... /usr/x86_64-suse-linux/bin/ld
checking if the linker (/usr/x86_64-suse-linux/bin/ld) is GNU ld... yes
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking whether ln -s works... yes
checking the maximum length of command line arguments... 1572864
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking how to convert x86_64-unknown-linux-gnu file names to x86_64-unknown-linux-gnu format... func_convert_file_noop
checking how to convert x86_64-unknown-linux-gnu file names to toolchain format... func_convert_file_noop
checking for /usr/x86_64-suse-linux/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for dlltool... no
checking how to associate runtime and link libraries... printf %s\n
checking for ar... ar
checking for archiver @file support... @
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for sysroot... no
checking for mt... mt
checking if mt is a manifest tool... no
checking how to run the C preprocessor... gcc -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... no
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/x86_64-suse-linux/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for g++... g++
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking dependency style of g++... gcc3
checking how to run the C++ preprocessor... g++ -E
checking for ld used by g++... /usr/x86_64-suse-linux/bin/ld -m elf_x86_64
checking if the linker (/usr/x86_64-suse-linux/bin/ld -m elf_x86_64) is GNU ld... yes
checking whether the g++ linker (/usr/x86_64-suse-linux/bin/ld -m elf_x86_64) supports shared libraries... yes
checking for g++ option to produce PIC... -fPIC -DPIC
checking if g++ PIC flag -fPIC -DPIC works... yes
checking if g++ static flag -static works... no
checking if g++ supports -c -o file.o... yes
checking if g++ supports -c -o file.o... (cached) yes
checking whether the g++ linker (/usr/x86_64-suse-linux/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether g++ is a working C++ compiler... yes
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking for g++ options to enable ANSI C++ mode... none required
checking if -lm is required for maths functions... no
checking if RTTI is supported... yes
checking for gettimeofday... yes
checking for ftime... yes
checking for sigaction... yes
checking for sigsetjmp and siglongjmp... yes
checking for getrusage... yes
checking for times... yes
checking for sysconf... yes
checking for setenv... yes
checking for closefrom... no
checking for dirfd... yes
checking for getrlimit... yes
checking return type of ftime... int
checking for sys/utsname.h... yes
checking for gethostname... yes
checking for ssize_t... yes
checking for pid_t... yes
checking for mode_t... yes
checking size of int... 4
checking size of long... 8
checking for perl... /usr/bin/perl
checking for fcntl.h... yes
checking for limits.h... yes
checking for sys/errno.h... yes
checking for sys/select.h... yes
checking for sys/resource.h... yes
checking for valgrind... /usr/bin/valgrind
checking if valgrind supports --log-file with %p... yes
checking for valgrind/memcheck.h... no
checking for working memcmp... yes
checking for strerror... yes
checking for hstrerror... yes
checking for working ISO C90 conforming snprintf... snprintf
checking for zlib.h... yes
checking for library containing zlibVersion... -lz
checking for uuid/uuid.h... yes
checking for library containing uuid_generate... -luuid
checking for fork... yes
checking for library containing socketpair... none required
checking for library containing gethostbyname... none required
checking for library containing gethostbyaddr... none required
checking for type to use for 5th parameter to getsockopt... socklen_t
checking for library containing fdatasync... none required
checking for fsync... yes
checking for pread... yes
checking for any prototype needed for pread... none required
checking for pwrite... yes
checking for any prototype needed for pwrite... none required
checking for link... yes
checking for scws... yes: $HOME/xunsearch
checking for scws_new in -lscws... no
configure: error: scws_new() NOT found in libscws, please check it first.

1.4.11 mb_convert_encoding Unknown encoding

PHP Fatal error: Uncaught [XSErrorException] xunsearch/lib/XS.class.php(567): mb_convert_encoding(): Unknown encoding

搜索语句的长度现在是192 吧，可是还是不够用，不知道可以不可以在给个对外的接口去自定义这个 属性呢？

如题，望尽快回复，谢谢

导入的时候使用了group by,导入的数据量比执行sql语句出来的数据量少很多。

这是正常的现象吗？

centos 5.8 安装失败

直接上安装日志：

mv -f .deps/regress-regress_minheap.Tpo .deps/regress-regress_minheap.Po
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I../compat -I../include -I../include -DTINYTEST_LOCAL     -O2 -Wall -fno-strict-aliasing  -MT regress-regress_thread.o -MD -MP -MF .deps/regress-regress_thread.Tpo -c -o regress-regress_thread.o `test -f 'regress_thread.c' || echo './'`regress_thread.c
mv -f .deps/regress-regress_thread.Tpo .deps/regress-regress_thread.Po
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I../compat -I../include -I../include -DTINYTEST_LOCAL     -O2 -Wall -fno-strict-aliasing  -MT regress-regress_zlib.o -MD -MP -MF .deps/regress-regress_zlib.Tpo -c -o regress-regress_zlib.o `test -f 'regress_zlib.c' || echo './'`regress_zlib.c
mv -f .deps/regress-regress_zlib.Tpo .deps/regress-regress_zlib.Po
/bin/sh ../libtool --tag=CC   --mode=link gcc  -O2 -Wall -fno-strict-aliasing    -o regress regress-regress.o regress-regress_buffer.o regress-regress_http.o regress-regress_dns.o regress-regress_testutils.o regress-regress_rpc.o regress-regress.gen.o regress-regress_et.o regress-regress_bufferevent.o regress-regress_listener.o regress-regress_util.o regress-tinytest.o regress-regress_main.o regress-regress_minheap.o regress-regress_thread.o regress-regress_zlib.o    ../libevent.la ../libevent_pthreads.la -lz   -lrt 
libtool: link: gcc -O2 -Wall -fno-strict-aliasing -o .libs/regress regress-regress.o regress-regress_buffer.o regress-regress_http.o regress-regress_dns.o regress-regress_testutils.o regress-regress_rpc.o regress-regress.gen.o regress-regress_et.o regress-regress_bufferevent.o regress-regress_listener.o regress-regress_util.o regress-tinytest.o regress-regress_main.o regress-regress_minheap.o regress-regress_thread.o regress-regress_zlib.o  ../.libs/libevent.so ../.libs/libevent_pthreads.so -lz -lrt -Wl,-rpath -Wl,/var/www/websearch/lib
make[4]: Entering directory `/var/www/xunso/xunsearch-full-1.4.8/libevent-2.0.21-stable/test'
make[4]: Nothing to be done for `install-exec-am'.
make[4]: Nothing to be done for `install-data-am'.
make[4]: Leaving directory `/var/www/xunso/xunsearch-full-1.4.8/libevent-2.0.21-stable/test'
make[3]: Leaving directory `/var/www/xunso/xunsearch-full-1.4.8/libevent-2.0.21-stable/test'
make[2]: Leaving directory `/var/www/xunso/xunsearch-full-1.4.8/libevent-2.0.21-stable/test'
make[1]: Leaving directory `/var/www/xunso/xunsearch-full-1.4.8/libevent-2.0.21-stable'
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gawk... (cached) gawk
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of g++... gcc3
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /bin/sed
checking whether ln -s works... yes
checking whether make sets $(MAKE)... (cached) yes
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking union semun for semaphore...... no
checking for sun_len in sys/un.h... no
checking for event2/event.h of libevent... yes: /var/www/websearch/
checking for correct libevent version... ok
checking for event_base_new in -levent_core... yes
checking for scws.h of scws... yes: /var/www/websearch/
checking for scws_new in -lscws... yes
checking for xapian.h of xapian-core... yes: /var/www/websearch/
checking for correct xapian version... ok
checking for library of xapian-core... ok
checking for pthread_mutex_init in -lpthread... yes
checking how to run the C++ preprocessor... g++ -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking arpa/inet.h usability... yes
checking arpa/inet.h presence... yes
checking for arpa/inet.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for strings.h... (cached) yes
checking sys/param.h usability... yes
checking sys/param.h presence... yes
checking for sys/param.h... yes
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for unistd.h... (cached) yes
checking for stdbool.h that conforms to C99... yes
checking for _Bool... no
checking for inline... inline
checking for off_t... yes
checking for pid_t... yes
checking for size_t... yes
checking size of off_t... 8
checking for error_at_line... yes
checking vfork.h usability... no
checking vfork.h presence... no
checking for vfork.h... no
checking for fork... yes
checking for vfork... yes
checking for working fork... yes
checking for working vfork... (cached) yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... yes
checking for stdlib.h... (cached) yes
checking for unistd.h... (cached) yes
checking for sys/param.h... (cached) yes
checking for getpagesize... yes
checking for working mmap... yes
checking for working strtod... yes
checking for setproctitle... no
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating sdk/Makefile
config.status: creating src/xs-ctl.sh
config.status: creating src/xs-optimize.sh
config.status: creating config.h
config.status: executing depfiles commands
Making clean in sdk
make[1]: Entering directory `/var/www/xunso/xunsearch-full-1.4.8/xunsearch-1.4.8/sdk'
make[1]: Nothing to be done for `clean'.
make[1]: Leaving directory `/var/www/xunso/xunsearch-full-1.4.8/xunsearch-1.4.8/sdk'
Making clean in src
make[1]: Entering directory `/var/www/xunso/xunsearch-full-1.4.8/xunsearch-1.4.8/src'
test -z "xs-import xs-indexd xs-logging xs-searchd" || rm -f xs-import xs-indexd xs-logging xs-searchd
rm -f user_mm.c
rm -f *.o
make[1]: Leaving directory `/var/www/xunso/xunsearch-full-1.4.8/xunsearch-1.4.8/src'
Making clean in .
make[1]: Entering directory `/var/www/xunso/xunsearch-full-1.4.8/xunsearch-1.4.8'
make[1]: Nothing to be done for `clean-am'.
make[1]: Leaving directory `/var/www/xunso/xunsearch-full-1.4.8/xunsearch-1.4.8'
Making install in .
make[1]: Entering directory `/var/www/xunso/xunsearch-full-1.4.8/xunsearch-1.4.8'
make[2]: Entering directory `/var/www/xunso/xunsearch-full-1.4.8/xunsearch-1.4.8'
test -z "/var/www/websearch/etc" || /bin/mkdir -p "/var/www/websearch/etc"
 /usr/bin/install -c -m 644 'etc/py.xdb' '/var/www/websearch/etc/py.xdb'
test -z "/var/www/websearch/etc" || /bin/mkdir -p "/var/www/websearch/etc"
 /usr/bin/install -c -m 644 'etc/dict_user.txt' '/var/www/websearch/etc/dict_user.txt'
 /usr/bin/install -c -m 644 'etc/stopwords.txt' '/var/www/websearch/etc/stopwords.txt'
make[2]: *** [install-customDATA] 错误 1
make[2]: Leaving directory `/var/www/xunso/xunsearch-full-1.4.8/xunsearch-1.4.8'
make[1]: *** [install-am] 错误 2
make[1]: Leaving directory `/var/www/xunso/xunsearch-full-1.4.8/xunsearch-1.4.8'
make: *** [install-recursive] 错误 1

希望能解决下。

使用ADDRANGE后，即使setLimit到INT MAX 值，返回来的数量还是错误的

使用SDK在buildOther里面使用addRange之后，虽然有生效，但是setLimit(MAX)去获取精确总数的时候，返回来的总数还是估算的。

Compile error when configure with custom xapian location

I installed

1. libevent1 to /Volumes/KDr2/Developer/libevent1
2. scws to /Volumes/KDr2/Developer/scws
3. xapian-core(scws patch) to /Volumes/KDr2/Developer/xapian-scws

Then I configure xunsearch with command:

./configure --prefix=/Volumes/KDr2/Developer/xunsearch \
--with-libevent=/Volumes/KDr2/Developer/libevent1 \
--with-scws=/Volumes/KDr2/Developer/scws \
--with-xapian=/Volumes/KDr2/Developer/xapian-scws

Till here all things go well, here's the configure output:

KDr2@KDr2-MacBookPro:~/Work/seas/xunsearch$ ./configure --prefix=/Volumes/KDr2/Developer/xunsearch --with-libevent=/Volumes/KDr2/Developer/libevent1 --with-scws=/Volumes/KDr2/Developer/scws --with-xapian=/Volumes/KDr2/Developer/xapian-scws
checking for a BSD-compatible install... /opt/local/bin/ginstall -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /opt/local/bin/gmkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking for gawk... (cached) gawk
checking for g++... g++
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether g++ accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of g++... gcc3
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking for a sed that does not truncate output... /usr/bin/sed
checking whether ln -s works... yes
checking whether make sets $(MAKE)... (cached) yes
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for special C compiler options needed for large files... no
checking for _FILE_OFFSET_BITS value needed for large files... no
checking union semun for semaphore...... yes
checking for sun_len in sys/un.h... yes
checking for event.h of libevent... yes: /Volumes/KDr2/Developer/libevent1
checking for correct libevent version... ok
checking for event_init in -levent... yes
checking for scws.h of scws... yes: /Volumes/KDr2/Developer/scws
checking for scws_new in -lscws... yes
checking for xapian.h of xapian-core... yes: /Volumes/KDr2/Developer/xapian-scws
checking for correct xapian version... ok
checking for library of xapian-core... ok
checking for pthread_mutex_init in -lpthread... yes
checking how to run the C++ preprocessor... g++ -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking arpa/inet.h usability... yes
checking arpa/inet.h presence... yes
checking for arpa/inet.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking for strings.h... (cached) yes
checking sys/param.h usability... yes
checking sys/param.h presence... yes
checking for sys/param.h... yes
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for unistd.h... (cached) yes
checking for stdbool.h that conforms to C99... yes
checking for _Bool... no
checking for inline... inline
checking for off_t... yes
checking for pid_t... yes
checking for size_t... yes
checking size of off_t... 8
checking for error_at_line... no
checking vfork.h usability... no
checking vfork.h presence... no
checking for vfork.h... no
checking for fork... yes
checking for vfork... yes
checking for working fork... yes
checking for working vfork... (cached) yes
checking for stdlib.h... (cached) yes
checking for GNU libc compatible malloc... yes
checking for stdlib.h... (cached) yes
checking for unistd.h... (cached) yes
checking for sys/param.h... (cached) yes
checking for getpagesize... yes
checking for working mmap... yes
checking for working strtod... yes
checking for setproctitle... no
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating sdk/Makefile
config.status: creating src/xs-ctl.sh
config.status: creating src/xs-optimize.sh
config.status: creating config.h
config.status: config.h is unchanged
config.status: executing depfiles commands
KDr2@KDr2-MacBookPro:~/Work/seas/xunsearch$

Then I make it, and an error occurs:

KDr2@KDr2-MacBookPro:~/Work/seas/xunsearch$ make
make  all-recursive
Making all in .
Making all in src
g++ -DHAVE_CONFIG_H -I. -I..     -g -O2 -I/Volumes/KDr2/Developer/libevent1/include -MT import.o -MD -MP -MF .deps/import.Tpo -c -o import.o import.cc
import.cc:20:20: error: xapian.h: No such file or directory
import.cc:21:28: error: xapian/unicode.h: No such file or directory
import.cc:61: error: ‘Xapian’ has not been declared
import.cc:61: error: expected initializer before ‘database’
import.cc:62: error: ‘Xapian’ has not been declared
import.cc:62: error: expected initializer before ‘indexer’
import.cc:63: error: ‘Xapian’ has not been declared
import.cc:63: error: expected initializer before ‘stemmer’
import.cc:64: error: ‘Xapian’ has not been declared
import.cc:64: error: expected initializer before ‘stopper’
import.cc:65: error: ‘std::string’ has not been declared
import.cc: In function ‘void show_usage()’:
import.cc:125: error: expected `)' before ‘XAPIAN_VERSION’
import.cc: In function ‘void batch_committed(int)’:
import.cc:140: error: ‘archive’ was not declared in this scope
import.cc:143: error: ‘database’ was not declared in this scope
import.cc:148: error: ‘database’ was not declared in this scope
import.cc:159: error: ISO C++ forbids declaration of ‘Xapian’ with no type
import.cc:159: error: expected `)' before ‘::’ token
import.cc:159: error: expected `{' before ‘::’ token
import.cc:159: error: ‘::Error’ has not been declared
import.cc:159: error: ‘e’ was not declared in this scope
import.cc:159: error: expected `;' before ‘)’ token
import.cc:163: error: expected primary-expression before ‘catch’
import.cc:163: error: expected `;' before ‘catch’
import.cc: In function ‘int signal_term(int)’:
import.cc:193: error: ‘database’ was not declared in this scope
import.cc: In function ‘void signal_reload(int)’:
import.cc:232: error: ‘database’ was not declared in this scope
import.cc:237: error: ‘database’ was not declared in this scope
import.cc: At global scope:
import.cc:258: error: expected ‘,’ or ‘...’ before ‘&’ token
import.cc:258: error: ISO C++ forbids declaration of ‘string’ with no type
import.cc: In function ‘bool should_stem(int)’:
import.cc:261: error: ‘Xapian’ has not been declared
import.cc:262: error: ‘Xapian’ has not been declared
import.cc:263: error: ‘Xapian’ has not been declared
import.cc:264: error: ‘Xapian’ has not been declared
import.cc:265: error: ‘Xapian’ has not been declared
import.cc:265: error: expected `;' before ‘u’
import.cc:266: error: ‘u’ was not declared in this scope
import.cc:266: error: ‘Xapian’ has not been declared
import.cc: In function ‘int doc_fetch()’:
import.cc:360: error: ‘Xapian’ has not been declared
import.cc:360: error: expected `;' before ‘doc’
import.cc:441: error: ‘Xapian’ has not been declared
import.cc:441: error: ‘syn_db’ was not declared in this scope
import.cc:442: error: ‘string’ was not declared in this scope
import.cc:442: error: expected `;' before ‘org_term’
import.cc:443: error: expected `;' before ‘syn_term’
import.cc:445: error: expected `;' before ‘org_stem’
import.cc:446: error: ‘org_term’ was not declared in this scope
import.cc:446: error: ‘string’ is not a class or namespace
import.cc:448: error: ‘org_stem’ was not declared in this scope
import.cc:448: error: ‘stemmer’ was not declared in this scope
import.cc:449: error: ‘syn_stem’ was not declared in this scope
import.cc:449: error: ‘syn_term’ was not declared in this scope
import.cc:453: error: ‘archive’ was not declared in this scope
import.cc:453: error: ‘database’ was not declared in this scope
import.cc:457: error: ‘org_term’ was not declared in this scope
import.cc:457: error: ‘syn_term’ was not declared in this scope
import.cc:461: error: ‘org_stem’ was not declared in this scope
import.cc:463: error: ‘syn_stem’ was not declared in this scope
import.cc:471: error: ‘syn_term’ was not declared in this scope
import.cc:474: error: ‘org_term’ was not declared in this scope
import.cc:478: error: ‘org_stem’ was not declared in this scope
import.cc:480: error: ‘syn_stem’ was not declared in this scope
import.cc:488: error: ‘org_term’ was not declared in this scope
import.cc:492: error: ‘org_stem’ was not declared in this scope
import.cc:515: error: ‘archive’ was not declared in this scope
import.cc:524: error: ‘database’ was not declared in this scope
import.cc:539: error: ‘indexer’ was not declared in this scope
import.cc:539: error: ‘doc’ was not declared in this scope
import.cc:584: error: ‘string’ was not declared in this scope
import.cc:584: error: expected `;' before ‘tt’
import.cc:585: error: expected `;' before ‘pp’
import.cc:589: error: ‘pp’ was not declared in this scope
import.cc:592: error: ‘pp’ was not declared in this scope
import.cc:592: error: ‘tt’ was not declared in this scope
import.cc:596: error: ‘pp’ was not declared in this scope
import.cc:596: error: ‘tt’ was not declared in this scope
import.cc:600: error: ‘tt’ was not declared in this scope
import.cc:602: error: expected `;' before ‘ss’
import.cc:603: error: ‘ss’ was not declared in this scope
import.cc:603: error: ‘pp’ was not declared in this scope
import.cc:603: error: ‘stemmer’ was not declared in this scope
import.cc:620: error: ‘string’ was not declared in this scope
import.cc:620: error: expected `;' before ‘enc’
import.cc:621: error: ‘enc’ was not declared in this scope
import.cc:661: error: ‘database’ was not declared in this scope
import.cc:666: error: ‘archive’ was not declared in this scope
import.cc:673: error: ‘database’ was not declared in this scope
import.cc:681: error: ISO C++ forbids declaration of ‘Xapian’ with no type
import.cc:681: error: expected `)' before ‘::’ token
import.cc:681: error: expected `{' before ‘::’ token
import.cc:681: error: ‘::Error’ has not been declared
import.cc:681: error: ‘e’ was not declared in this scope
import.cc:681: error: expected `;' before ‘)’ token
import.cc: In function ‘int main(int, char**)’:
import.cc:710: error: ‘stemmer’ was not declared in this scope
import.cc:710: error: ‘Xapian’ has not been declared
import.cc:750: error: ‘Xapian’ has not been declared
import.cc:779: error: ‘stopper’ was not declared in this scope
import.cc:894: error: ‘database’ was not declared in this scope
import.cc:894: error: ‘Xapian’ has not been declared
import.cc:894: error: ‘Xapian’ has not been declared
import.cc:898: error: ‘database’ was not declared in this scope
import.cc:898: error: ‘Xapian’ has not been declared
import.cc:901: error: ‘indexer’ was not declared in this scope
import.cc:902: error: ‘stopper’ was not declared in this scope
import.cc:903: error: ‘database’ was not declared in this scope
import.cc:907: error: ‘Xapian’ has not been declared
import.cc:910: error: ISO C++ forbids declaration of ‘Xapian’ with no type
import.cc:910: error: expected `)' before ‘::’ token
import.cc:910: error: expected `{' before ‘::’ token
import.cc:910: error: ‘::Error’ has not been declared
import.cc:910: error: ‘e’ was not declared in this scope
import.cc:910: error: expected `;' before ‘)’ token
import.cc:916: error: expected primary-expression before ‘catch’
import.cc:916: error: expected `;' before ‘catch’
import.cc:931: error: ‘archive’ was not declared in this scope
import.cc:931: error: ‘Xapian’ has not been declared
import.cc:931: error: ‘Xapian’ has not been declared
import.cc:940: error: ‘archive’ was not declared in this scope
import.cc:940: error: ‘Xapian’ has not been declared
import.cc:940: error: ‘Xapian’ has not been declared
import.cc:951: error: ‘database’ was not declared in this scope
import.cc:953: error: ‘database’ was not declared in this scope
import.cc:984: error: ‘database’ was not declared in this scope
import.cc:1001: error: ‘archive’ was not declared in this scope
import.cc:1021: error: ‘Xapian’ has not been declared
import.cc:1021: error: ‘Xapian’ has not been declared
make[2]: *** [import.o] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all] Error 2
KDr2@KDr2-MacBookPro:~/Work/seas/xunsearch$ 

I think there's some thing wrong with the XAPIAN_DIR substitute is configure.ac when configure with option --with-xapian=..., would you please check and fix this? Many Thanks!

PS:

1. My os: MacOSX 10.7, but I think this bug can be produced on linux.
2. xunsearch-full-version.tar.bz2 works(be installed) well on my computer.

搜索Total不准确

符合条件的索引共有161条，但offset不是最后一页的情况下，显示的total为 310条

# php Quest.php ~/dev/config/search/question.ini --query="answered:0 AND (status:0 OR status:1)" --limit 120,30|head -n 1
在 1,525 条数据中，大约有 310 条包含 answered:0 AND (status:0 OR status:1) ，第 121-150 条，用时：0.0234 秒。

# php Quest.php ~/dev/config/search/question.ini --query="answered:0 AND (status:0 OR status:1)" --limit 140,30|head -n 1
在 1,525 条数据中，大约有 161 条包含 answered:0 AND (status:0 OR status:1) ，第 141-161 条，用时：0.0200 秒。