Comments (10)
I see the searchd does:
[pid 1252352] openat(AT_FDCWD, "/var/lib/manticore/t2/t2.meta", O_RDONLY) = 2106
[pid 1252352] read(2106, "{\n\"meta_created_time_utc\":\"2024-"..., 635) = 635
[pid 1252352] close(2106) = 0
[pid 1252352] openat(AT_FDCWD, "wf", O_RDONLY) = -1 ENOENT (No such file or directory)
i.e. tries to open wf
in the current dir unlike t2.meta
which is opened by its absolute path.
echo "a > b" > /var/lib/manticore/wf
lets import table t2 from '/var/lib/manticore/wftest/wftest'
pass.
This task blocks #1788
from manticoresearch.
The issue is not new:
root@a2ad520fb15e:/var/lib/manticore# searchd -v
Manticore 6.2.12 dc5144d35@230822
root@a2ad520fb15e:/var/lib/manticore# echo "a > b" > /tmp/wf;
root@a2ad520fb15e:/var/lib/manticore# mysql -h0 -P9306 -e "create table wftest(f text) wordforms='/tmp/wf'"
mysql -h0 -P9306 -e "insert into wftest(id) values(1)"
mysql -h0 -P9306 -e "import table t2 from '/var/lib/manticore/wftest/wftest'"
ERROR 1064 (42000) at line 1: unable to import table 't2': failed to open wf: No such file or directory
from manticoresearch.
The same case with the exceptions
mysql> import table table3 from '/var/lib/manticore/table2/table2';
ERROR 1064 (42000): unable to import table 'table3': failed to prealloc: 'synonyms': failed to open exceptions.txt: No such file or directory
from manticoresearch.
fixed at b5a3b7f now import table should copy external files well
from manticoresearch.
@PavelShilin89
however the tests should be developed. All the tests should end as
show index imported settings
and all external files should be there- daemon restart and should be no warning about missed external files
- after restart
show index imported settings
and all external files should be there
all indexes should have exceptions
and multiple stopwords
files and multiple wordforms
files
cases:
- RT index defined in the config (not RT mode) with all external files located NOT at the index folder ( maybe /tmp ) should be imported into another table well
- RT index created in RT mode with all external files should be imported into another table well
- RT index created in RT mode with all external files and additional disk chunks should be imported into another table well
there could be another cases that I am unaware about maybe @PavelShilin89 or @sanikolaev could find more useful cases
from manticoresearch.
It partitionally work:
I tested it on test kit from your commit ghcr.io/manticoresoftware/manticoresearch:test-kit-b5a3b7f
docker run -it --rm ghcr.io/manticoresoftware/manticoresearch:test-kit-b5a3b7f bash
# searchd
Manticore 6.2.13 b5a3b7f1f@24051010 dev (columnar 2.2.5 478fff2@240417) (secondary 2.2.5 478fff2@240417) (knn 2.2.5 478fff2@240417)
....
# echo "a => b" > /var/lib/manticore/wordforms1.txt; echo "c => d" > /var/lib/manticore/exc.txt; echo "abcstop" > /var/lib/manticore/stop.txt
# ls -1 /var/lib/manticore/
binlog
exc.txt
state.sql
stop.txt
wordforms1.txt
mysql -h0 -P9306
mysql> CREATE TABLE table1 (title TEXT, tag INTEGER) exceptions='/var/lib/manticore/exc.txt' wordforms='/var/lib/manticore/wordforms1.txt' stopwords='/var/lib/manticore/stop.txt';
mysql> INSERT INTO table1 (id, title, tag) values (1, 'zxczxc', 77);
mysql> import table table2 from '/var/lib/manticore/table1/table1';
mysql> import table table3 from '/var/lib/manticore/table2/table2';
ERROR 1064 (42000): unable to import table 'table3': failed to prealloc: 'synonyms': failed to open exceptions.txt: No such file or directory
from manticoresearch.
seems case more complex and needs one of the following
- drop support of the indexes with binary
meta
andsph
headers forimport table
and rewrite load, fixup, save headers forimport table
- hack loading of the indexes and push there new boolean to not generate error on missed external but keep thΡΠ΅ info about that external
from manticoresearch.
at the 56052bf I fixed import table to copy external files.
from manticoresearch.
it could be better to add to test described here #2154 (comment) the case to test that all externals are applied (exceptions, workdforms, stopwords) to prevent the case like #2207
from manticoresearch.
Done in #2222
from manticoresearch.
Related Issues (20)
- improve documentation about test HOT 2
- fix windows github CI test rt_442
- python sdk insert knn columns error: 'numpy.float32' object has no attribute 'openapi_types' HOT 1
- mysql> SHOW VERSION; not working HOT 2
- if second instance is executed id doesn't stop, but uses hi % of CPU HOT 4
- improve documentation about Buddy statements HOT 2
- [BUDDY] Fatal error: Uncaught Manticoresearch\Buddy\Core\Error\ManticoreSearchClientError
- Demo dashboard for Kibana
- Disable secondary indexes for JSON attribute after updates
- Secondary indexes for JSON attributes don't work with exclude filters HOT 1
- Manticoresearch crashes with memory issues HOT 12
- prefix_fields & min_prefix_len not working for specific field HOT 1
- Limited number of characters HOT 1
- Moving Kibana-related endpoints to Buddy HOT 4
- FATAL: MATCH('ZONE:(title) HOT 7
- Sorting by any field in Queries with Vector Search Not Working as Expected HOT 8
- Problem with downloading or connecting to repositories while building a Docker image
- alter float vector broken HOT 1
- adding column with float_vector leads to index corruption
- Please support doubled single quote in SQL according to the ANSI standard HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from manticoresearch.