takearest118/elastic_hunter

You can export and import json files in Elasticsearch, manage data in elasticsearch by json format. Enjoy your data hunting.

elastic_hunter's Issues

Can't export from 10,000 docs

error message

Traceback (most recent call last):
File "app.py", line 49, in
export()
File "/Users/khcho/workspaces/pyvenv/elastic_hunter/lib/python3.8/site-packages/click/core.py", line 829, in call
return self.main(*args, **kwargs)
File "/Users/khcho/workspaces/pyvenv/elastic_hunter/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/khcho/workspaces/pyvenv/elastic_hunter/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/khcho/workspaces/pyvenv/elastic_hunter/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "app.py", line 37, in export
res = es.search(index=index, body={}, from_=i, size=CHUNK)
File "/Users/khcho/workspaces/pyvenv/elastic_hunter/lib/python3.8/site-packages/elasticsearch/client/utils.py", line 152, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
File "/Users/khcho/workspaces/pyvenv/elastic_hunter/lib/python3.8/site-packages/elasticsearch/client/init.py", line 1658, in search
return self.transport.perform_request(
File "/Users/khcho/workspaces/pyvenv/elastic_hunter/lib/python3.8/site-packages/elasticsearch/transport.py", line 392, in perform_request
raise e
File "/Users/khcho/workspaces/pyvenv/elastic_hunter/lib/python3.8/site-packages/elasticsearch/transport.py", line 358, in perform_request
status, headers_response, data = connection.perform_request(
File "/Users/khcho/workspaces/pyvenv/elastic_hunter/lib/python3.8/site-packages/elasticsearch/connection/http_urllib3.py", line 269, in perform_request
self._raise_error(response.status, raw_data)
File "/Users/khcho/workspaces/pyvenv/elastic_hunter/lib/python3.8/site-packages/elasticsearch/connection/base.py", line 315, in _raise_error
raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
elasticsearch.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'Result window is too large, from + size must be less than or equal to: [10000] but was [11000]. See the scroll api for a more efficient way to request large data sets. This limit can be set by changing the [index.max_result_window] index level setting.')