yiisoft/yii2-elasticsearch

Yii 2 Elasticsearch extension

Elasticsearch Query and ActiveRecord for Yii 2

This extension provides the Elasticsearch integration for the Yii framework 2.0. It includes basic querying/search support and also implements the ActiveRecord pattern that allows you to store active records in Elasticsearch.

For license information check the LICENSE-file.

Documentation is at docs/guide/README.md.

Requirements

Depending on the version of Elasticsearch you are using you need a different version of this extension.

• For Elasticsearch 1.6.0 to 1.7.6 use extension version 2.0.x
• For Elasticsearch 5.x or above use extension version 2.1.x

Installation

The preferred way to install this extension is through composer:

composer require --prefer-dist yiisoft/yii2-elasticsearch:"~2.1.0"

Configuration

To use this extension, you have to configure the Connection class in your application configuration:

return [
    //....
    'components' => [
        'elasticsearch' => [
            'class' => 'yii\elasticsearch\Connection',
            'nodes' => [
                ['http_address' => '127.0.0.1:9200'],
                // configure more hosts if you have a cluster
            ],
            'dslVersion' => 7, // default is 5
        ],
    ]
];

yii2-elasticsearch's Issues

"network.bind_host: localhost" must be uncommented for usage by yii2-elasticsearch

If I have the following option active (in the config file at /etc/elasticsearch/elasticsearch.yml)

network.bind_host: localhost

then I cannot connect the elasticsearch server by my Yii 2 code.

But I "can" connect to it via cURL from the command line. E.g.:

curl -X POST 'http://localhost:9200/tutorial/helloworld/1' -d '{ "message": "Hello World!" }'

So currently I'm using elasticsearch with "network.bind_host: localhost" switched off. This is unsecure. Everybody from the internet can access the index. How can I fix this?

Connection dies after a significant amount of requests

When inserting a massive amount of records the connection fails at some point when the system runs out of sockets:

Exception (Elasticsearch Database Exception) 'yii\elasticsearch\Exception' with message 'Elasticsearch request failed: 7 - Failed to connect to 192.168.178.50: Cannot assign requested address' 

in /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2-elasticsearch/Connection.php:376

Stack trace:
#0 /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2-elasticsearch/Connection.php(237): yii\elasticsearch\Connection->httpRequest('POST', Array, '{"version":"1.1...', false)
#1 /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2-elasticsearch/Command.php(138): yii\elasticsearch\Connection->post(Array, Array, '{"version":"1.1...')
#2 /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2-elasticsearch/ActiveRecord.php(434): yii\elasticsearch\Command->insert('yiiframework-de...', 'api-type', Array, NULL, Array)
#3 /home/cebe/dev/yiisoft/yiiframework.com/models/ApiType.php(83): yii\elasticsearch\ActiveRecord->insert(false)
#4 /home/cebe/dev/yiisoft/yiiframework.com/commands/ApiController.php(159): app\models\ApiType::createRecord(Object(yii\apidoc\models\ClassDoc), '1.1')
#5 /home/cebe/dev/yiisoft/yiiframework.com/vendor/igorw/retry/src/retry.php(11): app\commands\ApiController->app\commands\{closure}()
#6 /home/cebe/dev/yiisoft/yiiframework.com/commands/ApiController.php(165): igorw\retry(3, Object(Closure))
#7 /home/cebe/dev/yiisoft/yiiframework.com/commands/ApiController.php(74): app\commands\ApiController->populateElasticsearch1x(Array, '/home/cebe/dev/...')
#8 [internal function]: app\commands\ApiController->actionGenerate('1.1')
#9 /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2/base/InlineAction.php(55): call_user_func_array(Array, Array)
#10 /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2/base/Controller.php(151): yii\base\InlineAction->runWithParams(Array)
#11 /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2/console/Controller.php(91): yii\base\Controller->runAction('', Array)
#12 /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2/base/Module.php(455): yii\console\Controller->runAction('', Array)
#13 /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2/console/Application.php(161): yii\base\Module->runAction('api', Array)
#14 /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2/console/Application.php(137): yii\console\Application->runAction('api', Array)
#15 /home/cebe/dev/yiisoft/yiiframework.com/vendor/yiisoft/yii2/base/Application.php(375): yii\console\Application->handleRequest(Object(yii\console\Request))
#16 /home/cebe/dev/yiisoft/yiiframework.com/yii(27): yii\base\Application->run()
#17 {main}

http://elasticsearch-users.115913.n3.nabble.com/Failed-to-connect-to-127-0-0-1-Cannot-assign-requested-address-td4023793.html

You did not set the SO_REUSEADDR flag, so after "40k -50k documents", you have forced your
system to ran out of sockets. See also http://php.net/manual/de/function.socket-set-option.php

solution:

http://elasticsearch-users.115913.n3.nabble.com/Failed-to-connect-to-127-0-0-1-Cannot-assign-requested-address-tp4023793p4029385.html

AWS Elasticsearch service compatibility

Hey guys!
As you may have heard, AWS have launched Elasticsearch service recently which is really great, and I decided to take it for a spin, but this extension doesn't work with it in its current state. There are two points of failure with it.
Firstly, AWS ES endpoint returns 'text/plain' content type instead of 'application/json' - that's where the Connection->httpRequest() method fails. And secondly, ES nodes there don't have 'http_address' property for some reason, that's why Connection->open() method and a couple of others that rely on that property fail.
I hope that will be fixed, so that those who need it don't need to fork or hack the code :) Thanks!

Add documentation for how to setup mapping

i can make a pull request for this.

Document the fact that updating complex fields in elasticsearch AR is always a merge

This issue has originally been reported by @cebe at yiisoft/yii2#6709.
Moved here by @cebe.

---

removed fields will not be removed but merged with the existing record.

Adding aggregation to queries confusion

First of all, the docs do need some updates as there are a lot of confusions of how to use the methods available.

I am trying to do a bit of statistics on some fields as follows:

$record = Record::find()->addAgg('somename', 'avg', ['field' => 'myfield'])->where(['some conditions'])->all()

in the es debug panel, i do get the result in 'aggregations', but how do i get in my activerecord?

How to use Facets Filter

I use Facets with addAggregation as below
$query->addAggregation('category', 'terms', ['field' => 'category_id']);

Now, I only want facets with category with parent category id = 30. How to filter it?

Errors are not handled correctly with elasticsearch 2.0.0

decodeErrorBody method of Connection class tries to edit error message as a string, but in 2.0.0 it is an associative array.

Elastic Search - "missing features or docs"

This issue has originally been reported by @philippfrenzel at yiisoft/yii2#5351.
Moved here by @cebe.

---

Hi,

trying to use elastic search extension, I run into the following issues:

• How can I define mappings, analyzers etc. (default_analyzers, etc)
• How can I define complex queries on an index? Like several filters, terms, querys)

I have an solution running in yii 1 which is using "elastica.io" to do all the crazy elasticsearch stuff... I'm looking forward to see more about the extension, as I see the potential in es is amazing... ;)

Thanks Philipp

how to update the setting in elasticsearch for analyser

how to update the setting in elasticsearch for analyser.
there are not any description in this extension related to update setting.
if u have any solution for the same then let me know

How to get the all search result record in elasticsearch without limit

Is there a any way to fetch the all search record in elasticsearch without limit in activerecordset.
because it's display default 10 records.
let me know if u have any solutions for the same.

Add more features to elasticsearch

This issue has originally been reported by @cebe at yiisoft/yii2#1313.
Moved here by @cebe.

---

To make elasticsearch support complete.

Moved these from #1295 as we can easily add them later and they are nice to have.

• make use of native optimistic locking support
• implement versioning
• api to retrieve parents and children (see also yiisoft/yii2#5758 (comment))
• make index and type available in instantiate (#2289)
• implement updateAllCounters(), see for details elastic/elasticsearch#1583
• implement copy() and move() as PK change is not supported

Problem with search across all types

I'm getting empty search result, If type equals to _all

$ curl -XGET 'http://localhost:9200/_all/_all/_search?pretty=1' -d '{
    "query" : {
        "term" : { "_all" : "ленина" }
    }
}
'
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 8,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 0,
    "max_score" : null,
    "hits" : [ ]
  }
}

Result without specifying type

$ curl -XGET 'http://localhost:9200/_all/_search?pretty=1' -d '{
    "query" : {
        "term" : { "_all" : "ленина" }
    }
}
'
{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 8,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 1,
    "max_score" : 0.3125,
    "hits" : [ {
      "_index" : "api-index",
      "_type" : "news",
      "_id" : "206",
      "_score" : 0.3125,
      "_source":{"id":206,"title":"Привет","description":"В нашем меню новый салат!","tags":["Рязань","Ленина"]}
    } ]
  }
}

Example from the official documentation

$ curl -XGET 'http://localhost:9200/twitter/_search?q=user:kimchy'

So, I think, we shouldn't use type _all, if type is null
https://github.com/yiisoft/yii2-elasticsearch/blob/master/Command.php#L61

My code

        $r = $elasticSearch->createCommand([
            'queryParts' => [
                'query' => [
                    'term' => [
                        '_all' => 'ленина',
                    ],
                ],
            ],
        ])->search();

Or I'm doing something wrong?

query: Primary key for elasticsearch

Hey team,

I would like to know few thing..

1. why i need to call getPrimaryKey to get primary key everytime?
2. why i can't assign this primary key to mapped in Active Record so it will work fine with yii\grid\ActionColumn?

I also have few observation.

1. In `ActiveRecordsearch/find is callingmget``method which will only set``_id``as primary key which is a``Private member`` rather we can make it configuration so that we can assign this value to other value.

method name populateRecord

$pk = static::primaryKey()[0];//TODO should always set ID in case of fields are not returned
        if ($pk === '_id') {
            $record->_id = $row['_id'];
        }

1. If i also override static::primaryKey then mget will not be able to set primary key.

regards
Mithun Mandal

Running out of memory on mass indexing

If I'm indexing a mass of data (1000 - 2000 data rows) into my elasticsearch index and I'm running out of memory. This is the error message:

PHP Fatal Error 'yii\base\ErrorException' with message 'Out of memory (allocated 2359296) (tried to allocate 3072 bytes)'

I'm indexing via the following code in class method:

    $otable = table::find()->each();

    foreach ($otable as $table) {

      $mymodel = new MyModel();
      $mymodel->id = $table->id;
      $mymodel->title = $table->title;
      $mymodel->teaser = $table->teaser;
      $mymodel->category = $table->category;
      $mymodel->sub_category = $table->sub_category;
      $mymodel->save();

    }

  }

Via each(); I'm already saving memory at "reading" data from database via batches.

Is there any method which saves from running out of memory on the indexing process?

Query multiple fields without scopes []

$query->filter([
    'bool' => [
        'should' => [
            'match' => ['field1' => 'test1'],
            'match_phrase_prefix' => ['field2' => 'test2'],
        ],
    ],
]);

result:

"filter": {
        "bool": {
            "should": {
                "match": {
                    "field1": "test1"
                },
                "match_phrase_prefix": {
                    "field2": "test2"
                }
            }
        }
    }

Error is [_na] query malformed, no field after start_object
So, what is matter?

Migration support for elasticsearch

Migrated from yiisoft/yii2#6712

• provide compatible migrate command
• helper methods for simple schema changes
• allow convenient re-indexing for more complex changes http://www.elasticsearch.org/blog/changing-mapping-with-zero-downtime/

error field is an array, but treated like a string

In Connection.php around line 442:

$decoded['error'] = preg_replace('/\b\w+?Exception\[/', "<span style=\"color: red;\">\\0</span>\n               ", $decoded['error']);

However, $decoded['error'] looks like this:

array (
  'root_cause' =>
  array (
    0 =>
    array (
      'type' => 'merge_mapping_exception',
      'reason' => 'Merge failed with failures ...',
    ),
  ),
  'type' => 'merge_mapping_exception',
  'reason' => 'Merge failed with failures ...',
)

Elasticsearch 2.x, PHP 5.6, yii2-elasticsearch 2.0.3, yii2 2.0.6

Batch insert rows

When i iterative query and insert data. will has follow error:

Exception 'yii\elasticsearch\Exception' with message 'Elasticsearch request fail
ed: 7 - '

in D:\www\hiscaler\bdam\vendor\yiisoft\yii2-elasticsearch\Connection.php:3
76

Error Info:
Array
(
    [requestMethod] => PUT
    [requestUrl] => http://192.168.1.222:9200/elastic/news/18408
    [requestBody] => {"id":"18408","nodeId":"183","nodeName":"璩囪▕","title":"
鑻卞浗鏀瑰杽鍦ㄥ崕绛捐瘉鏈嶅姟 灏嗗姞蹇暀瀛︾璇侀€熷害","keywords":"鑻卞浗,
鏀瑰杽,鍦ㄥ崕,绛捐瘉鏈嶅姟,灏?鍔犲揩,鐣欏绛捐瘉","description":"鐢宠鑻卞浗绛
捐瘉鐨勫悓瀛︿滑鍙互蹇€熷彇寰楃粨鏋滐紒杩戞棩锛屽氨鑻卞浗鍐呮斂閮ㄦ壙璇烘敼
鍠勫湪鍗庣璇佹湇鍔′互婊¤冻涓浗瀹㈡埛鐨勯渶瑕侊紝鑻卞浗椹讳腑鍥藉ぇ浣块璇
︾粏鍒楀嚭鏈€鏂版敼杩涙帾鏂斤細濡傚揩閫熺璇佹湇鍔°€佸湪绾跨敵璇蜂互","autho
r":"璐ｄ换缂栬緫","source":"[db:鏉ユ簮]","hitsCount":"68","publishedAt":"1393999
500","createdAt":"1397116921","updatedAt":"1393999500"}
    [responseHeaders] => Array
        (
        )

    [responseBody] =>
)

Stack trace:
#0 D:\www\hiscaler\bdam\vendor\yiisoft\yii2-elasticsearch\Connection.php(2
54): yii\elasticsearch\Connection->httpRequest('PUT', Array, '{"id":"18408","...
', false)
#1 D:\www\hiscaler\bdam\vendor\yiisoft\yii2-elasticsearch\Command.php(136)
: yii\elasticsearch\Connection->put(Array, Array, '{"id":"18408","...')
#2 D:\www\hiscaler\bdam\console\controllers\ElasticsearchController.php(11
8): yii\elasticsearch\Command->insert('elastic', 'news', Array, '18408')
#3 [internal function]: console\controllers\ElasticsearchController->actionNews(
)
#4 D:\www\hiscaler\bdam\vendor\yiisoft\yii2\base\InlineAction.php(55): cal
l_user_func_array(Array, Array)
#5 D:\www\hiscaler\bdam\vendor\yiisoft\yii2\base\Controller.php(151): yii\
base\InlineAction->runWithParams(Array)
#6 D:\www\hiscaler\bdam\vendor\yiisoft\yii2\console\Controller.php(91): yi
i\base\Controller->runAction('news', Array)
#7 D:\www\hiscaler\bdam\console\controllers\ElasticsearchController.php(20
): yii\console\Controller->runAction('news')
#8 [internal function]: console\controllers\ElasticsearchController->actionIndex
()
#9 D:\www\hiscaler\bdam\vendor\yiisoft\yii2\base\InlineAction.php(55): cal
l_user_func_array(Array, Array)
#10 D:\www\hiscaler\bdam\vendor\yiisoft\yii2\base\Controller.php(151): yii
\base\InlineAction->runWithParams(Array)
#11 D:\www\hiscaler\bdam\vendor\yiisoft\yii2\console\Controller.php(91): y
ii\base\Controller->runAction('', Array)
#12 D:\www\hiscaler\bdam\vendor\yiisoft\yii2\base\Module.php(455): yii\con
sole\Controller->runAction('', Array)
#13 D:\www\hiscaler\bdam\vendor\yiisoft\yii2\console\Application.php(161):
 yii\base\Module->runAction('elasticsearch', Array)
#14 D:\www\hiscaler\bdam\vendor\yiisoft\yii2\console\Application.php(137):
 yii\console\Application->runAction('elasticsearch', Array)
#15 D:\www\hiscaler\bdam\vendor\yiisoft\yii2\base\Application.php(375): yi
i\console\Application->handleRequest(Object(yii\console\Request))
#16 D:\www\hiscaler\bdam\yii(27): yii\base\Application->run()
#17 {main}

Data is valid. Each time the error occurred in the different data

ElasticSearch Aggregation Buckets

This issue has originally been reported by @chuntley at yiisoft/yii2#5695.
Moved here by @cebe.

---

The current ElasticSearch implementation of aggregations is missing the ability to group aggregations into buckets. As stated on their documentation (http://www.elasticsearch.org/guide/en/elasticsearch/reference/1.x/search-aggregations.html):

A family of aggregations that build buckets, where each bucket is associated with a key and a document criterion. When the aggregation is executed, all the buckets criteria are evaluated on every document in the context and when a criterion matches, the document is considered to "fall in" the relevant bucket. By the end of the aggregation process, we’ll end up with a list of buckets - each one with a set of documents that "belong" to it.

A simple implementation of this is returning aggregation results by day, for example:

{
  "size": 0,
  "query": {
    "filtered": {
      "filter": {
        "bool": {
          "must": [
            {
              "range": {
                "log_timestamp": {
                  "from": "2014-10-01",
                  "to": "2014-10-30"
                }
              }
            }
          ]
        }
      }
    }
  },
  "aggregations": {
    "by_day": {
      "date_histogram": {
        "field": "timestamp",
        "interval": "day"
      },
      "aggregations": {
        "run_time": {
          "avg": {
            "field": "run_time"
          }
        }
      }
    }
  }
}

Adding the ability to have parent/children in addAgg() should be sufficient for adding aggregation buckets.

AND/OR/NOT filters/queries deprecated in favor of bool query

As of ES 2.0.0-beta1 the preferred way of handling boolean logic in filters and queries is the bool filter/query.

Should QueryBuilder class be updated to create a bool filter by default?

ActiveQuery's updateAll only updates first 10 entries

When updating multiple models with updateAll, only first 10 entries get updated. This query attempts to fetch all primary keys that satisfy the provided $condition:
https://github.com/yiisoft/yii2-elasticsearch/blob/master/ActiveRecord.php#L580

However, by default its limit is set to 10 (at least in my ES 1.7), so it only gets the first 10 PKs. They get correctly updated afterwards using bulk API.

The problem is with such approach in general, as there is a limit to how far we can set the limit (at least in ES 2.0).

One possible solution would be to either use an external plugin that can update all documents based on a query, or somehow mimic its behavior using scripts.

Another option is getting those PKs with scan-and-scroll API and then updating using bulk API.

P.S.: Phpdoc summary of this function refers to SQL and is not accurate.

Relation don't work if you use "asArray()"

My elasticsearch model:

class User extends \yii\elasticsearch\ActiveRecord {

    public function attributes()
    {
        return ['id', 'name', 'address', 'id_company'];
    }

    public function getCompany()
    {
        return $this->hasOne(Company::className(), ['id' => 'id_company']);
    }
}

Problem:

$models = User::find()->where(['id'=>1])->with('company')->one();

// $models->company !== null;

$models = User::find()->where(['id'=>1])->with('company')->asArray()->one();

// isset($models['company']) === false;

😖

I have failure in this line

https://github.com/yiisoft/yii2-elasticsearch/blob/master/ActiveRecord.php#L329

What's the reset for? When I removed it, it works properly. With it, I received this error:

Invalid operand type was used: expecting an array

    1. in /opt/application/urbanindov2/vendor/yiisoft/yii2-elasticsearch/ActiveRecord.php  at line 329 

Pagination in ActiveDataProvider error (Result window is too)

What steps will reproduce the problem?

Paginate to last page. for example size is 10 and from 12000 document.

What's expected?

Load the request page.
The request body is:

{
   "fields":[
      "_id",
      "title",
      "date"
   ],
   "size":10,
   "from":11880,
   "query":{
      "filtered":{
         "filter":{
            "bool":{
               "must":[

               ]
            }
         }
      }
   },
   "sort":[
      {
         "date":{
            "order":"desc"
         }
      }
   ]
}

What do you get instead?

Result window is too
large, from + size must be less than or equal to: [10000] but was
[11890]. See the scroll api for a more efficient way to request large
data sets. This limit can be set by changing the
[index.max_result_window] index level parameter.

Additional info

Q	A
Yii vesion	dev-master
PHP version	5.6
Operating system	Ubuntu 16.04

How to define index method in elasticsearch in yii2?

This issue has originally been reported by @nishanthpininti at yiisoft/yii2#8877.
Moved here by @cebe.

---

I am using filters in Elasticsearch in yii2, I want to have an exact match to a field for example item_name="tshirt", so I want to define an index for this field as 'not_analyzed', how do I define index in yii2-elasticsearch?

review elasticsearch-php

This issue has originally been reported by @cebe at yiisoft/yii2#2590.
Moved here by @cebe.

---

Check completeness and correctness of yii implementation.
related to #2427

https://github.com/elasticsearch/elasticsearch-php

[Proposal] Official PHP API from Elastic

What do you think about using official API for ElasticSearch (https://github.com/elastic/elasticsearch-php)?

Remove the dependency of CURL extension

Originally reported at yii2-authclient repo at:
yiisoft/yii2-authclient#27
It affects yii2-elasticsearch as well.

The PHP stream extension (context for file_get_contents() method) can be used for HTTP requests as well as CURL library.
PHP stream extension is built-in, while CURL should be installed manually producing redundant dependency.

Implement support for elasticsearch scroll API

This issue has originally been reported by @cebe at yiisoft/yii2#2410.
Moved here by @cebe.

---

Interface could be similar to #2409

Unexpected handling of and/or filter conditions

Adding several conditions using orWhere results in an unexpected query structure:

$query->where(['_id' => ['1']]);
$query->orWhere(['_id' => ['2']]);
$query->orWhere(['_id' => ['3']]);
$query->orWhere(['_id' => ['4']]);

This produces the following query structure:

{
  "filter":{
    "or":[{
        "or":[{
            "or":[
              {"ids":{"values":["1"]}},
              {"ids":{"values":["2"]}}
            ]
          },
          {"ids":{"values":["3"]}}
        ]
      },
      {"ids":{"values":["4"]}}
    ]
  }
}

I would expect something like:

{
  "filter":{
    "or":[
      {"ids":{"values":["1"]}},
      {"ids":{"values":["2"]}},
      {"ids":{"values":["3"]}},
      {"ids":{"values":["4"]}}
    ]
  }
}

Disabling the search result limit is not working as it should

I have the following code at the moment that does a working search, everything is working as it should until I wanted to disable the (very) low limit on search results.

$query->query(["query_string" => ["query" => $string]])->limit(null)->search();

The docs mention that putting the limit to null or a negative value should disable the limit, but I still only get 10 results returned. Putting the limit above 10 however does work as expected.

I'm using Yii 2.0.8 and yii2-elasticsearch 2.0.4

Not able to fetch _id value as that is defined as private

In ActiveRecord

private $_id;

due to this i am not able to access / read primary key value from _id from DetailView

when i change to

public $_id;

then it worked fine. When I do public then I can read in child class.

Let me know if i missing something or its a bug.

regards
Mithun Mandal

Allow to connect to ES cluster using HTTPS

Currently, HTTP protocol is hardcoded. There should be a way to connect to ES cluster using HTTPS (useful for AWS).

To quote the original comment:

Hi, i am using elasticsearch on AWS, and i get host like this: https://foo.bar.ap-southeast-1.es.amazonaws.com:443
How to force http_address using "https://", I read elasticsearch connection code, they hardcoded "http://",
Dirty way, I changed "http://" to "https://" and it works, Is there any proper way to use https?

yii2-elasticsearch deleteAll(null) delete only 10 rows according default limit

Migrated from yiisoft/yii2#5734

// yii\elasticsearch\ActiveRecord

public static function deleteAll($condition = [])
    {
        $pkName = static::primaryKey()[0];
        if (count($condition) == 1 && isset($condition[$pkName])) {
            $primaryKeys = is_array($condition[$pkName]) ? $condition[$pkName] : [$condition[$pkName]];
        } else {
            $primaryKeys = static::find()->where($condition)->column($pkName); // TODO check whether this works with default pk _id
        }

But on other hand you have

yii\elasticsearch\Query 

/**
     * @inheritdoc
     */
    public function init()
    {
        parent::init();
        // setting the default limit according to elasticsearch defaults
        // http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/search-request-body.html#_parameters_3
        if ($this->limit === null) {
            $this->limit = 10;
        }
    }

Eventually

$primaryKeys = static::find()->where($condition)->column($pkName); // TODO check whether this works with default pk _id

will return just 10 rows according to limit. I'm not sure about other methods. But personally faced with this problem in deleteAll

Also I would consider use http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-delete-index.html
i.e. delete the whole index if $condition is null. But need to check mapping in this case.

Support like query

es support wildcard query, something like this:

    private function buildLikeCondition($operator, $operands)
    {
        if (!isset($operands[0], $operands[1])) {
            throw new InvalidParamException("Operator '$operator' requires two operands.");
        }

        if( $operator == 'like') {
            $like_text = "*" . $operands[1] . "*";
            $query = ['wildcard' => [$operands[0]=>$like_text]];
        }
        return $query;
//         throw new NotSupportedException('like conditions are not supported by elasticsearch.');
    }

Elasticsearch log target

ElasticSearch Connection fallback

This issue has originally been reported by @chuntley at yiisoft/yii2#6377.
Moved here by @cebe.

---

With ElasticSearch you can connect to any query against any node on the cluster. The way the Yii's ElasticSearch connection is setup you set an array of IP addresses of the nodes in your cluster, then Yii randomly chooses a server to connect to then begins running commands against it.

          'nodes' => [
                ['http_address' => '10.111.55.50:9200'],
                ['http_address' => '10.111.55.51:9200'],
                ['http_address' => '10.111.55.52:9200'],
                ['http_address' => '10.111.55.53:9200'],
                ['http_address' => '10.111.55.54:9200'],
                ['http_address' => '10.111.55.55:9200'],
                ['http_address' => '10.111.55.56:9200'],
            ]

This is great, however ElasticSearch is designed to allow servers to fail on the cluster gracefully while others take over that server's responsibilities. Yii could randomly select a failed server, then report back that it is unable to connect to ElasticSearch. However this is not true, as any of the other servers in the connection string would be alive and active.

It would be a great addition to have Yii continue to try all nodes in the connection string before sending back a failure.

Setting a different port for elasticsearch does not work.

I set my elastic search in config as follows:

        'elasticsearch' => [
            'class' => 'yii\elasticsearch\Connection',
            'nodes' => [
                ['http_address' => '127.0.0.1:6273'],
                // configure more hosts if you have a cluster
            ],
        ],

Then i use a yii\elasticsearch\ActiveRecord instance to find the count.

$results = Topic::find()->count();

It turns out that elasticsearch is still trying to connect to the 9200 port.

Elasticsearch request failed: 7 - Failed to connect to 127.0.0.1 port 9200: Connection refused

Q	A
Yii vesion	2.0.8
PHP version	7.0.2
Operating system	win 7

Return boolean values instead of integer on ActiveRecord::updateInternal

I'm having an issue where BaseActiveRecord::save() do

return $this->update($runValidation, $attributeNames) !== false;

and the return from \yii\elasticsearch\ActiveRecord::updateInternal returns integer, which makes the model who is calling save() receive true even when the update was not successful.

If you guys agree with the change I can make the pull request for it.

Introducing a common $options attribute for all queries

Elasticsearch supports a number of options for all commands. For example, search_type, request_cache, preference, and other options are used in search.

Let's say I need to configure the preference parameter for some of my queries to target local shards only:

/myindex/mytype/_search?preference=_local

A Command has $options attribute, but a Query doesn't. So I could create a Command that does the search, but it would be much more convenient to run an ActiveQuery and get prepared models. Actually, Query has search() method that accepts $options, but setting options is not possible in one(), all(), batch() and each().

Here is my suggestion. We add the $options attribute to the Query class and a setter method for it. Something like this:

MyModel::find()->options(['preference' => '_local'])->all()

I need to solve this problem for my own project so I'm open to suggestions on how to do this best.

Create ElasticSearch `ActiveDataProvider`

Need to consider creating ElasticSearch specific ActiveDataProvider, which can use search result meta information for total item count and pagination.
It should be similar to the one created for Sphinx:
https://github.com/yiisoft/yii2-sphinx/blob/master/ActiveDataProvider.php

Invalid argument supplied for foreach()

/vendor/yiisoft/yii2-elasticsearch/DebugPanel.php:163 has

$messages = $this->data['messages'];

But $this->data['messages'] is null, thus looping over it throws the error.

Improve elasticsearch cluster and node API

This issue has originally been reported by @cebe at yiisoft/yii2#1314.
Moved here by @cebe.

---

Elastics 2.1.0 - php note: Array to string conversion Connection.php#L492

I have query search
http://prntscr.com/9dhon1

https://github.com/yiisoft/yii2-elasticsearch/blob/master/Connection.php#L492

http://prntscr.com/9dhp5r
$decoded['error'] type array not an string
http://prntscr.com/9dhr66

post_filter is not supported

The post_filter option is not supported, this is often required when using aggregations. For more info please see: https://www.elastic.co/guide/en/elasticsearch/guide/current/_post_filter.html

Would it be possible to add this feature?

Implementation of with()

I implemented support of with() for hasOne relations and I need your voice for it.

First thing is to allow $model->_id usage. It need for ActiveQuery::getModelKey() method. It cannot find any value in _id property and sets NULL key for each found record. Fixed with:

File: ActiveRecord

    public function __get($name)
    {
        if ($name == '_id') {
            return parent::__get('primaryKey');
        } else {
            return parent::__get($name);
        }
    }

    public function __set($name, $value)
    {
        if ($name == '_id') {
            parent::__set('primaryKey', $value);
        } else {
            parent::__set($name, $value);
        }
    }

    public function __isset($name)
    {
        if ($name == '_id') {
            return parent::__isset('primaryKey');
        } else {
            return parent::__isset($name);
        }
    }

    public function __unset($name)
    {
        if ($name == '_id') {
            parent::__unset('primaryKey');
        } else {
            parent::__unset($name);
        }
    }

Second thing is to increase size of limit when requesting all() in ActiveQuery::populateRecord. It needs because of ElasticSearch returns only 10 items if size is not set.

Fixed with:

File: ActiveQuery, line: 65

$models = $this->limit(count($primaryModels))->all();

With this modifications this code works as designed:

class User extends \yii\elasticsearch\ActiveRecord {
...
    public function getDbModel() {
        return $this->hasOne(\common\models\User::className(), ['_id' => 'id']);
    }
...
}

foreach(User::find()->with(['dbModel'])->all() as $elasticUser) {
    echo $user->dbModel->id . "\n";
}

no hightlighth result return?

$query = [];
        $keyword = trim($keyword);
        if ($keyword) {
            $query = [
                'multi_match' => [
                    'query' => $keyword,
                    'fields' => ['title', 'keywords', 'description', 'content']
                ]
            ];
        }

        return [
            'items' => (new Query())->from('apd', 'news')->query($query)->highlight([
                'fields' => [
                    'title' => []
                ]
            ])->all()
        ];

no hightlighth result return?

The configuration file should add a prefix

1. main-local.php
   'elasticsearch' => [
   'class' => 'yii\elasticsearch\Connection',
   'nodes' => [
   ['http_address' => '127.0.04:9200'],
   // configure more hosts if you have a cluster
   ],
   'keyPrefix' => 'dev_'
   ],
2. Connection.php
   class Connection extends Component
   {
   public $keyPrefix = '';
   }
3. ActiveRecord.php
   public static function index()
   {
   $index = Inflector::pluralize(Inflector::camel2id(StringHelper::basename(get_called_class()), '-'));
   $keyPrefix = self::getDb()->keyPrefix;
   return $keyPrefix.$index;
   }

ActiveRecord::deleteAll() delete only 10 items

On good method is to completely clear the database but only removes 10 records, this is due to the fact that the default Elastic returns 10 records (https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request- from-size.html)
Temporarily problem was solved: Model :: getDb () -> createCommand () -> deleteIndex (Model :: index ())