ongr-io / elasticsearchbundle Goto Github PK
View Code? Open in Web Editor NEWSymfony bundle for Elasticsearch with steroids
License: MIT License
Symfony bundle for Elasticsearch with steroids
License: MIT License
Create in cache virtual document classes with setters and getters. If document class already have setter or getter, use it if not create virtual.
Index import isn't working with data set sizes of 1000 and its multiples, cause of the bulk size.
With data sets of sizes like 1001 or 999 it works as expected.
[Elasticsearch\Common\Exceptions\BadRequest400Exception]
{"error":"ElasticsearchParseException[Failed to derive xcontent from org.elasticsearch.common.bytes.BytesArray@1]","status":400}
Something is wrong with this logic.
duplicating code at
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L82
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L108
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L129
etc.
FAILURES! Tests: 306, Assertions: 650, Failures: 6, Errors: 1.
Terms aggregation has new field in response, script filter completely fails.
For the given document mapping:
<?php
namespace Acme\AcmeDemoBundle\Document;
use ONGR\ElasticsearchBundle\Annotation as ES;
use ONGR\ElasticsearchBundle\Document\DocumentInterface;
use ONGR\ElasticsearchBundle\Document\DocumentTrait;
/**
* @ES\Document
*/
class Content implements DocumentInterface
{
use DocumentTrait;
/**
* @var string
*
* @ES\Property(name="name", type="string")
*/
private $name;
/**
* @var string
*
* @ES\Property(name="meta", type="object", objectName="AcmeDemoBundle:Meta")
*/
private $meta;
/**
* {@inheritdoc}
*/
public function getName()
{
return $this->name;
}
/**
* {@inheritdoc}
*/
public function setName($name)
{
$this->name = $name;
return $this;
}
/**
* {@inheritdoc}
*/
public function getMeta()
{
return $this->meta;
}
/**
* {@inheritdoc}
*/
public function setMeta(Meta $meta)
{
$this->meta = $meta;
return $this;
}
}
With the meta object definition:
<?php
namespace Acme\AcmeDemoBundle\Document;
use ONGR\ElasticsearchBundle\Annotation as ES;
/**
* @ES\Object
*/
class Meta
{
/**
* @var string
*
* @ES\Property(name="title", type="string", index="not_analyzer")
*/
public $key;
/**
* @var string
*
* @ES\Property(name="value", type="string", index="not_analyzer")
*/
public $value;
/**
* {@inheritdoc}
*/
public function getKey()
{
return $this->key;
}
/**
* {@inheritdoc}
*/
public function setKey($key)
{
$this->key = $key;
return $this;
}
/**
* {@inheritdoc}
*/
public function getValue()
{
return $this->value;
}
/**
* {@inheritdoc}
*/
public function setValue($value)
{
$this->value = $value;
return $this;
}
}
If you create one
$manager = $this->get("es.manager");
$content = new \Acme\AcmeDemoBundle\Document\Content();
$content->setId("myCustomId");
$content->setName("Acme");
$meta = new \Acme\AcmeDemoBundle\Document\Meta();
$meta->setKey("title");
$meta->setValue("Some value");
$content->setMeta($meta);
$manager->persist($content);
$manager->commit();
When you try to get an existing document an error is fired
$manager = $this->get("es.manager");
$repository = $manager->getRepository('AcmeDemoBundle:Content');
$document = $repository->find('myCustomId');
Fatal error: Argument 1 passed to Acme\AcmeDemoBundle\Document\Content::setMeta() must be an instance of Acme\AcmeDemoBundle\Document\Meta, instance of ONGR\ElasticsearchBundle\Result\ObjectIterator given.
I think that the problem is that the rawData must be converted to a document in this line https://github.com/ongr-io/ElasticsearchBundle/blob/master/Result/Converter.php#L90. Maybe a call to $value = $value->current(); fix that
Hey,
I am using a multifield to have the raw version of a field available for aggregations. I annotate the field in my document like this:
/**
* @var string
*
* @ES\Property(name="manufacturer", type="string", analyzer="simple", fields={@ES\MultiField(name="raw", type="string", index="not_analyzed")})
*
*/
private $manufacturer;
(ES
is mapped to ONGR\ElasticsearchBundle\Annotation
)
When I am trying to use the manufacturer.raw
field in my aggregation i get an empty result. The analyzed manufacturer
field works, but of course in an analyzed way, which is not what I want.
My kibana installation doesn't seem to know anything about the manufacturer.raw
field, either.
Am I missing something here?
Version of the bundle used: v0.2.0
This will allow us to change testing bundle more freely.
duplicating code at
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L82
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L108
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Tests/Functional/DSL/SearchTest.php#L129
etc.
Hi,
My symfony2 bundle is DemoBundl, but i want to customize the document_dir path set to DomoBundle/Search/Document. plz tell me how to configure?
Currently it is not possible to define custom format for date type fields:
@ES\Property(name="dateField", type="date", format="YYYY-MM-dd h:i:s")
Use these documents for set and get data.
Add the ability for mapping tool to compare tree levels by parsing precision.
Elasticsearch parses precision on mapping put, so the precision returned is actually tree levels.
To compare new mapping to the old one we need to parse it, since the old mapping is return by Elasticsearch and the new mapping is collected from documents. A few ideas of mine on how to solve this:
Make sure that proper query structure is formed for each operation.
f.e. now update operation wouldn't execute.
There are some documents, which need fields like createdAt or updatedAt. There are useful extensions for doctrine, which we could remake for ESB and use:
https://github.com/l3pp4rd/DoctrineExtensions/blob/master/doc/timestampable.md
https://github.com/l3pp4rd/DoctrineExtensions/blob/master/doc/sluggable.md
Consider the following code:
/**
* Demonstrates unexpected behaviour of ElasticsearchTestCase.
*/
class ESBProblem extends ElasticsearchTestCase {
/**
* Test for success - succeeds.
*/
public function testSuccess()
{
$repository=$this->getManager()->getRepository('AcmeTestBundle:Product');
usleep(250000);
$repository->execute($repository->createSearch());
}
/**
* Test for fail - fails.
*
* @expectedException \Exception
*/
public function testFail()
{
$repository=$this->getManager()->getRepository('AcmeTestBundle:Product');
// Let's remove usleep(250000) and see what happens.
$repository->execute($repository->createSearch());
}
}
For example, for the given document
<?php
namespace Acme\AcmeDemoBundle\Document;
use ONGR\ElasticsearchBundle\Annotation as ES;
use ONGR\ElasticsearchBundle\Document\DocumentInterface;
use ONGR\ElasticsearchBundle\Document\DocumentTrait;
/**
* @ES\Document
*/
class Content implements DocumentInterface
{
use DocumentTrait;
/**
* @var string
*
* @ES\Property(name="name", type="string")
*/
private $name;
/**
* {@inheritdoc}
*/
public function getName()
{
return $this->name;
}
/**
* {@inheritdoc}
*/
public function setName($name)
{
$this->name = $name;
return $this;
}
}
If you create one
$manager = $this->get("es.manager");
$content = new \Acme\AcmeDemoBundle\Document\Content();
$content->setId("myCustomId");
$content->setName("Acme");
$manager->persist($content);
$manager->commit();
When you try to update an existing document you never can update it
$manager = $this->get("es.manager");
$repository = $manager->getRepository('AcmeDemoBundle:Content');
$document = $repository->find('myCustomId');
$content->setName("Another Name");
$manager->persist($document); // The persist method never send an "update" bulk operation to elasticsearch
$manager->commit();
And look in the index
curl -XGET localhost:9200/my_index/my_type/_search?pretty
The content never change. I think that the problem is in the persist method of the Manager class https://github.com/ongr-io/ElasticsearchBundle/blob/master/ORM/Manager.php#L103-L107 that always sends the "create" operation. The Connection class never send any update option upsert, doc_as_upsert ... in the bulk method https://github.com/ongr-io/ElasticsearchBundle/blob/master/Client/Connection.php#L93-L115
see
ElasticsearchBundle/Tests/Functional/ORM/RepositoryTest.php
Lines 235 to 239 in 1fc70e1
Tests fails with elasticsearch 1.2.4 because this version does not have top_hits aggregation type.
/Client/Connection.php two functions have identical documentation.
/**
* Send refresh call to index.
*/
public function refresh()
{
$this->client->indices()->refresh();
}
/**
* Send refresh call to index.
*/
public function flush()
{
$this->client->indices()->flush();
}
$repository->execute($search, Repository::RESULTS_ARRAY) returns:
Array
(
[0] => Array
(
[title] => test_prod2
[description] => test_desc2
[price] => 7.79
)
[1] => Array
(
[title] => test_prod
[description] => test_desc
[price] => 0.99
)
)
There are some fields missing, particularly id
and score
. Is this by design?
Override elasticsearch-php serializer to pretty print json.
Add support for 'is' getter for bool type document fields (getActive() => isActive()).
Now when document have private bool type field ($active) ElasticsearchBundle check is declarated set and get methods (getActive(), setActive()) for it.
https://github.com/ongr-io/ElasticsearchBundle/blob/master/Mapping/MetadataCollector.php#L257
For the given document
<?php
namespace Acme\AcmeDemoBundle\Document;
use DateTime;
use ONGR\ElasticsearchBundle\Annotation as ES;
use ONGR\ElasticsearchBundle\Document\DocumentInterface;
use ONGR\ElasticsearchBundle\Document\DocumentTrait;
/**
* @ES\Document
*/
class Content implements DocumentInterface
{
use DocumentTrait;
/**
* @var string
*
* @ES\Property(name="name", type="string")
*/
private $name;
/**
* @var string
*
* @ES\Property(name="createdAt", type="date")
*/
private $createdAt;
/**
* Constructor.
*/
public function __construct()
{
$this->createdAt = new DateTime();
}
/**
* {@inheritdoc}
*/
public function getName()
{
return $this->name;
}
/**
* {@inheritdoc}
*/
public function setName($name)
{
$this->name = $name;
return $this;
}
/**
* {@inheritdoc}
*/
public function getCreatedAt()
{
return $this->createdAt;
}
/**
* {@inheritdoc}
*/
public function setCreatedAt(DateTime $createdAt)
{
$this->createdAt = $createdAt;
return $this;
}
}
If you create one
$manager = $this->get("es.manager");
$content = new \Acme\AcmeDemoBundle\Document\Content();
$content->setId("myCustomId");
$content->setName("Acme");
$manager->persist($content);
$manager->commit();
When you try to get an existing document an error is fired
$manager = $this->get("es.manager");
$repository = $manager->getRepository('AcmeDemoBundle:Content');
$document = $repository->find('myCustomId');
Fatal error: Argument 1 passed to Acme\AcmeDemoBundle\Document\Content::setCreatedAt() must be an instance of DateTime, array given.
In the elasticsearch index the createAt field has a array value, not a date value.
This invalidation is wrong
Converted and document arrays keys are being checked 1:1. So if converted array has values after which is being unset, they all will shift.
Implement PrePersist and PreUpdate annotations. They should be written above methods which:
es:index:create - OK
app/console es:index:create --manager="default" - takes a long time
after clearing the cache, it creates an index imediately
But when the index is dropped, the problem persist when trying to create another index.
Also sometimes it takes longer to drop indexes.
Member variable "index_analyzer" is not in valid camel caps format
Member variable "saerch_analyzer" is not in valid camel caps format
ElasticsearchBundle/Annotation/MultiField.php
ElasticsearchBundle/Annotation/Property.php
$params was never initialized. Although not strictly required by PHP, it is generally a good practice to add $params = array(); before regardless.
It should throw Document not found.
$foo = $repository->findBy(['foo' => 'a']); // Generates empty result set, nothing was found.
$foo->current();
Generates following notice:
Notice: Undefined index: in /var/www/vendor/ongr/elasticsearch-bundle/Result/AbstractResultsIterator.php line 96
There should be no notice, error or empty result is preferred.
This would be useful if tests are run on older versions when some features are not supported. We could skip them by knowing from documentation if they are supported by current running elasticsearch version or not.
Related: #60
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.