friendsofsymfony/foselasticabundle

Elasticsearch PHP integration for your Symfony project using Elastica.

foselasticabundle's Issues

Mapping pdf content

Hi,

i have an entity Article with a Title, now I need to save and index a PDF file. Is this correct?

$em = $this->getDoctrine()->getEntityManager(); $entity=new Article(); $entity->setTitle('This is the title'); $em->persist($entity); $em->flush(); $doc = new Elastica_Document($entity->getId()); $doc->addFile('file', 'urlToUploadedFile.pdf'); $entity->addDocument($doc);

Thanks in advance
v.

Index wide finder is not registered

I'm trying to use the index wide finder and have registered it as mentioned in the README:

foq_elastica:
    clients:
        default: { host: localhost, port: 9200 }
    indexes:
        fmdb:
            client: default
            finder:
            // rest of my configuration

Trying to get the service from the container just gives me an 500 and dumping the container reveals no registered service. Did I miss something? The finders for my types are all registered.

Add support for composer

It would be great if this bundle would support composer.

Exception handling

In the ObjectPersister whether exceptions are thrown or not is controlled by a class property that is hard coded to true. This needs to be settable or if we want it to always throw exceptions then the catching and checking this property can be removed.

In the Doctrine listeners there is checking for exceptions which are logged if there is a logger and just suppressed if there isn't. This does not fit in with the idea of letting exceptions escape the bundle unless the client is over ridden to catch them. If this is done then no exceptions should reach the listener anyway.

I think that all the exception handling in these classes should be removed as it can be handle in the overridden client. Likewise as logging is done in the client there is no need to do it in these classes. If it needs to be done here then it only needs to be done in the ObjectPersister and not the Doctrine listeners as well. @jmikola what do you think?

[Feature Request] Source Excludes/Includes

See http://www.elasticsearch.org/guide/reference/mapping/source-field.html

For example:

{
    "my_type" : {
        "_source" : {
            "includes" : ["path1.*", "path2.*"],
            "excludes" : ["pat3.*"]
        }
    }
}

In Elastica:

$mapping = new Elastica_Type_Mapping();
$mapping->setSource(array('excludes'=>array('file')));  // Don't save raw file data

Use ClassMetadataInfo::getFieldValue() instead of calling getter methods

This came up in #67, and @stof mentioned that we were generating method names for field getters elsewhere in the bundle. Those instances should be replaced with calls to ClassMetadataInfo::getFieldValue().

Add custom configuration parameters

Hi,

I'm using ElasticaBundle to index the contents of pages through a custom model_to_elastica_transformer service in order to add computed fields that have no entry in the database (Entity class). For example:

From my configuration:
doctrine:
model_to_elastica_transformer:
service:
company_search.transformers.nodetransformer
mappings:
id:
title:
content: { handlerclass: Company\AdminNodeBundle\Entity\Node, handlermethod: getSearchContentForNode }

My transformer loops over the fields and gets the mapping definitions. If the field has no handlerclass/handlermethod parameter, the getter for that field is executed to fetch the contents of that field. If however there is a handlerclass/handlermethod, that gets called for the value of the field. Adding a getter for the content field in the entity would also solve this in theory but for various reasons it would not be good behaviour to solve this that way.

So in conclusion I would need to add the handlerclass and handlermethod as scalarNode entries to the configuration class. I've tried using FOQElastica as a parent and override, but since the call to the configuration class is inside the FOQElasticaExtension class it won't work. Overriding the FOQElasticaExtension class also seems impossible.

Non-unique mapping field possible

As pointed out by @stof:

The bundle will currently never complain if you configure it to use a non-unique field for the mapping between ES and Doctrine

The best solution would probably be a validate command, checking all configured types.
It cannot be done in the DI extension as you don't have the Doctrine mapping yet and doing it at runtime would add overhead

Boolean field not indexed

Hi guys,

Could you please confirm this issue ?

I have a boolean field in this config wich is not indexed by the bundle, but it works with the raw Elastica Client:

foq_elastica:
    clients:
        default: { host: localhost, port: 9200 }
    indexes:
        comparator:
            client: default
            settings:
            types:
                product:
                    mappings:
                        ...
                        isDiscount: { type: boolean, store: yes, index: not_analyzed }
                        ....
                    persistence:
                        driver: orm
                        model: Ac\CoreBundle\Entity\Product
                        provider:
                        listener:

Mapping ManyToOne/OneToOne entity

Hello,

I would like the possibility to add external data to my indexed entity, but I don't see any way to map others fields than its current attributes.

For example, I have a Foobar entity that is joined with a Bazfoo entity

<?php

/** @ORM\Entity */
class Foobar
{
    /** @ORM\Column(type="string") */
    protected $myvar;

    /**
     * @ORM\JoinColumn
     * @ORM\ManyToOne(targetEntity="Bazfoo")
     */
    protected $bazfoo;

    // ...
}

/** @ORM\Entity */
class Bazfoo
{
    /** @ORM\Column(type="string") */
    protected $othervar;

    // ...
}

Then under the config foq_elastica.indexes.website.types.foobar, I'd also like to map $otherway to use it during my research with the Finder::find() method:

mappings:
    myvar:
    # bazfoo.othervar: ?
persistence:
    driver: orm
    model: Path\To\Foobar
    finder:

The only solution I found is to add getters in Foobar (ie. getOthervar() { return $this->bazfoo->getOtherwar(); }), but I would like a less hackish solution... Plus if indexed data could be updated when Foobaror Bazfoo are changed, that would be awesome.

null_value is missing in mapping

Hi guys,

The "null_value" for default value of null entries is missing in the mapping.

Here is the doc http://www.elasticsearch.org/guide/reference/mapping/core-types.html

I really need to keep a field as null in my index.

Thanks,

Pierre.

Couldnt connect to host, ElasticSearch down?

When I write " php app/console foq:elastica:populate " I get the following error:

[Elastica_Exception_Client]
Couldnt connect to host, ElasticSearch down?

My config.yml is:

foq_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
website:
client: default
types:
user:
mappings:
username: { boost: 5 }
persistence:
driver: orm
model: myAplicattion\MyBundle\Entity\User
provider:
query_builder_method: createIsActiveQueryBuilder
finder:

And I have the method " createIsAtiveQueryBuilder " in my userRepository.php.

I do not know that I have to do now.
I do not speak much English, would appreciate long answers. ; )

ODM Reference Mapping

I have two documents defined in Doctrine Mongo ODM: Item and Category.
They are related through the field "categories" in the Item document, which is defined as a ReferenceMany, i.e. an Item can have many Categories.

The problem is I can't manage to have FOQElasticaBundle index it right. If I try to map the categories field, running foq:elastica:populate results this error message:

Catchable Fatal Error: Object of class Proxies\PWCategoryBundleDocumentCategoryProxy could not be converted to string in ...vendor/bundles/FOQ/ElasticaBundle/Transformer/ModelToElasticaAutoTransformer.php line 74

How do I get elastica to index the referenced documents?
Are references not supported in ElasticaBundle, or am I missing out on something?

postRemove event listener exception

In the ORM postRemove event listener, the id of the entity is used to delete the corresponding record in elasticsearch. But at this moment, the id of the entity is no more available.

Indeed, when removing the entity, doctrine seems to set the id to null. See : https://github.com/doctrine/doctrine2/blob/master/lib/Doctrine/ORM/UnitOfWork.php#L980

This leads to an exception in elastic search : https://github.com/ruflin/Elastica/blob/master/lib/Elastica/Type.php#L209

In the log : at Elastica_Type ->deleteById (null)

Searching by part of word

Hi. How can I do it?

Searching word is 'Infran'.

In my index is 'InfranView' not 'Infran'. How can I fetch 'InfranView' typing only 'Infran'?

Help

doctrine.odm.mongodb vs doctrine.odm.mongodb.document_manager

hey there, just a heads up,
in the Resources/config/mongodb.xml you have changed the injected service ids to 'doctrine.odm.mongodb' from 'doctrine.odm.mongodb.document_manager' and this change breaks symfony2 as the 'doctrine.odm.mongodb' is an unknown service.
thank you for your great work!

Implement own ElasticaToModelTransformer

Hi,

I'm trying to implement a custom ElasticaToModelTransformer. In the config for the index I've added the following parameters:

elastica_to_model_transformer:
service: my_search.transformers.elasticatransformer

And the service definition is:
my_search.transformers.elasticatransformer:
class: Company\SearchBundle\Transformers\ElasticaTransformer

When I try to run I get the following error:
Warning: Missing argument 1 for FOQ\ElasticaBundle\Doctrine\AbstractElasticaToModelTransformer::__construct()

Do you have any idea how I can add the parameters to the service so AbstractElasticaToModelTransformer has all parameters the constructor needs? When I look at the constructor I see it takes the EM, which I can take care off, but I don't know how to access the string and array with options.

Regards,
Kim

Use finder and access ES response not possible

In my opinion it should be possible to use the finder function to get back the entities but at the same time get access to the ElasticSearch response.
This is important e.g. to access the facets and build a faceted search.
From my understanding of the code this is not possible at the moment.

Call to undefined method getManagerForClass()

Hi all,

I try to install (and use) the FOQElasticaBundle following this tuto (http://l3l0.eu/lang/en/2011/06/elastic-search-in-symfony-2/)
and the official documentation (http://knpbundles.com/Exercise/FOQElasticaBundle)

When I run the command "sudo app/console foq:elastica:populate"

I have this error :

PHP Fatal error: Call to undefined method Symfony\Bundle\DoctrineBundle\Registry::getManagerForClass() in /path-to-my-project/vendor/bundles/FOQ/ElasticaBundle/Doctrine/ORM/Provider.php on line 55

I think I made something wrong because it seems I am the one to have this error (a search on google return me only one result : https://github.com/Exercise/FOQElasticaBundle/pull/63)

Help me, you're my only hope....

Thank you
Greg

Instalation namespace problem

is

// app/autoload.php

$loader->registerNamespaces(array(
...
'FOQ' => DIR.'/../src',
));

should be

// app/autoload.php

$loader->registerNamespaces(array(
...
'FOQ' => DIR.'/../vendor/bundles',
));

Unrecognized options "listener"

According to the docs, automatical index updating will be enabled by defining the "listener" option. Tried with the latest git version, seems this option is unknown?

doctrine:
    driver: mongodb
    model: Application\UserBundle\Document\User
    provider:
    finder:
    listener:

=>

Fatal error: Uncaught exception 'Symfony\Component\Config\Definition\Exception\InvalidConfigurationException' with message 'Unrecognized options "listener" under "foq_elastica.indexes.website.types.item.doctrine"`

thanks

Remove ES connection error suppression from master

Revert eb19002 in master and reimplement in a separate branch. Additionally, consider enabling the original solution as a configurable behavior of the bundle.

Getting an undefined index error using the finder

I indexed my entities and used the finder to search across all of my types. But I only got the following exception:

Notice: Undefined index: -33 in /opt/local/apache2/htdocs/fmdb/vendor/exercise/elastica-bundle/FOQ/ElasticaBundle/Transformer/ElasticaToModelTransformerCollection.php line 55

Some debugging revealed that the class map used in ElasticaToModelTransformerCollection.php showed that all entity classes except one point to the entities. The class in question points to the proxy which seems to be wrong.

This worked in a previous version of the bundle but I can't tell exactly when the error occured.

Finder result limit

According to the docs the limit argument is not required when using the finder, but doing so will throw a warning:

$items = $finder->find('test');

Warning: Missing argument 2 for FOQ\ElasticaBundle\Finder\TransformedFinder::find()

Propel support is not in the 2.0 branch

Is there a reason for this? Or was it just an oversight that the Propel PR was made against master? I honestly don't recall if we created the 2.0 branch before or after Propel was integrated.

CC: @willdurand

Can't user another server than localhost

Hi,

I am trying to use this bundle using an external server and when I put the ip in the configuration instead of localhost it doesn't change anything. Anyone have an idea?

When I put this code, it still get connected to the localhost, but when I used to third part software to redirect the localhost and the port to another ip then it works perfectly.

foq_elastica: clients: default: { host: "the ip of the server (other than localhost)" port: "the port configured on the server" } indexes: website: client: default types: post: mappings: title: { boost: 8 } doctrine: driver: orm model: ......\Entity\Post provider: batch_size: 10 listener: finder: user: mappings: username: { boost: 5 } firstName: { boost: 3 } lastName: { boost: 3 } top1: { boost: 2 } top2: { boost: 2 } top3: { boost: 2 } doctrine: driver: orm model: ......\Entity\User provider: batch_size: 10 listener: finder:

Need a way to do populate in batches

It seems that even when setting clear_object_manger: true there is still enough of a memory leak (in Doctrine?) that a large set of objects cannot be indexed without going over memory limits.

Using --no-debug when starting the command helps, but not enough.

This may be something that has to be done on a case by case basis in one's own project (like I am working on), but I think it might be generalizable as well.

symfony 2.0.5 autoloading

Hello,
I am not sure but it seems that classes like "Elastica_Client" cannot be loaded. I have registered "'Elastica' => DIR.'/../vendor/elastica/lib'," and files exist.

php app/console -e=dev foq:elastica:populate
Fatal error: Class 'Elastica_Client' not found in /Applications/MAMP/htdocs/bop/src/FOQ/ElasticaBundle/Client.php on line 12

Do I do something wrong?

Dynamic mapping

Hi,

Would it be possible to have a "dynamic mapping" feature ? It would be great to be able to get the mapping information (fields + boost, type, etc.) from an entity (MyEntity::getMapping()).

Mapping attachment-type

I'm looking for a strategy to map @mongodb\File to ES attachment-type.
http://www.elasticsearch.org/guide/reference/mapping/attachment-type.html
https://github.com/ruflin/Elastica/blob/master/lib/Elastica/Document.php

What I want is a class Article where a "private $articlePdfContent" is indexed into ES attachment-type (with tika).

May you help me?
v.

Multiple EntityManagers?

It it possible to use multiple EntityManagers? It looks like the Bundle always uses the default manager. I couldn't see a way to configure different EMs for different indexes or types.

PS Thank you for this wonderful, easy-to-use and well-documented bundle.

conf client

if i set a client
//config.yml
foq_elastica:
clients:
default: { host: test.com, port: 9200 }

I need to change the ...\elastica\lib\Elastica\Client.php to see an effect:
from
public function __construct(array $config = array()) {
$this->setConfig($config);
}
to
public function __construct(array $config = array()) {
$this->setConfig($config[0]);
}

InvalidArgumentException when deleting an entity

Hi,

I tried to delete one of my entities and it failed with an InvalidArgumentException:

Stack Trace
in /opt/local/apache2/htdocs/fmdb_rel/vendor/elastica/lib/Elastica/Type.php at line 178   
     */
    public function deleteById($id) {
        if (empty($id) || !trim($id)) {
            throw new InvalidArgumentException();
        }
        return $this->request($id, Elastica_Request::DELETE);
    }
at Elastica_Type ->deleteById (null) 
in /opt/local/apache2/htdocs/fmdb_rel/vendor/bundles/FOQ/ElasticaBundle/Persister/ObjectPersister.php at line 81    
at ObjectPersister ->deleteOne (object(Title)) 
in /opt/local/apache2/htdocs/fmdb_rel/vendor/bundles/FOQ/ElasticaBundle/Doctrine/ORM/Listener.php at line 43    
at Listener ->postRemove (object(LifecycleEventArgs)) 
in /opt/local/apache2/htdocs/fmdb_rel/vendor/doctrine-common/lib/Doctrine/Common/EventManager.php at line 64    
at EventManager ->dispatchEvent ('postRemove', object(LifecycleEventArgs)) 
in /opt/local/apache2/htdocs/fmdb_rel/vendor/doctrine/lib/Doctrine/ORM/UnitOfWork.php at line 812    
at UnitOfWork ->executeDeletions (object(ClassMetadata)) 
in /opt/local/apache2/htdocs/fmdb_rel/vendor/doctrine/lib/Doctrine/ORM/UnitOfWork.php at line 316    
at UnitOfWork ->commit () 
in /opt/local/apache2/htdocs/fmdb_rel/vendor/doctrine/lib/Doctrine/ORM/EntityManager.php at line 334    
at EntityManager ->flush () 
in /opt/local/apache2/htdocs/fmdb_rel/src/Fmdb/CoreBundle/Controller/TitleController.php at line 118    
at TitleController ->deleteAction ('1') 
in at line     
at ReflectionMethod ->invokeArgs (object(TitleController), array('1')) 
in /opt/local/apache2/htdocs/fmdb_rel/vendor/bundles/JMS/SecurityExtraBundle/Security/Authorization/Interception/MethodSecurityInterceptor.php at line 73    
at MethodSecurityInterceptor ->invoke (object(MethodInvocation), array('roles' => array('ROLE_ADMIN'), 'run_as_roles' => array(), 'param_permissions' => array(), 'return_permissions' => array())) 
in kernel.root_dir/cache/dev/classes.php(8394) : eval()'d code at line 1    
at {closure} ('1') 
in at line     
at call_user_func_array (object(Closure), array('1')) 
in kernel.root_dir/cache/dev/classes.php at line 3880    
at HttpKernel ->handleRaw (object(Request), '1') 
in kernel.root_dir/cache/dev/classes.php at line 3850    
at HttpKernel ->handle (object(Request), '1', true) 
in kernel.root_dir/cache/dev/classes.php at line 4785    
at HttpKernel ->handle (object(Request), '1', true) 
in kernel.root_dir/bootstrap.php.cache at line 547    
at Kernel ->handle (object(Request)) 
in /opt/local/apache2/htdocs/fmdb_rel/web/app_dev.php at line 20

I used the following code to delete the entity identified by its id:

public function deleteAction($id)
    {
        $em = $this->getDoctrine()->getEntityManager();
        $title = $em->find('Fmdb\CoreBundle\Entity\Title', $id);

        $em->remove($title);
        $em->flush();

        return new Response;
    }

Any ideas why this happens?

Parameterized search with Finder?

Hello Guys, i'm wondering, how to make parameterized search with finder?, i.e. fix one mapping parameter, and search across records with that fixed param? Thanks in advance.

Inconsistent searches

So I'm using this on a food database, but the searches are very inconsistent... well they seem to consistently rotate between the following two objects..

Search Query: "butter salted"

##################
# Initial search #
##################

object(Elastica_ResultSet)#106 (4) {
  ["_results":protected]=>
  array(10) {
    [0]=>
    object(Elastica_Result)#105 (1) {
      ["_hit":protected]=>
      array(5) {
        ["_index"]=>
        string(11) "website"
        ["_type"]=>
        string(4) "food"
        ["_id"]=>
        string(4) "1001"
        ["_score"]=>
        float(37.11759)
        ["_source"]=>
        array(1) {
          ["name"]=>
          string(14) "Butter, salted"
        }
      }
    }

[...]


#################
# After refresh #
#################

object(Elastica_ResultSet)#106 (4) {
  ["_results":protected]=>
  array(10) {
    [0]=>
    object(Elastica_Result)#105 (1) {
      ["_hit":protected]=>
      array(5) {
        ["_index"]=>
        string(11) "website"
        ["_type"]=>
        string(4) "food"
        ["_id"]=>
        string(5) "16132"
        ["_score"]=>
        float(9.256613)
        ["_source"]=>
        array(1) {
          ["name"]=>
          string(31) "Tofu, salted and fermented fuyu"
        }
      }
    }
[...]

"Butter, salted" should always be shown first, but it keeps rotating between the above two objects that is way wrong... anyone know why?

IndexMissingException[[website] missing]

I have the error : IndexMissingException[[website] missing]

I have this in my config:

foq_elastica:
    clients:
        default: { host: localhost, port: 9200 }
    indexes:
        website:
            client: default
            types:
                application:
                    mappings:
                        name: { boost: 5 }
                        description: { boost: 3 } 
                    doctrine:
                        driver: orm
                        model: HALL\TestBunde\Entity\Application
                        provider: 
                            query_builder_method: getAllQueryBuilder
                        finder:

I run the command:

php app/console foq:elastica:populate

The result I have is:

Reseting indexes
Setting mappings
Populating indexes
Indexing foq_elastica.provider.website.application, 100.0% (2/2), 9 objects/s
Done

In my controller I have:

        $finder   = $this->get('foq_elastica.finder.website.application');
        $messages = $finder->find('name:test', 10);

And the following error is thrown:et thrown the error:

IndexMissingException[[website] missing]
500 Internal Server Error - Elastica_Exception_Response 

Any ideas? Thanks! I've tried different versions of Elastica......same problem.

Paginating results for query with script

Hi,

I was having an issue with paginating a query with a script in it. I would get results, but the page count would cause an error in elasticsearch(saying its not properly formed json) and return 0.

Anyways I wanted to log some changes that fixed pagination for me.

https://gist.github.com/1824935

Thanks
Mic Johnson

Skipping fields with false value while indexing

this line https://github.com/Exercise/FOQElasticaBundle/blob/f071c98143fd9c99fd6521284bc29833a0d2296d/Transformer/ModelToElasticaAutoTransformer.php#L56 removes fields with false value (actually array_filter does) while creating indexes, so if i have a boolean field in the db, fields with true will be indexed, but fields with false will not be indexed. So i cant search by false value.

Example of what i have after indexing http://cl.ly/1W1l0E2t282k1z1v2N3T, expecting http://cl.ly/020P1X2s2Y2Y2e0c0Q0C . Notice the field "showAsPublic" exists in the second screenshot with true value, but doesnt exist at first screenshot because its a false value.

get highlights for search results

I'm indexing pages. Every page has a title and text. Now together with my pagetitle I want to show the highlights in my searchresults, kind of like google does. Is there a way to do this?

Memory crash for batch

Hi guys,

I've this crash when trying to populate 60 000 objects:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 392213 bytes) in /home/sites/myproject/prod/vendor/elastica/lib/Elastica/Transport/Http.php on line 78

Any idea please ?

[Feature Request] Propel Support

Hi,

It would be nice to support Propel :)

Thanks !

[Feature Request] Joins

It would be nice to have the ability to do joins within the results... ie:

$finder = $this->get('foq_elastica.finder.website.index');

$finder->addJoin('i.someTableA', 'a');
$finder->addJoin('a.someTableB', 'b');

$results = $finder->find('foo', 10);

index a field with 0 or NULL

if I index a field with 0 or NULL (php app/console foq:elastica:populate), the field is not stored in the index .
Exists a way to store this values?

method_exists doesn't find php magic methods

Inside the ModelToElasticaAutoTransformer there is a check on the entity class for the given property

if (!method_exists($class, $getter)) {

for me, this is a problem, because in my doctrine entities I have __call __get and __set magic methods for translations, and method_exists return false.

Is there any problem by using is_callable instead? If you wish I could send a pr.

Regards.

Entity relations

Hi,

is there any way to do index all realation between models. For example:

I have a product model and this entiy join to another model by yml. I would like this related model indexing too.

E.g:

mappings:
name: { boost: 5, analyzer: my_analyzer} //Acme/Entity/Product
description: { boost: 3, analyzer: my_analyzer} //Acme/Entity/Product
geoData.placeName: { boost: 1 } //Acme/Entity/GeoData

i hope someone know what i mean

thx!

Missing License

.

ElasticSearch and Doctrine Extensions translatable

Hi,
I am using Doctrine Extensions Translatable and Elastic search. So for it worked well for default locale, but when i try to
insert translation it bumps into an error:

ElasticSearchParseException[Failed to derive xcontent from (offset=0, length=2): [91, 93]] 

Elastica_Exception_Response: ElasticSearchParseException[Failed to derive xcontent from (offset=0, length=2): [91, 93]] (uncaught exception) at /var/www/site.com/Symfony/vendor/elastica/lib/Elastica/Transport/Http.php line 103 

second locale is en:

        $article = $em->getRepository('WebsiteSharedBundle:News')->find(1);
        $article->setTitle('News 1 in EN');
        $article->setContent('News 1 in EN');
        $article->setTranslatableLocale('en_us'); 
        $em->persist($article);
        $em->flush();       

Search returns nothing

I got beautifully working Elastica using this bundle. Everything works perfectly, but just befor deploy it stoped returning anything. It always returns null, despite the fact indexes are loaded and I can reload it anytime. My confing is like that:

config.yml

foq_elastica:
    clients:
        default: { host: localhost, port: 9020 }
    indexes:
        website:       
            client: default
            types:
                product:
                    mappings:
                        name:    { boost: 5 }
                        subname: { boost: 3 }
                        sn:      { boost: 3 }
                    doctrine:
                        driver: orm
                        model:  Cataloger\CatalogBundle\Entity\Product
                        provider:
                        finder:
                        listener:
                            insert: true
                            update: false
                            delete: true

Product.class.php

namespace Cataloger\CatalogBundle\Entity;

use Doctrine\ORM\Mapping as ORM;
use Symfony\Component\Validator\Constraints as Assert;
use Doctrine\Common\Collections\ArrayCollection;
/**
 * @ORM\Entity(repositoryClass="Cataloger\CatalogBundle\Repository\ProductRepository")
 * @ORM\Table(name="products")
 * @ORM\HasLifecycleCallbacks()
 * 
 */
class Product
{
    /**
     * @ORM\Id
     * @ORM\Column(type="integer")
     * @ORM\GeneratedValue(strategy="AUTO")
     */
    protected $id;

    /**
     * @ORM\Column(type="string")
     * @ORM\Column(unique=true)
     * @Assert\NotBlank()
     */
    protected $name;

    /**
     * @ORM\Column(type="string")
     * @ORM\Column(unique=true)
     */
    protected $slug;

    /**
     * @ORM\Column(type="string")
     * @ORM\Column(unique=true)
     * @Assert\NotBlank()
     */
    protected $sn;

    /**
     * @ORM\Column(type="string")
     * @ORM\Column(nullable=true)
     */
    protected $subname;

    /**
     * @ORM\ManyToOne(targetEntity="Category", cascade={"persist", "remove"})
     */
    protected $manufacture;

    /**
     * @ORM\Column(type="text")
     * @ORM\Column(nullable=true)
     */
    protected $description;

    /**
     * @ORM\Column(type="boolean")
     * @ORM\Column(nullable=true)
     */
    protected $is_new;

    /**
     * @ORM\Column(type="boolean")
     * @ORM\Column(nullable=true)
     */
    protected $is_bestseller;

    /**
     * @ORM\Column(type="datetime")
     */
    protected $created;

    /**
     * @ORM\OneToMany(targetEntity="Photo", mappedBy="refId", cascade={"persist", "remove"})
     */
    protected $photos;

    /**
    * @ORM\ManyToOne(targetEntity="Category", inversedBy="products")
    */
    protected $category;

    ....

}

Controller

    public function searchAction()
    {
    $request = $this->getRequest();
        if($request->isXmlHttpRequest()){

            $phrase = $request->request->get('phrase');
            $finder = $this->container->get('foq_elastica.finder.website.product');
            $products = $finder->find('szczypce', 12);
            return $this->render('CatalogerCatalogBundle:Category:search.xml.twig', array('products'=>$products));
        }else{
            throw $this->createNotFoundException('Not XmlHttpRequest!');
        }
    }

Did i do something wrong? I con not sole this so I'm asking for help.

MongoCursorTimeoutException

Hi,

I'm getting this error when populating the index, any ideas how can I increase the timeout ? Thanks

Indexing foq_elastica.provider.website.vendor, 38.3% (57900/151360), 4 objects/s

[MongoCursorTimeoutException]
cursor timed out (timeout: 30000, time left: 0:0, status: 0)

foq:elastica:populate

Queries generated by the Doctrine ORM Provider slow down untill they are processing < 1 obj/s

Our use case is indexing a table from mysql that has ~ 10M rows.

The queries generated by the orm provider look like

SELECT ... FROM Customer c0_ LIMIT 100 OFFSET 141300
SELECT ... FROM Customer c0_ LIMIT 100 OFFSET 141400
SELECT ... FROM Customer c0_ LIMIT 100 OFFSET 141500
SELECT ... FROM Customer c0_ LIMIT 100 OFFSET 141600
SELECT ... FROM Customer c0_ LIMIT 100 OFFSET 141700

This query means that most sql servers will have to walk through the result set to return 100 items, horribly inefficient and will never finish on a table with 10M rows.

I propose the queries issued via ORM should use

Query::iterate(), along with $this->objectManager->detach() for each returned object.