whatwedo/searchbundle

Full text search on Doctrine entities without external dependencies

ATTENTION: This bundle is abandoned. It is now part of araise: https://github.com/araise-dev/SearchBundle

---

whatwedoSearchBundle

This bundle is part of the araise.dev framework.

Documentation

whatwedo.github.io/SearchBundle

License

This bundle is under the MIT license. See the complete license in the bundle: LICENSE

searchbundle's Issues

Document a procedure to uninstall the bundle?

Hi,

I'm removing this bundle from an application and it left some not common footprint.

As you ship a Doctrine Migration to create the table and inject it automatically:

SearchBundle/src/DependencyInjection/whatwedoSearchExtension.php

Lines 41 to 46 in 733aefc

	$doctrineConfig = $container->getExtensionConfig('doctrine_migrations');
	$container->prependExtensionConfig('doctrine_migrations', [
	'migrations_paths' => array_merge(array_pop($doctrineConfig)['migrations_paths'] ?? [], [
	'whatwedo\SearchBundle\Migrations' => '@whatwedoSearchBundle/Migrations',
	]),
	]);

This is automatically added:

The thing is when removing this bundle from an existing application, this migration stays in the migration_versions table and this message is then displayed by Doctrine:

[WARNING] You have 1 previously executed migrations in the database that are not registered migrations.

There is no way for an application developer to clean this excepts manually:

DELETE FROM doctrine_migration_versions WHERE version = 'whatwedo\\SearchBundle\\Migrations\\Version20220602150539';

Maybe that should be documented, or maybe you had this issue before and there is a better way?

Cheers

Rollback Transaction if an error occured

Allow predefined filters on Search

in some cases, a list of archived entities is shown. or some other type of filtered list.

the search does not consider this filter.

Allow to filter the search results by a given filter.

pre and post Search hook

preSearch Hook -> modify queryBuilder
postSearch Hook -> modify Id's

Add new Anntotation

php8 Attributes

Allow using Attributes instead of Annotations.

eg:

#[Index]
private string $firstname;

Error installing in Symfony 4.4.4

When I try to install the bundle, I got the next message in cache:clear script:

Cannot autowire service "whatwedo\SearchBundle\Manager\IndexManager": argument "$doctrine" of method "__construct()" references interface "Symfony\Bridge\Doctrine\RegistryInterface" but no such service exists. Try changing the type-hint to "Doctrine\Persistence\ManagerRegistry" instead.

I see a pull request making the change necessary to fix this issue, is going to be merged?

Search results always empty

Hi!,

I've installed this bundle; no problem what so ever. Also indexing is good, migrated properly.
But when i try to search, the results are always empty. I've debugged the code, and it seems that :minscore in the QueryBuilder is not working properly. For example, in the whatwedo_search_index table there is a row with content 'home'.

When searching on 'home' using the IndexRepository, zero results.

 public function indexAction()
    {
        $results = $this->searchRepository->search('home', Page::class);

        dd($results);
     }

When commenting the minscore in the query builder, I do get the result. The minScore for home is 6 (strlen('home') * 1.5). But the minscore of the actual result is 5.5.

Any idea? Using MySQL 5.7 here.

SQL query error while conducting search

Exception:

DriverException
An exception occurred while executing 'SELECT w0_.foreign_id AS foreign_id_0, MATCH(w0_.content) AGAINST (?) AS sclr_1 FROM whatwedo_search_index w0_ WHERE MATCH(w0_.content) AGAINST (?) > 0 OR w0_.content LIKE ? GROUP BY w0_.foreign_id ORDER BY sclr_1 DESC' with params ["test", "test", "%test%"]:

SQLSTATE[42000]: Syntax error or access violation: 1055 Expression #2 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'pm.w0_.content' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by

Stacktrace:

in AbstractMySQLDriver.php line 106
--
at AbstractMySQLDriver->convertException('An exception occurred while executing \'SELECT w0_.foreign_id AS foreign_id_0, MATCH(w0_.content) AGAINST (?) AS sclr_1 FROM whatwedo_search_index w0_ WHERE MATCH(w0_.content) AGAINST (?) > 0 OR w0_.content LIKE ? GROUP BY w0_.foreign_id ORDER BY sclr_1 DESC\' with params ["test", "test", "%test%"]:SQLSTATE[42000]: Syntax error or access violation: 1055 Expression #2 of SELECT list is not in GROUP BY clause and contains nonaggregated column \'pm.w0_.content\' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by', object(PDOException))in DBALException.php line 166
at DBALException::wrapException(object(Driver), object(PDOException), 'An exception occurred while executing \'SELECT w0_.foreign_id AS foreign_id_0, MATCH(w0_.content) AGAINST (?) AS sclr_1 FROM whatwedo_search_index w0_ WHERE MATCH(w0_.content) AGAINST (?) > 0 OR w0_.content LIKE ? GROUP BY w0_.foreign_id ORDER BY sclr_1 DESC\' with params ["test", "test", "%test%"]:SQLSTATE[42000]: Syntax error or access violation: 1055 Expression #2 of SELECT list is not in GROUP BY clause and contains nonaggregated column \'pm.w0_.content\' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by')in DBALException.php line 146
at DBALException::driverExceptionDuringQuery(object(Driver), object(PDOException), 'SELECT w0_.foreign_id AS foreign_id_0, MATCH(w0_.content) AGAINST (?) AS sclr_1 FROM whatwedo_search_index w0_ WHERE MATCH(w0_.content) AGAINST (?) > 0 OR w0_.content LIKE ? GROUP BY w0_.foreign_id ORDER BY sclr_1 DESC', array('test', 'test', '%test%'))in Connection.php line 902
at Connection->executeQuery('SELECT w0_.foreign_id AS foreign_id_0, MATCH(w0_.content) AGAINST (?) AS sclr_1 FROM whatwedo_search_index w0_ WHERE MATCH(w0_.content) AGAINST (?) > 0 OR w0_.content LIKE ? GROUP BY w0_.foreign_id ORDER BY sclr_1 DESC', array('test', 'test', '%test%'),array(2, 2, 2), null)in SingleSelectExecutor.php line 50
at SingleSelectExecutor->execute(object(Connection), array('test', 'test', '%test%'), array(2, 2, 2))in Query.php line 334
at Query->_doExecute()in AbstractQuery.php line 967
at AbstractQuery->executeIgnoreQueryCache(null, 3)in AbstractQuery.php line 922
at AbstractQuery->execute(null, 3)in AbstractQuery.php line 750
at AbstractQuery->getScalarResult()in IndexRepository.php line 68
at IndexRepository->search('test')in SearchController.php line 38

How to use xml instead of using annotation.

Hello, in my project I'm using XML to declare a doctrine entity; How do i declare whatwedo\SearchBundle\Annotation\Index in an XML file?

Use clean queryBuilder

if the getQueryBuilder() is overwritten in the Repository the indexing fails

Symfony4 compatibility

See: http://symfony.com/blog/helping-prepare-for-symfony-4-bundle-support

• Make bundle Symfony4 compitable
• Create a receipt for Flex
• Mark it as done on the Symfony4 Bundle Compatibility List

Mistake in the documentation

https://github.com/whatwedo/SearchBundle/blob/master/Resources/doc/index.md#use-the-bundle

$users $this->em->getRepository('whatwedoSearchBundle:Index') should be $users $this->em->getRepository('AgencyUserBundle:User')

Wildcards

% are currently working as wildcard. we should think about adding * as a wildcard.

Support other types of primary key?

May I know if this project have a plan to support other types of primary key such as:

Symfony uuid:

     * @ORM\Column(name="id", type="guid")
     * @ORM\GeneratedValue(strategy="UUID")

&

Ramsey uuid-doctrine:

     * @ORM\Column(type="uuid", unique=true)
     * @ORM\GeneratedValue(strategy="CUSTOM")
     * @ORM\CustomIdGenerator(class=UuidGenerator::class)

do not populate search with inherited Values

If a value is inherited, populate the search only with the inherited model/value

exp.

class Identity {
  // @Index()
  public $name
}

class Customer extends Identity {
}

class Person extends Identity {
}

only Identity::name should be populated in the search Table

in the Search results the inherited Models should be included

Injecting FormatterManager in IndexListener causes ServiceCircularReferenceException

After upgrading a project to Symfony 5.1.10 an ServiceCircularReferenceException occours.

Circular reference detected for service "doctrine.orm.default_entity_manager", path: "doctrine.orm.default_entity_manager -> doctrine.orm.default_entity_manager".

removing the injectiion of the FormatterManager in IndexListener constructor resolves the issue.

GC every 5'000 objects

in addition to run GC after every entity, do a gc after every 5'000 objects in addition.

Populate Command should use Repository::findAll()

if the findAll() function filters the result. the PopulateCommand doesn't filter these entities.

public function findAll()
{
    $queryBuilder = $this->createQueryBuilder('m');
    $queryBuilder->where('m.isActive=true');
    return $queryBuilder->getQuery()->getResult();
}

other possibility is to define a "findAllEntitiesToIndex()" method on the Repo. And called instead of the findAll(), if defined.

Check if argument entity is a Doctrine Entity

Global Search functionality

Add functionality for Global searching:

• define on Entity by annotation: search-Fields/Methods (ex. fullname, lastname,...); route for Search detail (ex. "person/show/{entity}"

Interface for repositories

Create an interface to get a queryBuilder for the searchables entities

interface PopulateSearchQueryBuilderInterface {
    public funtion getPopulateSearchQueryBuilder(): QueryBuilder;
}

Escape string for use in MySQL fulltext search

Is

Search for a query like s) returns an error response

Should

Should work nomally.

Acceptance Criteria

• search is possible with all chars.

Symfony\Bridge\Doctrine\RegistryInterface deprecated

Symfony\Bridge\Doctrine\RegistryInterface now has been deprecated and have to use Doctrine\Persistence\ManagerRegistry instead, can we have an update on this ?

Circular reference detected for service "doctrine.dbal.default_connection"

I've tried to install SearchBundle on symfony 3.3.6 (also a clear fresh install) and on 3.1.10 but got the same error:

[Symfony\Component\DependencyInjection\Exception\ServiceCircularReferenceException]
Circular reference detected for service "doctrine.dbal.default_connection", path: "doctrine.dbal.default_connection".

or

PHP Fatal error: Uncaught Symfony\Component\DependencyInjection\Exception\ServiceCircularReferenceException: Circular reference detected for service "doctrine.dbal.default_connection", path: "doctrine.dbal.default_connection". in /home/dido/Documents/work/sk_futurespace/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Dumper/PhpDumper.php:515
Stack trace:
#0 /home/dido/Documents/work/sk_futurespace/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Dumper/PhpDumper.php(653): Symfony\Component\DependencyInjection\Dumper\PhpDumper->addServiceInlinedDefinitionsSetup('doctrine.dbal.d...', Array, true)
#1 /home/dido/Documents/work/sk_futurespace/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Dumper/PhpDumper.php(678): Symfony\Component\DependencyInjection\Dumper\PhpDumper->addService('doctrine.dbal.d...', Object(Symfony\Component\DependencyInjection\Definition))
#2 /home/dido/Documents/work/sk_futurespace/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Dumper/PhpDump in /home/dido/Documents/work/sk_futurespace/vendor/symfony/symfony/src/Symfony/Component/DependencyInjection/Dumper/PhpDumper.php on line 515

And what I've done was just require, add bundle to AppKernel and on schema update got this error. It seems that You are injecting entity manager to IndexManager than injecting IndexManager into IndexListener which is a doctrine event listener. This might work on older versions of symfony (maybe?) but not on version 3.3.6 (nor 3.1.10). Could you please look into this issue and let me know if what am I doing wrong?

Check config whatwedo_search.entities is set in Configuration

Manager/IndexManager.php (Line 107)
if (isset($this->config['entities'])) { foreach ($this->config['entities'] as $entityConfig) {

noresults: minScore set to high

SearchBundlde is not working properly in some cases.

I suspect the persist problem if short words are searched.

Search for "pure", minScore is 6

SELECT w0_.content, w0_.foreign_id AS foreign_id_0, MATCH(w0_.content) AGAINST ('pure') AS sclr_1 FROM whatwedo_search_index w0_ 
WHERE MATCH(w0_.content) AGAINST ('pure') > :minScore AND w0_.content LIKE '%pure%' AND w0_.model = 'Sensopro\\Entity\\Trainer' GROUP BY w0_.foreign_id ORDER BY sclr_1 DESC;
:minScore =  6

Result with minScore = 0

content	foreign_id	sclr_1
'Pure 1 rechts LAN'	'2'	'4.997587203979492'
'Pure 2 links WLAN'	'3'	'4.997587203979492'
'purepurepure pure asdflaöd asdffiapweosd, afss assödkfaweirasssd fasdflaksdfasdf'	'13'	'4.997587203979492'
'VK18134, Boutique 98 by pure senses'	'166'	'4.997587203979492'

---

Search for "purepurepure", minScore is 18

SELECT w0_.content, w0_.foreign_id AS foreign_id_0, MATCH(w0_.content) AGAINST ('purepurepure') AS sclr_1 FROM whatwedo_search_index w0_ 
WHERE MATCH(w0_.content) AGAINST ('purepurepure') > :minScore AND w0_.content LIKE '%purepurepure%' AND w0_.model = 'Sensopro\\Entity\\Trainer' GROUP BY w0_.foreign_id ORDER BY sclr_1 DESC;
:minScore =  18

Result with minScore = 0

content	foreign_id	sclr_1
'purepurepure pure asdflaöd asdffiapweosd, afss assödkfaweirasssd fasdflaksdfasdf'	'13'	'8.051908493041992'

problem is located in the IndexRepository.php

->where("MATCH_AGAINST(i.content, :query) > :minScore")
->setParameter('minScore', round(strlen($query) * 1.5))

reducing the minScore factor solves the Problem, but can be still wrong,
in my opinion the minScore "where" should be removed.