flowpack/flowpack.elasticsearch

Elasticsearch support for Flow / Neos

flowpack.elasticsearch's Issues

Optional parameter declared before required parameter

More information here:
https://php.watch/versions/8.0/deprecate-required-param-after-optional

This happens to our code base as well, but a change of the signature will also be breaking.

Deprecated: Optional parameter $id declared before required parameter $response is implicitly treated as a required parameter in /var/www/html/Packages/Application/Flowpack.ElasticSearch/Classes/Domain/Factory/DocumentFactory.php on line 40

Indexing not possible when peer cannot be verified

Unfortunately it is not possible to index the nodes if the ElasticSearch instance is running on a self signed certificate and only accessible via https.

This is due to the inability to configure the curl option for ssl verification.

Fix realtimeIndexing setting

With the fix for #19 another problem came up, see discussion on 053e95c

So the setting should become "self-contained" again.

remove object from index removes complete index

when you have a model with Custom Id Property and not standard-name "persistence_object_identifier", the aspect on the delete method trys to find the Id of the object which was removed. this fails because the object is already deleted so it returns NULL and the call to elastic-search is DELETE/[index]/[type]/

I could track the issue down to "ObjectIndexer.removeObject"

$id = $this->persistenceManager->getIdentifierByObject($object);

Here the Persistence Manager returns NULL

If I change the Aspect to run before, everything works fine:

• @flow\Before("setting(TYPO3.TYPO3CR.Search.realtimeIndexing.enabled) && within(TYPO3\Flow\Persistence\PersistenceManagerInterface) && method(public .+->(remove)())")

So we have different possible solutions for this:

1. Change the Aspect
2. Change how the PersistenceManager->getIdentifierByObject works
3. do something else ;-)

Can somebody support me for this case please?

'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2

Hello there

Just noticed while checking php update compatibility of our packages that PHPCompatibility spits out some errors regarding the use of soft reserved keyword object which seems to be prohibited from being used in namespaces. Ref. https://www.php.net/manual/en/reserved.other-reserved-words.php

FILE: Flowpack.ElasticSearch/Tests/Functional/Indexer/Object/IndexInformerTest.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Tests/Functional/Indexer/Object/ObjectIndexerTest.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/Signal/SignalEmitter.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/Signal/Doctrine/EmitterAdapter.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/Signal/EmitterAdapterInterface.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/ObjectIndexer.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 4 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/Transform/StringCastTransformer.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/Transform/TextCastTransformer.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/Transform/DateTransformer.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/Transform/TransformerInterface.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/Transform/CollectionStringCastTransformer.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/Transform/TransformerFactory.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/Transform/ObjectIdentifierTransformer.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 2 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------


FILE: Flowpack.ElasticSearch/Classes/Indexer/Object/IndexInformer.php
-----------------------------------------------------------------------------------------------------------------------------------------
FOUND 1 ERROR AFFECTING 1 LINE
-----------------------------------------------------------------------------------------------------------------------------------------
 4 | ERROR | 'object' is a soft reserved keyword as of PHP version 7.0 and a reserved keyword as of PHP version 7.2 and should not be
   |       | used to name a class, interface or trait or as part of a namespace (T_NAMESPACE)
-----------------------------------------------------------------------------------------------------------------------------------------

Cheers

Allow mapping annotations at class level

It would be cool to be able to define mapping at the class level.
This way, we could address the mapping for the _all field.

/**
 * @Flow\Entity
 * @ElasticSearch\Indexable("acme", typeName="organization")
 * @ElasticSearch\Mapping(index_name="_all", analyzer="standard_french")
 */
class Organization {

}

Enable fielddata in mapping

In order to enable fielddata in the mapping-annotations the class Flowpack\ElasticSearch\Annotations\Mapping should have a property 'fielddata' (boolean).

Call to a member function process() on null (since v5.1.0)

Release 5.1.0 seems to break things for us.

Call to a member function process() on null

  Type: Error
  File: Data/Temporary/Production/Cache/Code/Flow_Object_Classes/Flowpack_ElasticSe
        arch_Domain_Model_Index.php
  Line: 209

Seems DynamicIndexSettingService is not yet available during injectSettings()?

Non-breaking but IMHO the default value for CURLOPT_SSL_VERIFYHOST should be set to 2. curl_setopt_array(): CURLOPT_SSL_VERIFYHOST no longer accepts the value 1, value 2 will be used instead (see https://www.php.net/manual/en/function.curl-setopt.php)

Disable realtimeIndexing

I thought, if i disable realtimeIndexing in Flowpack.ElasticSearch nothing should be indexed while publishing a change.
But i get an Error while publishing a content-change: Elastic-Server can't be reached.

How do i disable ElasticSearch Indexing on my local environment without a ES-Server?

Flowpack:
  ElasticSearch:
    realtimeIndexing:
      enabled: false

Im using Flowpack.ElasticSearch.ContentRepositoryAdaptor but i didn't found any option there.

Neos: 5.3
"flowpack/elasticsearch-contentrepositoryadaptor": "^7",

realtimeIndexing.enabled should be taken from TYPO3CR.Search settings

Right now there is Flowpack.ElasticSearch.realtimeIndexing.enabled but the same setting is also present in the TYPO3.TYPO3CR.Search settings. To avoid confusion, the setting from TYPO3CR.Search should be used.

Setting port via environment variable impossible

When using environment variables in settings to define the port number, the result is always a string. The type hint on setPort() blocks this.

Set fullMappings

Is it correct, that there's currently no possibility to specify the fullMapping property of any Flowpack\ElasticSearch\Domain\Model\Mapping object?

For my current project I need to configure the _all field. That's exactly what the fullMapping property is there for.

In my opinion it would be best to have the option to define the fullMapping as a settings array inside Settings.yaml like this:

Flowpack:
  ElasticSearch:
   indexes:
     [...]
    mapping:
      'person':
        fullMapping:
          _all:
            analyzer: 'nGram_analyzer'
            search_analyzer: 'whitespace_analyzer'

What do you think?

Issue with 5.1.0: CURLOPT_SSL_VERIFYHOST no longer accepts the value 1, value 2 will be used instead

The yaml settings value defaults to true, however the actual default value seems to be 2 according to

https://curl.se/libcurl/c/CURLOPT_SSL_VERIFYHOST.html

The resulting exception when executing the nodeindex:build command from the package Flowpack.ElasticSearch.ContentRepositoryAdaptor:

Setting the yaml value to 2 solves this issue for me.

As this might be important, I am working with DDEV on WSL2 Windows.

Indexing type='object'

In order to index objects, the mapping-annotations Flowpack\ElasticSearch\Annotations\Mapping should have a property 'properties' (array).

Indexing huge amounts of objectes eats up huge amount of memory

If you Index huge amounts ob objects, the indexer eats up more and more memory. I tracked it down to ObjectIndexer.indexObject.
The ObjectIndexer tracks the objects which are already indexed in handledObjects, which increases dramatically when indexing several 100.000 of objects.