alfresco/searchservices

Alfresco Search Services

searchservices's Issues

Docker image build fails

Trying to build the docker image according to the steps in documentation fails.

Docker 'solrhome' volume limits image configuration

We are trying to create a new docker image extending from this and making some modifications to configuration:
RUN set -x \ && sed -i '/alfresco.cross.locale.datatype/s/^#//g' $DIST_DIR/solrhome/conf/shared.properties
but 'solrhome' is a volume so our changes to the file are not recorded into de image.

We could provide a volume with 'solrhome' folder structure replicated but, Is there a better way?
Is it actually useful to include this directory as volume?

target without solr / alfresco artefacts only

we still prefer installing Alfresxo Search Services the traditional way without docker and solr from the solr project.
Could you please provide in future releases:

• release notes (containing the solr version as a dependency)
• a build with the alfresco artefacts only / without solr itself

We want to install solr the solr way as recommended by the solr team. Your

Dependabot couldn't find a pom.xml for this project

Dependabot couldn't find a pom.xml for this project.

Dependabot requires a pom.xml to evaluate your project's current Java dependencies. It had expected to find one at the path: /insight-engine/pom.xml.

If this isn't a Java project, or if it is a library, you may wish to disable updates for it from within Dependabot.

You can mention @dependabot in the comments below to contact the Dependabot team.

'insight-engine' child module is no longer exist.

Since the insight-engine module does not exist in this repository, it is better to delete the setting from pom.xml.

SearchServices/pom.xml

Line 43 in 6c01e74

<module>insight-engine</module>

Solr documents of custom content properties are removed from index when node is reindexed

Since ASS 2.0.1.5 the indexation of custom content properties was reintroduced by setting the property solr.enableIndexingCustomContent to true.

On a node update in the alfresco repository the solr-documents for that node is reindexd. Those for the custom content properties are removed for one reindex cycle. This leads to a different behaviour than the reindexation of all other node properties ...

the search results for the custom content properties simply disappear for one reindex cycle. from an enduser perspective this leads to inconsistent searchresults and the impression of a defect in the default search behaviour

can be reproduced by using the following sdk sample project.
alfresco-sdk-sample-reindex-content-properties.zip

1. build sample project with maven and start acs with run.sh start
2. call sample webscript from your browser : http://localhost:8080/alfresco/s/sample/helloworld
   now a test node was created here "/app:company_home/cm:test/cm:test.txt". it has 1 custom content-property acme:metadataIndex. you can verify in your node-browser
3. use the nodebrowser so search for the node: aftsQuery: "acme:metadataIndex:Alexander" and you will find the node as expected
4. call the sample webscript again
   on an already existing node - the cm:name property will be changed to cm:xtest.txt leading to a reindex
5. use the nodebrowser so search for the node: aftsQuery: "acme:metadataIndex:Alexander" . after solr tracks the update the result will disappear.
6. keep searching ... after 8 sec. it will appear again

• reference to hyland support case: #00838836
• initial github issue - leading to reintroduction of custom content property indexation : #403
  seems to be related to : https://alfresco.atlassian.net/jira/software/c/projects/MNT/issues/MNT-23525
  and changes made here: c3dd096

JavaDoc for the CommitTracker class

SearchServices/alfresco-search/src/main/java/org/alfresco/solr/tracker/CommitTracker.java

Line 31 in eda6705

/**

Hi guys, firstly I want to say what a great job you guys are doing here ! 💪 . I was wondering if we can get some JavaDoc briefly (1-2 sentences) on what the Commit tracker's purpose is if that's possible? Thanks. 😄

mechanism to temporarily prevent text retrieval

the content tracker still needs to run synchronously. We need a mechanism to temporarily prevent text retrieval to avoid scalability issues and timeouts (caused by async transactions) - especially if we already know that they are long running like for ocr.

In the old, sync transformer framework it was possible to fake such a feature by setting cm:isContentIndexed=false to prevent the node to be catched up from the repository before it has been transformed and to remove that aspect later when the text transformation is available.

#395 / SEARCH-2974 breakes this old "feature". So either we get a new feature to postpone the text retrieval or the mechanism for isContentIndexed is working again as expected e.g.

• if a new node get's cm:isContentIndexed=false property added by behavior it must not result into an empty index doc
• when removing the aspect or setting isContentIndexed to true later the text should be indexed

Container UID/GID

Several of the containers used to spin up an Alfresco install have different UID and GID from each other. It is a best practice for all containers in a stack to run with a non-root uid/gid that is consistent across containers.

One reference: https://github.com/hexops/dockerfile#use-a-static-uid-and-gid

At my customer we actually have users/groups in the 33007 range so we will have to do re-mapping: https://docs.docker.com/engine/security/userns-remap/

It is very challenging to do re-mapping if different containers in the stack use different UID/GID.

I considered taking making a derivative container that updates the UID/GID. Unfortunately the VOLUME directive freezes part of the filesystem and it is impossible to re-id.

There is a hacky tool that can remove the VOLUME metadata so we can re-id, but that feels dangerous.

Our current approach is to re-build the containers starting from the published Dockerfiles, this is also not idea as our customer has a best practice of starting from vendor published images (I guess in a way we are because the base image is vendor supplied, but still this is yuck.)

I would suggest that all Alfresco containers use consistent UID/GID and probably that they don't us VOLUME if at all possible.

I spoke with Angel on Discord and he asked me to post this issue so he can think about this upon his return from vacation.

sorting text don't work

Hi everyone,

I want to use the solr search from a Front end application that comunicate with alfresco for the pagination and sorting purpose.

the sorting for the text metadata is not in the natural order:

• Adam
• Adem
• Test
• Adam 1
• Adam 23
• Test 1
• test

I don't understand what kind of order it is using. I think that the problem is caused by the presence of numbers and special characters.

there is a solr configuration to make the natural order?

CascadeTracker creates veeery big indexes when using secondary assocs on folders

Creating secondary parent/child associations for nodes is a fairly common use case. Alfresco itself uses this feature for workflow folders, for example.
Unfortunately, this does not play well with CascadeTracker. For example, when a folder is "linked" to the companyhome (as Alfresco does for workflows) by adding the companyhome as a secondary parent, CascadeTracker goes through the entire repo again to be added as a list of paths to the index document.
This results in a very large index size that may even exceed the size of the entire repo.

Secondary child associations worked fine in the old solr4 implementation until ASS introduced CascadeTracker. The only option we have today is to disable PATH tracking completely (alfresco.cascade.tracker.enabled=false) or remove all secondary child associations on folders/containers, which may brake use cases and functionality.

As a compromise, we should introduce something like alfresco.cascade.tracksecondaries.enabled to support PATH queries but avoid multiplying index size and tracking time and resources.

Enable date-related functions

See https://hub.alfresco.com/t5/alfresco-content-services-forum/limitations-search-and-insights-engine/td-p/293384

It would be great if it would be possible to report on dates and more specific on throughput time (so date-differences). How long was a case open? What was the throughput time of a certain workflow per user/department? ...

soft timeout for long running text extractions

as discusse on Discord the new transformer framework degrades scalability/stability due to more long-running threads.

The only work around by today is to increase timeouts for the http client but that will pile up the number of threads which is not a good idea. e.g.

solr.http.socket.timeout=30000
solr.http.connection.timeout=10000

To fix this, the tracker or repo web script should support a soft timeout that offloads the threads and triggers a mechanism as discussed in #396 to mark a node so that it is not captured by the content tracker and that automatically restores visibility to the tracker once the content has been transformed by a T-Engine.

ASS won't start with Java 11.0.9.1 version

After I upgraded my ubuntu 18.04 lts server (included java upgrade also) Jetty did not started with the new openJDK java

Java version

java -version
openjdk version "11.0.9.1" 2020-11-04
OpenJDK Runtime Environment (build 11.0.9.1+1-Ubuntu-0ubuntu1.18.04)
OpenJDK 64-Bit Server VM (build 11.0.9.1+1-Ubuntu-0ubuntu1.18.04, mixed mode, sharing)`

solr-8983-console.log

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Invalid Java version 11.0.9.1

Usage: java -jar start.jar [options] [properties] [configs]
java -jar start.jar --help  # for more information

Current Jetty libs

ls -al /opt/alfresco/alfresco-search-services/solr/server/lib
total 2212
drwxrwxr-x 3 solr solr   4096 Aug 17 16:30 .
drwxr-xr-x 9 solr solr   4096 Aug 17 16:30 ..
drwxrwxr-x 2 solr solr   4096 Aug 17 16:30 ext
-rw-r--r-- 1 solr solr  43067 May  9  2014 gmetric4j-1.0.7.jar
-rw-r--r-- 1 solr solr  95806 Apr 26  2013 javax.servlet-api-3.1.0.jar
-rw-r--r-- 1 solr solr  15968 Apr 18  2019 jetty-continuation-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr  51260 Apr 18  2019 jetty-deploy-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr 150666 Apr 18  2019 jetty-http-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr 122464 Apr 18  2019 jetty-io-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr  22185 Apr 18  2019 jetty-jmx-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr  34392 Apr 18  2019 jetty-rewrite-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr  95072 Apr 18  2019 jetty-security-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr 527987 Apr 18  2019 jetty-server-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr 118957 Apr 18  2019 jetty-servlet-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr  86557 Apr 18  2019 jetty-servlets-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr 462141 Apr 18  2019 jetty-util-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr 114996 Apr 18  2019 jetty-webapp-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr  51738 Apr 18  2019 jetty-xml-9.3.27.v20190418.jar
-rw-r--r-- 1 solr solr 129402 Mar 20  2017 metrics-core-3.2.2.jar
-rw-r--r-- 1 solr solr   9916 Mar 20  2017 metrics-ganglia-3.2.2.jar
-rw-r--r-- 1 solr solr  22056 Mar 20  2017 metrics-graphite-3.2.2.jar
-rw-r--r-- 1 solr solr  20053 Mar 20  2017 metrics-jetty9-3.2.2.jar
-rw-r--r-- 1 solr solr  40188 Mar 20  2017 metrics-jvm-3.2.2.jar`

jetty/jetty.project#2090

gregw commented on 4 Jan 2018
@IvoLimmen it is definitely a new bug. We now need to be a lot more java version sensitive because of java 9 features... but we did not anticipate 4 part version numbers. Fix is coming.

gregw commented on 4 Jan 2018
I've pushed a fix to jetty-9.4.x branch. The JavaVersion now handles both old format and full JEP223 versions. I have also shaded the class into the jetty-start module rather than duplicate the code.

I also tried upgrade AAS 1.4.1 to 1.4.3 but it does have the same error

Both solr6 and solr6secondary share the same port.

Solr6Secondary fails to start and throws the following error

ERROR: for sharding_solr6secondary_1 Cannot start service solr6secondary: driver failed programming external connectivity on endpoint sharding_solr6secondary_1

Refer to the below code

SearchServices/e2e-test/generator-alfresco-docker-compose/generators/app/templates/7.0/docker-compose-ee.yml

Line 178 in cb2a194

- 8083:8983 <% if (httpMode == 'https') { %>

Both solr6 and solr6secondary share the same external port 8083. These external ports should be different for solr6 and solr6Secondary

Cannot override ssl.keystore.type in solrcore.properties

The design of CoreDescriptorDecorator and ConfigUtil allow a some solrcore.properties be overriden by external source: JNDI, Java System properties and environment variables. However, when "alfresco.secureComms=https", these 2 solrcore.properties cannot be overriden:

• alfresco.encryption.ssl.keystore.type
• alfresco.encryption.ssl.truststore.type

The bug is in the CoreDescriptorDecorator constructor's foreach loop at line 94 when ConfigUtil.locateProperty() return null, it will lead to a NullPointerException in put() and terminates the loop.

If solrcore.properties created from "solrhome/templates/..." have these two settings commented out or omitted:

• #alfresco.encryption.ssl.keystore.provider=
• #alfresco.encryption.ssl.truststore.provider=

the foreach loop will terminate due to the NPE when it reaches property "alfresco.encryption.ssl.keystore.provider", skipping both ...ssl.keystore.type and ...ssl.truststore.type set externally.

Sharding seems not to work as documented in Alfresco 6.1 CE

I created 4 shards as documented in https://docs.alfresco.com/search-community/tasks/solr-hash-shard.html
In alfresco-global.properties I configured

solr6.store.mappings.value.solrMappingAlfresco.nodeString=solr.mycompany.com:8983/solr/#/alfresco
solr6.store.mappings.value.solrMappingAlfresco.numShards=4
solr6.store.mappings.value.solrMappingAlfresco.replicationFactor=1

getting a "Request failed 405" in Alfresco:

Caused by: org.alfresco.repo.search.impl.lucene.LuceneQueryParserException: 11080028 Request failed 405 /solr/#/alfresco-2/alfresco?wt=json&fl=DBID%2Cscore&rows=50&shards=solr.mycompany.com:8983%2Fsolr%2F%23%2Falfresco-0,solr.mycompany.com:8983%2Fsolr%2F%23%2Falfresco-1,solr.mycompany.com:8983%2Fsolr%2F%23%2Falfresco-2,solr.mycompany.com:8983%2Fsolr%2F%23%2Falfresco-3&df=TEXT&start=0&locale=*&alternativeDic=DEFAULT_DICTIONARY&sort=%40cm%3Amodified+desc&fq=%7B%21afts%7DAUTHORITY_FILTER_FROM_JSON&fq=%7B%21afts%7DTENANT_FILTER_FROM_JSON
        at org.alfresco.repo.search.impl.solr.AbstractSolrQueryHTTPClient.postQuery(AbstractSolrQueryHTTPClient.java:79)
        at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.postSolrQuery(SolrQueryHTTPClient.java:1116)
        at es.keensoft.repo.search.impl.solr.EnhancedSolrQueryHTTPClient.postSolrQuery(EnhancedSolrQueryHTTPClient.java:43)
        at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.executeQuery(SolrQueryHTTPClient.java:583)
        at org.alfresco.repo.search.impl.solr.SolrQueryLanguage.executeQuery(SolrQueryLanguage.java:52)
        at org.alfresco.repo.search.impl.solr.SolrSearchService.query(SolrSearchService.java:354)
        at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:84)
        at jdk.internal.reflect.GeneratedMethodAccessor789.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at org.alfresco.repo.management.subsystems.SubsystemProxyFactory$1.invoke(SubsystemProxyFactory.java:79)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
        at com.sun.proxy.$Proxy36.query(Unknown Source)
        at org.alfresco.repo.search.impl.SearchServiceSubSystemDelegator.query(SearchServiceSubSystemDelegator.java:133)
        at jdk.internal.reflect.GeneratedMethodAccessor789.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.base/java.lang.reflect.Method.invoke(Unknown Source)
        at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:343)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
        at net.sf.acegisecurity.intercept.method.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:80)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.alfresco.repo.security.permissions.impl.ExceptionTranslatorMethodInterceptor.invoke(ExceptionTranslatorMethodInterceptor.java:53)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.alfresco.repo.audit.AuditMethodInterceptor.invoke(AuditMethodInterceptor.java:166)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:294)
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
        at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
        at com.sun.proxy.$Proxy36.query(Unknown Source)
        at org.alfresco.repo.jscript.Search.queryResultMeta(Search.java:1016)
        ... 100 more

please see full problem descrition on Stack Overflow:
Creating Solr shards in Alfresco 6.1 CE

Contentstore Volume missing

I may be missing something, but for me, README.md instructions for building a Docker image don't work. Mostly due to the base image used.

However, I think you are missing a volume instruction

VOLUME /opt/alfresco-search-services/contentstore

along with associated mkdir in the Dockerfile.

Incorrect configuration

According to the below code the keystore is loaded from /usr/local/tomcat/shared/classes/alfresco/extension/keystore/keystore. I assume this Keystore is bundled along with the alfresco image

SearchServices/e2e-test/generator-alfresco-docker-compose/generators/app/templates/7.0/docker-compose-ee.yml

Line 23 in cb2a194

-Dencryption.keystore.location=/usr/local/tomcat/shared/classes/alfresco/extension/keystore/keystore

But actually, we are mounting the generated keystore files as below.

SearchServices/e2e-test/generator-alfresco-docker-compose/generators/app/templates/7.0/docker-compose-ee.yml

Line 69 in cb2a194

- ./keystores/alfresco:/usr/local/tomcat/alf_data/keystore <% } %>

Looks like there is a misconfiguration here

Search performance problems with Alfresco Search Services

We are trying to migrate an Alfresco CE system from 5.2 with solr4 to Alfresco 6.1 with Alfresco Search Services (we tried 1.3 and 1.4) but we are facing massive performance problems using Alfresco Search Services / Solr6: Searches running on a similar setup take 3-5 x longer (RAM/storage/CPU) having same solr cache config. The strange thing is that cpu/RAM/storage is more or less idle. So ASS doen't seem to scale with given resources.

Please check also Stack Overflow question Search performance problems with Alfresco Search Services

release/v2.0.x branch

Hello,

where can we find the release/v2.0.x branch? The master is deprecated and we should use the release/v2.0.x branch, according to the Readme on master. This branch seems not to exist.

Best Regards
Jens

Cross-locale configuration

That's not exactly an issue but a feature petition.

We've been dealing with this for a year. I don't know why cross-locale is not enabled by default, I imagine there will be reasons to do so, but please, provide at least an environment variable to activate it from docker configuration, so we don't have to override default Dockerfile, or provide additional volumes with modified shared.properties file.

Thanks

CMIS queries involving comparisons to empty strings fail

On Alfresco Search Services v2.0.7, the following CMIS query (tested via the CMIS 1.1 browser binding) fails with HTTP 500:
SELECT cmis:objectId FROM cmis:document WHERE cmis:name = '' AND CONTAINS('(TEXT:* OR NOT TEXT:*)')

The CONTAINS() at the end is effectively a no-op, but it prevents executing the query as TMDQ. With it removed, the query is performed directly from the database, and it succeeds:

SELECT cmis:objectId FROM cmis:document WHERE cmis:name = ''
This correctly returns object IDs for documents with an empty name.

The failing query produces this stack trace in the Search Services container log:

2023-05-10 13:11:40.306 ERROR (qtp455538610-17) [ x:alfresco] o.a.s.s.HttpSolrCall null:org.alfresco.repo.search.impl.querymodel.QueryModelException: 04100015 No query time mapping for property cmis:name, it should not be allowed in predicates
at org.alfresco.repo.search.impl.querymodel.impl.lucene.functions.LuceneLike.addComponent(LuceneLike.java:78)
at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneFunctionalConstraint.addComponent(LuceneFunctionalConstraint.java:63)
at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneConjunction.addComponent(LuceneConjunction.java:71)
at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneQuery.buildQuery(LuceneQuery.java:110)
at org.alfresco.solr.AlfrescoSolrDataModel.getCMISQuery(AlfrescoSolrDataModel.java:1570)
at org.alfresco.solr.query.CmisQParserPlugin$CmisQParser.parse(CmisQParserPlugin.java:191)
at org.apache.solr.search.QParser.getQuery(QParser.java:168)
at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:161)
at org.apache.solr.handler.component.AlfrescoSearchHandler.handleRequestBody(AlfrescoSearchHandler.java:348)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base/java.lang.Thread.run(Thread.java:833)

An IS NULL query works (but has a different issue, where it interacts wrongly with LEFT JOINs), and so does any comparison string other than the empty string.

Rewriting the empty string comparison as a CONTAINS() also works fine:
SELECT cmis:objectId FROM cmis:document WHERE CONTAINS('=cmis:name:""') AND CONTAINS('(TEXT:* OR NOT TEXT:*)')

Using an IN query with a list containing the empty string, a different stack trace is printed:
SELECT cmis:objectId FROM cmis:document WHERE cmis:name IN ('', 'test') AND CONTAINS('(TEXT:* OR NOT TEXT:*)')

2023-05-10 13:23:22.267 ERROR (qtp455538610-22) [ x:alfresco] o.a.s.s.HttpSolrCall null:java.lang.NullPointerException: Query must not be null
at java.base/java.util.Objects.requireNonNull(Objects.java:233)
at org.apache.lucene.search.BooleanClause.(BooleanClause.java:60)
at org.apache.lucene.search.BooleanQuery$Builder.add(BooleanQuery.java:138)
at org.alfresco.solr.query.Lucene4QueryParserAdaptor$Lucene4QueryParserExpressionAdaptor.addOptional(Lucene4QueryParserAdaptor.java:338)
at org.alfresco.solr.query.Lucene4QueryParserAdaptor$Lucene4QueryParserExpressionAdaptor.addOptional(Lucene4QueryParserAdaptor.java:310)
at org.alfresco.opencmis.mapping.BaseLuceneBuilder.buildLuceneIn(BaseLuceneBuilder.java:82)
at org.alfresco.opencmis.search.CmisFunctionEvaluationContext.buildLuceneIn(CmisFunctionEvaluationContext.java:326)
at org.alfresco.repo.search.impl.querymodel.impl.lucene.functions.LuceneIn.addComponent(LuceneIn.java:81)
at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneFunctionalConstraint.addComponent(LuceneFunctionalConstraint.java:63)
at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneConjunction.addComponent(LuceneConjunction.java:71)
at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneQuery.buildQuery(LuceneQuery.java:110)
at org.alfresco.solr.AlfrescoSolrDataModel.getCMISQuery(AlfrescoSolrDataModel.java:1570)
at org.alfresco.solr.query.CmisQParserPlugin$CmisQParser.parse(CmisQParserPlugin.java:191)
at org.apache.solr.search.QParser.getQuery(QParser.java:168)
at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:161)
at org.apache.solr.handler.component.AlfrescoSearchHandler.handleRequestBody(AlfrescoSearchHandler.java:348)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base/java.lang.Thread.run(Thread.java:833)

The issue is not unique to cmis:name, or the cmis: pseudo-namespace, but also occurs with properties from aspects:
SELECT A.cmis:objectId FROM cmis:document A JOIN cm:titled B ON A.cmis:objectId = B.cmis:objectId WHERE B.cm:title = '' AND CONTAINS(A, '(TEXT:* OR NOT TEXT:*)')

2023-05-10 13:27:43.042 ERROR (qtp455538610-16) [ x:alfresco] o.a.s.s.HttpSolrCall null:org.alfresco.repo.search.impl.querymodel.QueryModelException: 04100016 No query time mapping for property cm:title, it should not be allowed in predicates
at org.alfresco.repo.search.impl.querymodel.impl.lucene.functions.LuceneEquals.addComponent(LuceneEquals.java:71)
at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneFunctionalConstraint.addComponent(LuceneFunctionalConstraint.java:63)
at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneConjunction.addComponent(LuceneConjunction.java:71)
at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneQuery.buildQuery(LuceneQuery.java:110)
at org.alfresco.solr.AlfrescoSolrDataModel.getCMISQuery(AlfrescoSolrDataModel.java:1570)
at org.alfresco.solr.query.CmisQParserPlugin$CmisQParser.parse(CmisQParserPlugin.java:191)
at org.apache.solr.search.QParser.getQuery(QParser.java:168)
at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:161)
at org.apache.solr.handler.component.AlfrescoSearchHandler.handleRequestBody(AlfrescoSearchHandler.java:348)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:513)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.base/java.lang.Thread.run(Thread.java:833)

Official support of OpenSSL 1.0.2 series seems to be ended.

Hi.

According to the official website of OpenSSL, support of OpenSSL 1.0.2 series seems to be ended.

Reference: Release Strategy

SearchServices/search-services/packaging/src/docker/Dockerfile

Line 9 in e9b0760

ENV OPENSSL_VERSION 1.0.2k-8.el7

Custom ACS 7.2 ContentModel Properties of type "d:content" are not indexed with ASS 2.0.3

Since SearchServices 2.x custom ACS Content-Model Properties of type "d:content" are no longer indexed. This is a different behaviour from previous versions (e.g. ACS 6.2 & ASS 1.4.3) where all ContentModel Properties of type "d:content" of a Repository Node are Indexed by SearchServices. In 2.x only a node´s "cm:content" Property is indexed.

@aborroy my colleague @aitseitz already discussed this issue with you. for our project this is a major blocker for the upgrade to ACS 7.x. i would be grateful for any advice on a workaround or fix for this problem.

i uploaded a sample sdk 4.4 project to reproduce
sdk-sample-project.zip

steps to reproduce:

1. startup the sdk docker setup from the sample project (extract and run ./run.sh build_start)

2. attach a debugger and set a break-point in org/alfresco/repo/web/scripts/solr/NodeContentGet.class:77 to check the indexing of the nodes content
   

3. call the sample Webscript from your browser http://localhost:8080/alfresco/s/sample/helloworld

4. in your nodebrowser the following node is created
   

5. the debugger stops when the solr Metadata Tracker requests the Content
   

observed behaviour:

1. Only the Content for the "cm:content" property is requested by the MetadataTracker
2. The "acme:metadataIndex" property of the node is never indexed.

expected behaviour:

1. the Content for the "cm:content" property is requested by the MetadataTracker
2. the Content for the "acme:metadataIndex" property is requested by the MetadataTracker. This would be the behavoiur we see in previous versions of SearchServices (e.g. 1.4.3)

additional information:
From checking the versions of the Class org.alfresco.solr.SolrInformationServer the behaviour is represented by the following implementation:

SearchServices 2.0.3
https://github.com/Alfresco/SearchServices/blob/master/search-services/alfresco-search/src/main/java/org/alfresco/solr/SolrInformationServer.java#L2714

previous SearchServices Version

SearchServices/search-services/alfresco-search/src/main/java/org/alfresco/solr/SolrInformationServer.java

Line 2443 in 6368156

private void addContentToDoc(SolrInputDocument doc, long dbId) throws AuthenticationException, IOException

'org.mockito.runners.MockitoJUnitRunner' is deprecated

org.mockito.runners.MockitoJUnitRunner is deprecated.

Class MockitoJUnitRunner

SearchServices/search-services/alfresco-solrclient-lib/src/test/java/org/alfresco/solr/client/SOLRAPIClientFactoryTest.java

Line 44 in 9c99b10

import org.mockito.runners.MockitoJUnitRunner;

Therefore, it is better to replace it with org.mockito.junit.MockitoJUnitRunner.

Fail to install serch-services

Hi.

II tried installing the Search-services project.However, it has failed.
The situation is as follows.

Subject

An error occurs due to lack of dependencies.

Performed steps

1. Clone project using git command below.

   $ git clone [email protected]:Alfresco/SearchServices.git
   

2. Install using mvn command below.

   $ cd SearchServices/search-services
   $ mvn clean install
   

Expected

Build suceeded.

Actual

Error occured with error messages below.

[INFO] --------------------< org.alfresco:alfresco-search >--------------------
[INFO] Building Alfresco Solr Search 1.5.0-SNAPSHOT                       [3/4]
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The POM for org.apache.lucene:lucene-analyzers-common:jar:6.6.5-patched is missing, no dependency information available
[WARNING] The POM for org.apache.solr:solr-core:jar:6.6.5-patched is missing, no dependency information available
[WARNING] The POM for org.apache.solr:solr-analysis-extras:jar:6.6.5-patched is missing, no dependency information available
[WARNING] The POM for org.apache.solr:solr-langid:jar:6.6.5-patched is missing, no dependency information available
[WARNING] The POM for org.apache.solr:solr-clustering:jar:6.6.5-patched is missing, no dependency information available
[WARNING] The POM for org.apache.solr:solr-test-framework:jar:6.6.5-patched is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Alfresco Solr Search parent 1.5.0-SNAPSHOT:
[INFO] 
[INFO] Alfresco Solr Search parent ........................ SUCCESS [  0.255 s]
[INFO] Alfresco Solr Client ............................... SUCCESS [  4.462 s]
[INFO] Alfresco Solr Search ............................... FAILURE [  0.108 s]
[INFO] Alfresco Search Services ........................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.936 s
[INFO] Finished at: 2019-10-14T16:53:31+09:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project alfresco-search: Could not resolve dependencies for project org.alfresco:alfresco-search:jar:1.5.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.lucene:lucene-analyzers-common:jar:6.6.5-patched, org.apache.solr:solr-core:jar:6.6.5-patched, org.apache.solr:solr-analysis-extras:jar:6.6.5-patched, org.apache.solr:solr-langid:jar:6.6.5-patched, org.apache.solr:solr-clustering:jar:6.6.5-patched, org.apache.solr:solr-test-framework:jar:6.6.5-patched: Failure to find org.apache.lucene:lucene-analyzers-common:jar:6.6.5-patched in https://artifacts.alfresco.com/nexus/content/groups/public was cached in the local repository, resolution will not be reattempted until the update interval of alfresco-public has elapsed or updates are forced -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :alfresco-search

Environment Info

$ java -version
openjdk version "13" 2019-09-17
OpenJDK Runtime Environment (build 13+33)
OpenJDK 64-Bit Server VM (build 13+33, mixed mode, sharing)
$ mvn -v
Apache Maven 3.6.2 (40f52333136460af0dc0d7232c0dc0bcf0d9e117; 2019-08-28T00:06:16+09:00)
Maven home: /usr/local/Cellar/maven/3.6.2/libexec
Java version: 13, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk-13.jdk/Contents/Home
Default locale: ja_JP, platform encoding: UTF-8
OS name: "mac os x", version: "10.15", arch: "x86_64", family: "mac"

Other Info

In Alfresco Nexus Repository, There seems to be no "org.apache.lucene:lucene-analyzers-common: jar:6.6.5-patched" in the repository.

Deprecated constructor is used.

KeyStoreParameters(String name, String type, String keyStoreProvider, String keyMetaDataFileLocation, String location) is deprecated.

SearchServices/search-services/alfresco-solrclient-lib/src/test/java/org/alfresco/solr/client/SOLRAPIClientTest.java

Line 257 in 9c99b10

KeyStoreParameters keyStoreParameters = new KeyStoreParameters("SSL Key Store", "JCEKS", null, "ssl-keystore-passwords.properties", "ssl.repo.client.keystore");

SearchServices/search-services/alfresco-solrclient-lib/src/test/java/org/alfresco/solr/client/SOLRAPIClientTest.java

Line 258 in 9c99b10

KeyStoreParameters trustStoreParameters = new KeyStoreParameters("SSL Trust Store", "JCEKS", null, "ssl-truststore-passwords.properties", "ssl.repo.client.truststore");

So it would be nice to use KeyStoreParameters(String name, String type, String keyStoreProvider, String keyMetaDataFileLocation, String location).

Reference:
https://github.com/Alfresco/alfresco-core/blob/bbe800b087cbf5b2d3e7f41e0c196e3b3f5d422f/src/main/java/org/alfresco/encryption/KeyStoreParameters.java#L73

CMIS query containing IS NULL predicate and LEFT JOIN inconsistent between Solr and TMDQ

The CMIS query
SELECT B.cm:owner FROM cmis:document A LEFT JOIN cm:ownable B ON A.cmis:objectId = B.cmis:objectId WHERE B.cm:owner IS NULL AND CONTAINS(A, '(TEXT:* OR NOT TEXT:*)')
fails to find documents without the cm:ownable aspect. The same query without the magic CONTAINS() to disable TMDQ works:
SELECT B.cm:owner FROM cmis:document A LEFT JOIN cm:ownable B ON A.cmis:objectId = B.cmis:objectId WHERE B.cm:owner IS NULL

Groups Sync API: Requesting only updated groups

For syncing groups to other platforms, we need new or deleted groups.
Also the updated groups, group is considered updated if there are any changes in its members.

Please have a look on this question:
https://stackoverflow.com/questions/59261575/alfresco-sync-groups-apis-and-cmis

Thank you

When T-Engine throws an Exception, the document gets indexed with no content

this is a reference ticket for SEARCH-2974 due to not beeing able to comment on alfresco.atlassian.net

details provided by @binduwavell

What we discovered is if the T-Engine throws, the node gets indexed with no content and the only way to update the content cache is to update the content on the node or to > PURGE/INDEX the node so the transform runs again.

We extended the NodeContentGet webscript and if the transformer fails we throw in this webscript, then Solr marks the node as having a content transform error. Once we've > grabbed our OCR text it is written as a rendition, which moves the node into a new transaction and Solr re-indexes the node automatically and re-attempts content transform....

I think that if a T-Engine fails, Solr should probably not cache empty content... I don't think Solr necessarily needs to retry getting the content, but the next time the node > is indexed, Solr should not trust the empty content cache and should re-attempt NodeContentGet.

This leads to an unreliable index because technical errors inevitably lead to incomplete index. It is not possible to fix these errors afterwards.
Writing zero content to the index if the node content is not empty should be always seen as wrong index content and therefore as a bug.

Please fix/remove broken hyperlink url 'insight-engine'

in the master project there is a broken hyperlink url 'insight-engine' , the hyperlink url is : https://github.com/Alfresco/SearchServices/blob/master/insight-engine

this return 404 Not found when click. Please fix/remove broken hyperlink url 'insight-engine'

====found in https://github.com/Alfresco/SearchServices/blob/master/ in below paragraph ===

Insight Engine
Following resources will not be available for Community users

More details are available at insight-engine folder.

===

continuous process increase index inconsistency

Hi,
we see that if a processing creates transactions in a continuous and frequent way, this has side effect to prevent Solr from updating these indexes continuously.
In fact, after investigation, we see that the MetadataTracker is executed every 10s (default cron) but exit without indexing documents related to the transactions it has found. These transactions will be indexed when the traffic will "decrease" on next iterations
After research, this is due to this line of code.

MetatDataTracker / line 979
        if (transactions.getTransactions()
          .stream()
          .anyMatch(transaction -> (transaction.getCommitTimeMs() > this.state.getTimeToStopIndexing()))) {
          getWriteLock().release();
          break;
        } 

We can see that if a transaction has a date greater than the date at getTimeToStopIndexing(), then it exits without indexing any transactions that might be.
Why not simply remove these transactions from the list in order to keep those that can be indexed and let executing the tracker indexation process ?

Regards,
Thibaut

alfresco-solrclient-lib does not compile

When running mvn install with JDK 1.8, all search services projects compile with the exception of alfresco-solrclient-lib. The alfresco-solrclient-lib only seems to compile with JDK 11 when it should work with JDK 8 as well.

Replication no longer working

The changes made in acf67f0 are not reflectd in the config dockr bash script search-services/packaging/src/docker/search_config_setup.sh.

findStringMaster='<requestHandler name="\/replication" class="org\.alfresco\.solr\.handler\.AlfrescoReplicationHandler">'
is no longer found inside the solrconfig.xml files since the class was change in the mentioned commit.

strange behaviour when using searchservices v2.0.0

hi,

we are getting a strange behaviour when trying to reindex a not so big repository with acs 6.2.0ga:

• ACS 6.2.0GA generated with latest alfresco-docker-installer (uses searchservices 2.0.0)
• about 1.5 mil docs, 400GB size
• indexsize < 10GB

the reindex process is running through in about 3 hours. so far so good.

• during indexing: high cpu/io usage (as expected)
• after indexer is done: cpu/io usage drops perfectly fine to near zero (test system, almost no users active).
• searching old/docs new docs: working fine.

BUT: after a redeployment of the containers (docker-compose down, docker-compose up --build --force-recreate -d, data volumes are external and exposed to the containers with bind-mounts) we have the usual cpu spike during startup of the system, and then the alfresco and solr containers are showing permanent cpu usage of about combined 75% - 100%. postgresql cpu usage also permanently high, and this never stops.
searching old docs/new docs still working fine though.

the solr weboverview shows both indexes as current (green checkmark).
the ootb-admin-console shows in the view "solr tracking status", that indexing is "active" for both indexes???

because there are no users active, and indexing should be done, i can't explain what acs/search services is trying to do (and obviously failing to do here, it seems stuck somewhere)?

when changing searchservices back to v1.4.3 i don't get this behaviour by the way:

• reindex runs through
• cpu/io staying stable low even after redeployment of the stack.

any idea how i could find out/debug the behaviour v2.0.0 is showing?

Generator Alfresco Docker Compose

Hi,

i can't install generator-alfresco-docker-compose and run yo alfresco-docker-compose.

I have yo successfully installed, and nodejs as well.

I try to install without using the npm link and it does not find.

What is the best way forward?

Thanks

Daylight saving time in logs

Daylight saving time in the Solr logs is not taken into account.

Configuration

SOLR_OPTS contains -Duser.timezone=Europe/Berlin

Expected Result

(inside solr.log)
2022-10-28 13:00:00.000 ...

Current result

2022-10-28 12:00:00.000 ...

Affected version

Search services 2.0.3

Issue with maven build on Mac OS Catalina

When we build the Alfresco Search Services on Mac OS Catalina the final solr6 zip file in target is always corrupted.
Had to comment the following lines in the POM file

<filtering>true</filtering>

This filtering seems to corrupt the jar file and also the final zip file which is created. So need to check if this issue is specific to Mac

workflow-search

I am reaching out to you as an Alfresco user who created a workflow approximately one year ago. Currently, I am attempting to locate this workflow within the system but am facing challenges.
I would appreciate your assistance in locating this specific workflow and retrieving detailed information about it. Could you guide me on the best way to search for this workflow in Alfresco or provide any necessary assistance to achieve this?

Invalid yourkit URL in the Dockerfile

In the Dockerfile there is an invalid URL: https://download.yourkit.com/yjp/2019.8/YourKit-JavaProfiler-2019.8-b142-docker.zip
Instead it should be (the latest): https://download.yourkit.com/yjp/2019.8/YourKit-JavaProfiler-2019.8-b142-docker.zip
OR (the same version but with the valid URL): https://archive.yourkit.com/yjp/2019.8/YourKit-JavaProfiler-2019.8-b142-docker.zip
Lines 18 and 64 should be modified.

It might be the best to host it on the Alfresco's nexus server (https://nexus.alfresco.com/)

GC / OOM problems with Alfresco Search Services and OpenJDK11

I updated Alfresco Search Services from 1.3 to 1.4 which forced me to also update OpenJDK from 8 to 11. Running Alfresco Search Services 1.3 with jdk8 worked without any OutOfMemoryExceptions during (re)indexing but with jdk11 we see repeatable growing the heap until the solr oom killer kills the solr process. During indexing the jvm performs continuously GC with jdk8 but I guess jdk11 changed GC in a way that the objects will stay longer in memory. Continuous GC indicates inefficient object creatrion but this is nothing I can influence. I tried with UseConcMarkSweepGC and G1 garbage collector but with the same behavior. What is the recommended way to configure GC with OpenJDK 11 to behave similar as in OpenJDK8 with Alfresco Search Services / solr6?

My parameters in solr.in.sh

SOLR_JAVA_MEM="-Xms16g -Xmx30g"
SOLR_OPTS="$SOLR_OPTS -Dsolr.jetty.request.header.size=1000000 -Dsolr.jetty.threads.stop.timeout=300000 -Ddisable.configEdit=true -Dsolr.allow.unsafe.resourceloading=true"
SOLR_OPTS="$SOLR_OPTS -XX:+UseConcMarkSweepGC -XX:-DisableExplicitGC -XX:-UseGCOverheadLimit"
SOLR_OPTS="$SOLR_OPTS -server -Djava.net.preferIPv4Stack=true -Duser.language=en -Duser.country=US -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.net.preferIPv6Addresses=false"
SOLR_OPTS="$SOLR_OPTS -Dsun.security.ssl.allowUnsafeRenegotiation=true -Dsolr.allow.unsafe.resourceloading=true"

please also check Question on Stack Overview:
GC / OOM problems with Alfresco Search Services and OpenJDK11

SolrException: Invalid Number when searching on d:int property with value exceeding Integer.MAX_VALUE

Steps to reproduce

• perform a search on a property of type d:int, with query value exceeding the Integer.MAX_VALUE value.
• same for properties of type d:long when searching with value exceeding Long.MAX_VALUE

Context
To fix MNT-19252, following code has been introduced in the Solr4QueryParser to check if the queryText is a valid numeric:

            isNumeric = (schemaField.getType().getNumericType() != null);
            if (isNumeric)
            {
                //Check to see if queryText is numeric or else it will fail.
                try
                {
                    Double.valueOf(queryText);
                }
                catch (NumberFormatException e)
                {
                    return new TermQuery(new Term("_dummy_", "_miss_"));
                }
            }

However in the edge case where queryText is a number, but exceeds e.g. Integer.MAX_VALUE, Double.valueOf(queryText); will not throw a NumberFormatException and later a SolrException: Invalid Number exception will be thrown.

Stacktrace
Reproducable by searching for sys:node\-dbid: 123456789123456789123456789 in the Alfresco admin panel.
This results in following stacktrace in SearchServices:

2020-06-23 15:11:33.435 ERROR (qtp440938038-20) [   x:alfresco] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Invalid Number: 123456789123456789123456789
	at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:455)
	at org.apache.solr.schema.TrieField.readableToIndexed(TrieField.java:427)
	at org.apache.solr.schema.TrieField.toInternal(TrieField.java:462)
	at org.apache.solr.schema.FieldType$DefaultAnalyzer$1.incrementToken(FieldType.java:508)
	at org.alfresco.solr.query.Solr4QueryParser.getFieldQueryImpl(Solr4QueryParser.java:1621)
	at org.alfresco.solr.query.Solr4QueryParser.getFieldQueryImplWithIOExceptionWrapped(Solr4QueryParser.java:1129)
	at org.alfresco.solr.query.Solr4QueryParser.getSuperFieldQuery(Solr4QueryParser.java:3830)
	at org.alfresco.solr.query.Solr4QueryParser$FieldQuery.getQuery(Solr4QueryParser.java:3936)
	at org.alfresco.solr.query.Solr4QueryParser.attributeQueryBuilder(Solr4QueryParser.java:4392)
	at org.alfresco.solr.query.Solr4QueryParser.getFieldQuery(Solr4QueryParser.java:631)
	at org.alfresco.solr.query.Lucene4QueryParserAdaptor.getFieldQuery(Lucene4QueryParserAdaptor.java:67)
	at org.alfresco.solr.query.Lucene4QueryParserAdaptor.getFieldQuery(Lucene4QueryParserAdaptor.java:48)
	at org.alfresco.repo.search.impl.querymodel.impl.lucene.functions.LuceneFTSTerm.addComponent(LuceneFTSTerm.java:75)
	at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneFunctionalConstraint.addComponent(LuceneFunctionalConstraint.java:63)
	at org.alfresco.repo.search.impl.querymodel.impl.lucene.LuceneQuery.buildQuery(LuceneQuery.java:110)
	at org.alfresco.solr.AlfrescoSolrDataModel.getFTSQuery(AlfrescoSolrDataModel.java:1475)
	at org.alfresco.solr.query.AlfrescoFTSQParserPlugin$AlfrescoFTSQParser.parse(AlfrescoFTSQParserPlugin.java:105)
	at org.apache.solr.search.QParser.getQuery(QParser.java:168)
	at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:161)
	at org.apache.solr.handler.component.AlfrescoSearchHandler.handleRequestBody(AlfrescoSearchHandler.java:323)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2477)
	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:723)
	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:529)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
	at org.eclipse.jetty.server.Server.handle(Server.java:534)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
	at java.base/java.lang.Thread.run(Thread.java:834)