After disable "indexing in frontend" (indexed_search) nothing will be indexed by crawler anymore

[mediaessenz]

After disable "indexing in frontend" (indexed_search) nothing will be indexed by crawler anymore

[tomasnorre]

I don't recall this from the top of my head, but I know there is a "hook" the other way around that when a page is flushed, it's automatically added to the crawler queue again.

But it sounds like it could be the issue as pages are already cached. I would have to think of a solution for this.

[github-actions]

Hi there, thank you for taking your time to create your first issue. Please give us a bit of time to review it.

[mediaessenz]

After some more research I found out, that the problem only exists, if I add multiple crawler configurations (Argument "conf") to a crawler:buildQueue task.

In my case I added three configurations by a comma separated list of its names (pages,news,press). One for the normal pages, one for news (tx_news) and one for press messages (also tx_news).

If I only enter the configuration for the normal pages, the indexing works as expected, also with disabled frontend indexing.

Having realised this I created extra tasks for news and press indexing, but it seems, that they are not getting indexed with disabled frontend indexing.

The crawler config for news looks like this:
name:
news

pidsonly:
880

configuration
&tx_news_pi1[controller]=News&tx_news_pi1[action]=detail&tx_news_pi1[news]=[_TABLE:tx_news_domain_model_news; _PID:879; _WHERE: hidden = 0]

pid 880 is the news detail page. pid 879 is the sysfolder with the news records.

In the corresponding crawler:buildQueue task, I entered the detail pid 880 into the Argument "page" field

I will dig a bit deeper now and hopefully find the cause of the problem ...

[tomasnorre]

Thanks for your report. We will look into this. The response time, I currently longer than normally, I will get back to you though.

[nhovratov]

Hey, I had a little crawler configuration session today and had the same constellation with disableFrontendIndexing enabled. This flag is really useful, as we have cached pagination pages, which should not be indexed when the user browses through them. The problem however is, that as soon as any page is cached, crawling them will not trigger re-indexing. I don't know if there is already a solution for this, but theoretically the page cache must be flushed before each crawl to the page. Correct me if I'm wrong.

So maybe the problem described here is also about having already cached pages?