indexDocument not handled when bucket contains > 1k objects

[oxyno-zeta]

indexDocument not handled when bucket contains > 1k objects

[oxyno-zeta]

This should be fixed with the 3.1.1 release 🎉

[oxyno-zeta]

Hello @chiefy !

Thanks for this issue and all informations you give me. You are right. I must admit that I didn't though that this project will be used for this kind of huge bucket 😄 .

I will work on a quick fix within next days (I have it in mind, need to be tested) and I probably found a solution to allow more results to be served by allowing people to configure the max results limit they want (to avoid looping over all elements and eating all memory !).

I will keep you posted.

Regards,

Oxyno-zeta

[oxyno-zeta]

This should be ok and now you can manage the pagination if needed. I'm closing this :)

[chiefy]

Looking some more into this, our bucket has 1.4k objects, and it looks like by default the s3 client will only return MaxKeys of 1k by default

[chiefy]

@oxyno-zeta thanks. As it turns out our CI is publishing stuff to s3 without cleaning the bucket beforehand. So I'd say it's an edge-case.

[oxyno-zeta]

Hello, Can you confirm or not that the issue is fixed ? Just to know I need to think about another solution / another fix 😅 . Thanks !