Out of Memory issues with frequent requests

[conor-nsurely]

Out of Memory issues with frequent requests

[gen2brain]

Close() should close and drop the document, context, and stream, i.e. it should clean all the memory. How large are PDFs? I am guessing that it is related to the context created with FZ_STORE_UNLIMITED hard-coded. You didn't show all your code, but it looks like you are doing more things, extracting text, etc. I know for example ImageMagick can use a LOT of memory when dealing with PDF and can be controlled with the -limit option, but here it is hard-coded.

You can try to change that to FZ_STORE_DEFAULT (256 << 20), or just set some value. I should probably allow optional parameters and fallback to the default value, unlimited is not a good idea.

[conor-nsurely]

@gen2brain
Hey,
Thanks for getting back to me.

The pdfs are about 2-3 MiB on average, largest being 7-8MiB.
The pdf pages are converted to images and then sent to the Google vision api to perform OCR.

Once that is set will it clean up the old memory once it reaches the limit or what happens?

So in order to change FZ_STORE_UNLIMITED I would have to build the package myself right?

[gen2brain]

I have added the MaxStore global variable, so you can set the size you need, default is 256 << 20.
You can check the header files for docs, i.e. https://github.com/gen2brain/go-fitz/blob/master/include/mupdf/fitz/context.h#L306.

[conor-nsurely]

Hey @gen2brain thanks for fixing this so quickly.

Do you know when this might be released?

[gen2brain]

When the new mupdf libraries are built, probably will not happen during summer, just use @latest for now.

[conor-nsurely]

Okay will do.

Thanks