are this times normal?

[pedroorellana]

are this times normal?

[anttttti]

3-4 seconds for 300 words sounds too high.

Your script could be loading or copying in memory large objects with each call, which would cause this. If you use multiprocessing as the backend, each subprocess will copy all of the global variables, and variables referred in the passed data. If per-document transforms with "serial" as the method are fast, then this subprocess memory copying is likely the issue. If you want to use multiprocessing, then make sure your global variables don't include big objects, and you don't pass references to them.

[pedroorellana]

thanks a lot!
At the end I just use the multiprocessing batch call to all the tf-idf transform that I need to do and the time for all the thousands of transformation is the same !!( so practically all the time is loading my model to the memory).

Thanks again for your amazing tool, greetings from Chile.