Make sure file is preserved on import failure

[the-paperless-project]

Make sure file is preserved on import failure

[danielquinn]

It looks like some of the more recent changes we made to the consumption process introduced this bug. I've now broken up the .cleanup() method into two parts, so now we can delete the temporary working dir upon failure, but only delete the source document upon success.

Thanks for catching this!

Hi Daniel,

It's not obvious from the code changes you committed how the file is represented in the database, if at all. Will the consumer get stuck in a retry loop?

Cheers,
Jason

[danielquinn]

Basically the logic for the loop is:

1. Look in the consumption directory
2. If there's a file, try to consume it
3. If it fails, add that file to the list of files we don't try to consume anymore.
4. GOTO 1.

The trick is that if you restart the consumer, it will indeed try to consume the file again because to do-not-consume list is in memory.