Dead letter handling with multiprocQueueBase implementation

[mistrey]

Dead letter handling with multiprocQueueBase implementation

[kquick]

Is this something you can provide a minimal sample that reproduces the issue (or a modification of thespian/tests/test_deadLettering.py)?

[mistrey]

After replacing multiprocUDPBase by multiprocQueueBase in your tests/test_deadLettering.py, I got 7 failures:

=================================================== short test summary info ===================================================
FAILED test_deadLettering.py::TestFuncDeadLettering::test02_GetDeadLetter[multiprocUDPBase] - assert 0 > 0
FAILED test_deadLettering.py::TestFuncDeadLettering::test03_DLRegisterOnlyOnce[multiprocUDPBase] - assert 0 > 0
FAILED test_deadLettering.py::TestFuncDeadLettering::test13_DLRegisterOnlyOnce[multiprocUDPBase] - assert 0 > 0
FAILED test_deadLettering.py::TestFuncDeadLettering::test04_DLMultipleHandlers[multiprocUDPBase] - assert 0 > 0
FAILED test_deadLettering.py::TestFuncDeadLettering::test14_DLMultipleHandlers[multiprocUDPBase] - assert 0 > 0
FAILED test_deadLettering.py::TestFuncDeadLettering::test05_DLAutoRemoval[multiprocUDPBase] - assert 0 > 0
FAILED test_deadLettering.py::TestFuncDeadLettering::test15_DLAutoRemoval[multiprocUDPBase] - assert 0 > 0
============================= 7 failed, 2079 passed, 254 skipped, 3 xfailed in 2070.49s (0:34:30) =============================

[kquick]

The lines where I assume you made your replacement are unstable_test(..., 'multiprocUDPBase') lines. Those actually mark the current test as unstable for the indicated base and therefore skipped unless overridden by config (e.g. $ pytest -o unstable=True ...). The effect of your changes was that you enabled the multiprocUDPBase tests, which are known to be unstable and resulted in the 7 failures you indicated above (this base is unstable for these tests because a UDP connection neither guarantees delivery nor does it make an active connection that would be reported as broken or which would report delivery failures if the remote node was no longer present, thus a multiprocUDPBase is not adviseable for this type of operation).

By default and without the changes you made, the multiprocQueueBase is checked for all tests in test_deadLettering.py. You can explicitly confirm this and limit the tests to that base via $ pytest -v -k Queue thespian/test/test_deadLettering.py. Hopefully one of those tests is close to the configuration where you are encountering the dead letter routing problems and you can modify it to demonstrate the behavior you are seeing.