resonatehq/durable-promise-test-harness

This test harness is a verification system that checks implementations for conformance to the Durable Promise Specification.

durable-promise-test-harness's Issues

add formatting validation

a user passes in an address (in the future a docker image). we should do more validation on those values and all inputs really.

add grpc support

currently we only test http implementation.

add unit tests

need to test the tests.

add handling for 500

the checker does not handle 500 status code well. In the short term, we probably want to just ignore 500 errors and pretend like they did not happen in the checker. we need a more long term plan for this though.

add timeout flag

this should be implemented with a graceful shutdown.

add docker support

rather than passing an address a user should be able to pass a docker image. the test harness should spin up the durable promise implementation before running tests. docker support would also be useful to spin up a postgres and test using that.

add a rate limit flag

in queries per second (QPS) per client.

add more variation in the generator

I have two in mind:

• headers (idempotency)
• cursor and limits for search.

This requires adding more logic in the checker's sequential specification to handle this correctly.

add ci

this includes regular ci plus a cron job that sync the openapi to the durable promise repo.