Potential need for a `pull-batch-delay` option

[shayonj]

Potential need for a `pull-batch-delay` option

[brycethornton]

Closing this as compressing WAL files helped get this done without the need for arbitrary sleeps.

[shayonj]

oh interesting! Yeah, I'd be down for that, but I wonder if a shorter pull-batch-count would work here? say 100 rows at a time? Or would that be quite slow?

I reckon the delete from is only adding to the additional WAL volume. I will try to see if there are some alternative design approaches we can perform to reduce the WAL level usage soon. In the meantime, I am open to pull-batch-delay also.

[brycethornton]

Yeah, I'm hoping to find a solution that doesn't involve adding another flag. I may tinker with pull-batch-count to see if it can help. I also found an option on Heroku to compress WAL files so you can fit more into the same space. I'll update here if/when I find a workaround. Thanks for your feedback!

[shayonj]

wonderful! thanks for sharing @brycethornton