Poor heuristic choices for sending incremental diffs

[amescornish]

Poor heuristic choices for sending incremental diffs

[RandomReaper]

👍 Same problem here.

[AmesCornish]

It's a bit hard to diagnose this without having the snapshots. Note that the heuristic considers factors other than the diff size, including how "tall" the diff stack is on the destination. i.e., it won't create a thousand one-day diffs each depending on the previous, because if any one of those thousand goes bad, you lose the whole thing. Buttersink is designed to occasionally diff from an "old" snapshot, so that your diff repo is more reliable.

In any event, I can see that it would at least be helpful to make the heuristic process more transparent, and maybe give some options for tweaking it. I'll leave this bug open to address that.

[RandomReaper]

Don't you think sending the snapshots in the order they are taken should be sufficient?
I mean, if my source disk has enough space for storing all snapshots, a destination disk of the same size will suffice for storing them, and this is clearly not the case using the current algorithm.

[AmesCornish]

Indeed. My comment should only be relevant when S3 is the destination. I'll investigate further.

[eugene-bright]

Base for diffs should be updated ones any snapshot transfer is finished.

[eugene-bright]

My extra note on optimizations #58.

[AmesCornish]

The case of transferring into a btrfs system should be addressed in d25e71e. "Tall" diff chains will only be avoided for S3, which is storing diffs.