Implement some sort of a cache

[AlexDaniel]

Implement some sort of a cache

[AlexDaniel]

Interesting issue. See #117 also.

Basically, we have something like this:

• Builds that are in long-term storage mode, where 20 builds are put into the same archive to save space
  • using lrz
  • extremely low disk usage (≈10 MiB per archive, which is ≈0.5 MiB per build)
  • <2 seconds to extract required build
• Normally archived builds, where each build is in its own archive
  • using zstd
  • low disk usage (≈4.7 MiB per archive/build)
  • <0.2 seconds to extract required build
• A new level proposed by this ticket
  • using nothing, the build is left on the disk uncompressed
  • 28 MiB per build
  • Low to zero delay to get the required build

[AlexDaniel]

I wonder if we should use lz4 for this. According to my measurements in #23, lz4 is just ≈38 milliseconds (≈30%) faster than zstd, so we are not winning anything this way? I'd consider 87ms delay to get the required build as too slow if we are talking about fast cache.

[MasterDuke17]

You mean use lz4 for the fast path? Instead of uncompressed?

[AlexDaniel]

@MasterDuke17 yea…

[AlexDaniel]

Anyway, IMO no need to use lz4, just leave the build uncompressed.