OTA update stopped working from a recent nightly

[rjd22]

OTA update stopped working from a recent nightly

[Frenzie]

Updating should work for all devices now.

[benoit-pierre]

Something did indeed change in the headers of the zsync files available on https://ota.koreader.rocks/:

• koreader-kindlepw2-latest-stable.zsync:

zsync: 0.6.2
Filename: koreader-kindlepw2-v2024.03.1.targz
MTime: Tue, 05 Mar 2024 20:46:42 +0000
Blocksize: 2048
Length: 101519360
Hash-Lengths: 2,2,5
Z-URL: koreader-kindlepw2-v2024.03.1.targz
SHA-1: 9db227f27cc414ce610d4f2e3d158c0f07b2c1df
Z-Map2: 61981

• koreader-kindlepw2-latest-nightly.zsync:

zsync: 0.6.2
Safe: Z-Filename Recompress MTime
Z-Filename: koreader-kindlepw2-v2024.03.1-63-gc345de5d7_2024-04-13.targz
Filename: koreader-kindlepw2-v2024.03.1-63-gc345de5d7_2024-04-13.targz
MTime: Sun, 14 Apr 2024 06:09:51 +0000
Blocksize: 2048
Length: 102236160
Hash-Lengths: 2,2,5
Z-URL: koreader-kindlepw2-v2024.03.1-63-gc345de5d7_2024-04-13.targz
SHA-1: 263b269f58cf25f899cbd95fa59a9bd784ec8c1e
Recompress: 1f8b0800000000000003 --rsync --no-name
Z-Map2: 76226

I believe this new one: Recompress: 1f8b0800000000000003 --rsync --no-name, triggers the code above, which can't work with the busybox implementation of gzip.

[yparitcher]

TLDR;
Passing -C to zsyncmake in nightswatcher might fix this.

Not a a zsync expert but after lookin gat the zsyncmake src for focal which runs nightswatcher.
The difference between the stable and nightly header (thanks @benoit-pierre for the pointer) is that the nightly has information for the client to do recompression.

-C
    Tells zsyncmake not to generate any instructions in the .zsync telling the client to compress the data it receives. This is implied by -z, but this option is here in case you compress a file yourself only for the transfer, but want the client to end up with the uncompressed file (e.g. you are transferring an ISO, which is held compressed on the server, but which the client cannot use unless it is uncompressed). Without -C, zsyncmake will produce directions for the client to compress the file it receives where appropriate; -C is here so you can stop it telling the client to do that. 

It looks like we want the client (device) to have the uncompressed tar (not the targz) as that is what we need for the update, and as discovered the busybox tar crashes trying to recompress the file. @NiLuJe ?

zsyncmake uses the following heuristic to determine whether gzip was called with compatible args and can be client recompressed:

Spoiler

/* opt_str = guess_gzip_options(filename_str)
* For the given (gzip) file, try to guess the options that were used with gzip
* to create it.
* Returns a malloced string containing the options for gzip, or NULL */
static const char *const try_opts[] =
  { "--best", "", "--rsync", "--rsync --best", NULL };
#define SAMPLE 1024

char *guess_gzip_options(const char *f) {
  char orig[SAMPLE];
  {   /* Read sample of the header of the compressed file */
      FILE *s = fopen(f, "r");
      if (!s) {
          perror("open");
          return NULL;
      }
      if (!read_sample_and_close(s, SAMPLE, orig))
          return NULL;
  }
  {
      int i;
      const char *o;
      char *enc_f = encode_filename(f);
      int has_mtime_fname;

      {
          int has_mtime = zhead_has_mtime(orig);
          int has_fname = zhead_has_fname(orig);

          if (has_mtime && !has_fname) {
              fprintf(stderr, "can't recompress, stream has mtime but no fname\n");
              return NULL;
          }
          else if (has_fname && !has_mtime) {
              fprintf(stderr, "can't recompress, stream has fname but no mtime\n");
              return NULL;
          }
          else {
              has_mtime_fname = has_fname; /* which = has_mtime */
          }
      }

      /* For each likely set of options, try recompressing the content with
       * those options */
      for (i = 0; (o = try_opts[i]) != NULL; i++) {
          FILE *p;
          {   /* Compose command line */
              char cmd[1024];
              snprintf(cmd, sizeof(cmd), "zcat %s | gzip -n %s 2> /dev/null",
                      enc_f, o);

              /* And run it */
              if (verbose)
                  fprintf(stderr, "running %s to determine gzip options\n",
                          cmd);
              p = popen(cmd, "r");
              if (!p) {
                  perror(cmd);
              }
          }

          if (p) {   /* Read the recompressed content */
              char samp[SAMPLE];
              if (!read_sample_and_close(p, SAMPLE, samp)) {
                  ;       /* Read error - just fail this one and let the loop
                           * try another */
              }
              else {
                  /* We have the compressed version with these options.
                   * Compare with the original */
                  const char *a = skip_zhead(orig);
                  const char *b = skip_zhead(samp);
                  if (!memcmp(a, b, 900))
                      break;
              }
          }
      }
      free(enc_f);

      if (!o) {
          return NULL;
      }
      else if (has_mtime_fname) {
          return strdup(o);
      }
      else {  /* Add --no-name to options to return */
          static const char noname[] = { "--no-name" };
          char* opts = malloc(strlen(o)+strlen(noname)+2);
          if (o[0]) {
              strcpy(opts, o);
              strcat(opts, " ");
          }
          else { opts[0] = 0; }
          strcat(opts, noname);
          return opts;
      }
  }
}

I didnt test further we updated the ci to 20.04 when this happened, and i assume the old tar files were failing this heuristic leading to our desired outcome -C, and the new tar make files that zsync understands causing it to try to tell the client to recompress.

[Commodore64user]

Not just pocketbook, I had the same issue on kindle 4. After like 3 more attempts it finally successfully updated.

[enigmasc]

Same with my Kobos.

[Frenzie]

Same.

Nothing should've relevantly changed since 3 months ago, so perhaps a more recent gzip on the input end causes different behavior on the device?
https://github.com/koreader/koreader/blob/0dbfac22b006d575472a72617eee26e17bac444d/Makefile#L223C29-L223C45

[Monirzadeh]

same in PW3

[yparitcher]

Just triggered this, the error is in zsync2 not being able to verify the file which indeed was updated in jan.

[Hzj-jie]

Same on kindle voyage and kobo aura hd.

[benoit-pierre]

I don't know what triggered this, but there's some code in zsync2 that clearly won't work with the busybox version of gzip available on some platforms (--rsync): https://github.com/NiLuJe/zsync2/blob/b4953c95611d9507bd6c7b94dd8447ae87ead734/lib/libzsync/zsync.c#L65.

Also, TIL: the documented gzip option is --rsyncable, but apparently, with the official binary, you can truncate long options. This works: gzip </dev/null >/dev/null --bes --no-na --rsy, but again, not with the busybox version.

[Frenzie]

Also, TIL: the documented gzip option is --rsyncable, but apparently, with the official binary, you can truncate long options. This works: gzip </dev/null >/dev/null --bes --no-na --rsy, but again, not with the busybox version.

This behavior is the same in tmux for example btw. (Anything from attach down to a.)

[benoit-pierre]

Indeed, zsync2 end up re-compressing the file after transfer. Regenerating koreader-kindlepw2-latest-nightly.zsync for example with -C and re-testing yield the correct behavior.

[NiLuJe]

Yep, IIRC, in case someone ever thought of updating, zsync2's make doesn't support this at all (or... something, can't recall ;D).

[Frenzie]

I'm afraid I don't see any difference with -C (only for Kobo atm, the rest are old).

Edit: in behavior I mean, obviously there's a difference

Broken

zsync: 0.6.2
Safe: Z-Filename Recompress MTime
Z-Filename: koreader-pocketbook-v2024.03.1-63-gc345de5d7_2024-04-13.targz
Filename: koreader-pocketbook-v2024.03.1-63-gc345de5d7_2024-04-13.targz
MTime: Sun, 14 Apr 2024 06:09:38 +0000
Blocksize: 2048
Length: 102492160
Hash-Lengths: 2,2,5
Z-URL: koreader-pocketbook-v2024.03.1-63-gc345de5d7_2024-04-13.targz
SHA-1: 60ad7208eb6b4d79b5b727d17bb4ec8f8c95f5f3
Recompress: 1f8b0800000000000003 --rsync --no-name
Z-Map2: 76502

New, should be good but doesn't work with abort and retry.

zsync: 0.6.2
Filename: koreader-kobo-v2024.03.1-68-g715f5aa74_2024-04-16.targz
MTime: Tue, 16 Apr 2024 20:26:25 +0000
Blocksize: 2048
Length: 103690240
Hash-Lengths: 2,2,5
Z-URL: koreader-kobo-v2024.03.1-68-g715f5aa74_2024-04-16.targz
SHA-1: 334fc54aaf8e78ea7f883a9084058d08390a552c
Z-Map2: 77313

Old, looks the same.

zsync: 0.6.2
Filename: koreader-kobo-v2024.03.1.targz
MTime: Tue, 05 Mar 2024 20:48:25 +0000
Blocksize: 2048
Length: 102952960
Hash-Lengths: 2,2,5
Z-URL: koreader-kobo-v2024.03.1.targz
SHA-1: 90397257014152ec2c39c365cf034df34d2d2aef
Z-Map2: 62777

[yparitcher]

Is the error the same?

[Frenzie]

Probably, it's something about gzip anyway.

[Frenzie]

Ah, false alarm, Cloudflare is delivering the cached old file to my Kobo.

[rjd22]

Confirmed working on Pocketbook :)

[Monirzadeh]

Thanks.
Working on PW3 too.

[enigmasc]

Working on Kobo, too. Thanks.