Artifact Downloading Fails on Github Redirect

[nerves-project]

Artifact Downloading Fails on Github Redirect

[voltone]

Suggested to reopen the Erlang issue that triggered the change in behaviour, since I think others will be bitten by this too. https://bugs.erlang.org/browse/ERL-316

[mmmries]

I dove a little bit into this so far to see that Nerves.Utils.HTTPClient is using httpc under the hood so I decided to try this with a synchronous httpc call and I get this result:

iex(10)> result = :httpc.request(:get, {'https://github.com/nerves-project/toolchains/releases/download/v0.10.0/nerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.darwin-x86_64.tar.xz', [{'Content-Type', 'application/octet-stream'}]}, [timeout: :infinity, autoredirect: true], [])
{:ok,
 {{'HTTP/1.1', 403, 'Forbidden'},
  [{'date', 'Thu, 16 Mar 2017 12:41:08 GMT'}, {'server', 'AmazonS3'},
   {'content-length', '3194'}, {'content-type', 'application/xml'},
   {'x-amz-request-id', '06E4B76EE84B978D'},
   {'x-amz-id-2',
    'cBUfmjaWk9PsjhOh/Nia6qUYN8TiRfszhc9KmVKYnspwgbAhNwdEnihU2mwUOhELbA7FGMbk2pc='}],
  '<?xml version="1.0" encoding="UTF-8"?>\n<Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your key and signing method.</Message><AWSAccessKeyId>AKIAISTNZFOVBIJMK3TQ</AWSAccessKeyId><StringToSign>AWS4-HMAC-SHA256\n20170316T124108Z\n20170316/us-east-1/s3/aws4_request\n29acdc36f287b78ae2be4287df19a449b05ca7bf31edf9526023b55d5b15a59e</StringToSign><SignatureProvided>d8986cb53df562af362a6adab6934b410c1adbd40237944b6052aea4a95b38de</SignatureProvided><StringToSignBytes>41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 0a 32 30 31 37 30 33 31 36 54 31 32 34 31 30 38 5a 0a 32 30 31 37 30 33 31 36 2f 75 73 2d 65 61 73 74 2d 31 2f 73 33 2f 61 77 73 34 5f 72 65 71 75 65 73 74 0a 32 39 61 63 64 63 33 36 66 32 38 37 62 37 38 61 65 32 62 65 34 32 38 37 64 66 31 39 61 34 34 39 62 30 35 63 61 37 62 66 33 31 65 64 66 39 35 32 36 30 32 33 62 35 35 64 35 62 31 35 61 35 39 65</StringToSignBytes><CanonicalRequest>GET\n/releases/63556949/79da9826-0324-11e7-9368-4165d3059da1.xz\nX-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIAISTNZFOVBIJMK3TQ%2F20170316%2Fus-east-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20170316T124108Z&amp;X-Amz-Expires=300&amp;X-Amz-SignedHeaders=host&amp;actor_id=0&amp;response-content-disposition=attachment%3B%20filename%3Dnerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.darwin-x86_64.tar.xz&amp;response-content-type=application%2Foctet-stream\nhost:github-cloud.s3.amazonaws.com:443\n\nhost\nUNSIGNED-PAYLOAD</CanonicalRequest><CanonicalRequestBytes>47 45 54 0a 2f 72 65 6c 65 61 73 65 73 2f 36 33 35 35 36 39 34 39 2f 37 39 64 61 39 38 32 36 2d 30 33 32 34 2d 31 31 65 37 2d 39 33 36 38 2d 34 31 36 35 64 33 30 35 39 64 61 31 2e 78 7a 0a 58 2d 41 6d 7a 2d 41 6c 67 6f 72 69 74 68 6d 3d 41 57 53 34 2d 48 4d 41 43 2d 53 48 41 32 35 36 26 58 2d 41 6d 7a 2d 43 72 65 64 65 6e 74 69 61 6c 3d 41 4b 49 41 49 53 54 4e 5a 46 4f 56 42 49 4a 4d 4b 33 54 51 25 32 46 32 30 31 37 30 33 31 36 25 32 46 75 73 2d 65 61 73 74 2d 31 25 32 46 73 33 25 32 46 61 77 73 34 5f 72 65 71 75 65 73 74 26 58 2d 41 6d 7a 2d 44 61 74 65 3d 32 30 31 37 30 33 31 36 54 31 32 34 31 30 38 5a 26 58 2d 41 6d 7a 2d 45 78 70 69 72 65 73 3d 33 30 30 26 58 2d 41 6d 7a 2d 53 69 67 6e 65 64 48 65 61 64 65 72 73 3d 68 6f 73 74 26 61 63 74 6f 72 5f 69 64 3d 30 26 72 65 73 70 6f 6e 73 65 2d 63 6f 6e 74 65 6e 74 2d 64 69 73 70 6f 73 69 74 69 6f 6e 3d 61 74 74 61 63 68 6d 65 6e 74 25 33 42 25 32 30 66 69 6c 65 6e 61 6d 65 25 33 44 6e 65 72 76 65 73 5f 74 6f 6f 6c 63 68 61 69 6e 5f 61 72 6d 76 36 5f 72 70 69 5f 6c 69 6e 75 78 5f 67 6e 75 65 61 62 69 2d 30 2e 31 30 2e 30 2e 64 61 72 77 69 6e 2d 78 38 36 5f 36 34 2e 74 61 72 2e 78 7a 26 72 65 73 70 6f 6e 73 65 2d 63 6f 6e 74 65 6e 74 2d 74 79 70 65 3d 61 70 70 6c 69 63 61 74 69 6f 6e 25 32 46 6f 63 74 65 74 2d 73 74 72 65 61 6d 0a 68 6f 73 74 3a 67 69 74 68 75 62 2d 63 6c 6f 75 64 2e 73 33 2e 61 6d 61 7a 6f 6e 61 77 73 2e 63 6f 6d 3a 34 34 33 0a 0a 68 6f 73 74 0a 55 4e 53 49 47 4e 45 44 2d 50 41 59 4c 4f 41 44</CanonicalRequestBytes><RequestId>06E4B76EE84B978D</RequestId><HostId>cBUfmjaWk9PsjhOh/Nia6qUYN8TiRfszhc9KmVKYnspwgbAhNwdEnihU2mwUOhELbA7FGMbk2pc=</HostId></Error>'}}

If I turn off auto redirects I get this as the first result:

iex(11)> result = :httpc.request(:get, {'https://github.com/nerves-project/toolchains/releases/download/v0.10.0/nerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.darwin-x86_64.tar.xz', [{'Content-Type', 'application/octet-stream'}]}, [timeout: :infinity, autoredirect: false], [])
{:ok,
 {{'HTTP/1.1', 302, 'Found'},
  [{'cache-control', 'no-cache'}, {'date', 'Thu, 16 Mar 2017 12:41:46 GMT'},
   {'location',
    'https://github-cloud.s3.amazonaws.com/releases/63556949/79da9826-0324-11e7-9368-4165d3059da1.xz?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAISTNZFOVBIJMK3TQ%2F20170316%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20170316T124146Z&X-Amz-Expires=300&X-Amz-Signature=c3454537ab4dd8188ae88f0dd268a0628021323595ef33f14dd741820b3c0bc9&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dnerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.darwin-x86_64.tar.xz&response-content-type=application%2Foctet-stream'},
   {'server', 'GitHub.com'}, {'vary', 'X-PJAX'}, {'content-length', '644'},
   {'content-type', 'text/html; charset=utf-8'}, {'status', '302 Found'},
   {'x-ua-compatible', 'IE=Edge,chrome=1'},
   {'set-cookie',
    'logged_in=no; domain=.github.com; path=/; expires=Mon, 16 Mar 2037 12:41:46 -0000; secure; HttpOnly'},
   {'set-cookie',
    '_gh_sess=eyJzZXNzaW9uX2lkIjoiMWUwZDFhMjAyYzQxZTE5NWZiZTljMmU0ZTUxNTE4ZDUiLCJzcHlfcmVwbyI6Im5lcnZlcy1wcm9qZWN0L3Rvb2xjaGFpbnMiLCJzcHlfcmVwb19hdCI6MTQ4OTY2ODEwNn0%3D--87187a5e30435eb1052f46d9b94e2a4abf1e2e94; path=/; secure; HttpOnly'},
   {'x-request-id', 'e8b152160c1eafd9e80579588ecdfb55'},
   {'x-runtime', '0.029039'},
   {'content-security-policy',
    'default-src \'none\'; base-uri \'self\'; block-all-mixed-content; child-src render.githubusercontent.com; connect-src \'self\' uploads.github.com status.github.com collector.githubapp.com api.github.com www.google-analytics.com github-cloud.s3.amazonaws.com wss://live.github.com; font-src assets-cdn.github.com; form-action \'self\' github.com gist.github.com; frame-ancestors \'none\'; img-src \'self\' data: assets-cdn.github.com identicons.github.com collector.githubapp.com github-cloud.s3.amazonaws.com *.githubusercontent.com; media-src \'none\'; script-src assets-cdn.github.com; style-src \'unsafe-inline\' assets-cdn.github.com'},
   {'strict-transport-security',
    'max-age=31536000; includeSubdomains; preload'},
   {'public-key-pins',
    'max-age=5184000; pin-sha256="WoiWRyIOVNa9ihaBciRSC7XHjliYS9VwUGOIud4PB18="; pin-sha256="RRM1dGqnDFsCJXBTHky16vi1obOlCgFFn/yOhI/y+ho="; pin-sha256="k2v657xBsOVe1PQRwOsHsw3bsGT2VzIqz5K+59sNQws="; pin-sha256="K87oWBWM9UZfyddvDfoxL+8lpNyoUB2ptGtn0fv6G2Q="; pin-sha256="IQBnNBEiFuhj+8x6X8XLgh01V9Ic5/V3IRQLNFFc7v4="; pin-sha256="iie1VXtL7HzAMF+/PVPR9xzT80kQxdZeJ+zduCB3uj0="; pin-sha256="LvRiGEjRqfzurezaWuj8Wie2gyHMrW5Q06LspMnox7A="; includeSubDomains'},
   {'x-content-type-options', 'nosniff'}, {'x-frame-options', 'deny'},
   {'x-xss-protection', '1; mode=block'},
   {'x-served-by', '29885c8097c6d503a86029451b2e021c'},
   {'x-github-request-id', 'C928:48A8:257F75B:3BD8E5F:58CA880A'}],
  '<html><body>You are being <a href="https://github-cloud.s3.amazonaws.com/releases/63556949/79da9826-0324-11e7-9368-4165d3059da1.xz?X-Amz-Algorithm=AWS4-HMAC-SHA256&amp;X-Amz-Credential=AKIAISTNZFOVBIJMK3TQ%2F20170316%2Fus-east-1%2Fs3%2Faws4_request&amp;X-Amz-Date=20170316T124146Z&amp;X-Amz-Expires=300&amp;X-Amz-Signature=c3454537ab4dd8188ae88f0dd268a0628021323595ef33f14dd741820b3c0bc9&amp;X-Amz-SignedHeaders=host&amp;actor_id=0&amp;response-content-disposition=attachment%3B%20filename%3Dnerves_toolchain_armv6_rpi_linux_gnueabi-0.10.0.darwin-x86_64.tar.xz&amp;response-content-type=application%2Foctet-stream">redirected</a>.</body></html>'}}

So it appears that httpc is not following that redirect correctly. I tested the request with curl on the command line and in my browser and they both get a successful download.

[mobileoverlord]

I was unable to reproduce this after creating a new project and forcing the artifacts to download again. Are you on OTP 19.3 by chance?

[mobileoverlord]

womp womp confirmed on 19.3

[mmmries]

yep, I'm on 19.3 as well. Should I got ahead and try to manually handle the redirect on 19.3 or maybe just document this as a known issue with 19.3?

[mobileoverlord]

It appears that inets in 19.3 adds the port number to the host header which breaks aws signing signature on redirects as per OTP-14097 from the 19.3 release notes http://erlang.org/download/otp_src_19.3.readme

It seems the only way to handle it is to handle the redirects ourselves.

[voltone]

I created a PR to fix the root issue in httpc: erlang/otp#1381

When I tested the toolchain download with the patch it worked. Anyway, even if it gets merged into 19.3.1 I guess you'll want to keep your workaround to avoid issues for people on 19.3...

[mmmries]

@voltone I think that's a good fix for erlang. Agree that we probably want to leave our shim in place to avoid confusing users, but maybe we leave a comment in the code about why we needed this shim so we can remove it once 19.3 is no longer generally used?