HEAD requests return forbidden

[adoptopenjdk]

HEAD requests return forbidden

[bmuskalla]

May I ask why using Gradle as User-Agent gets a different treatment and works? Funny enough, my interest in this API is driven by supporting AdoptOpenJDK in Gradle ;)

[ParkerM]

I can reproduce your curl request error, and adding -H "user-agent: Gradle" to the end seems to fix it. Not sure if that behavior was intentional but seems kinda buggy to me. (Unless it's a weird limitation of the service provider or something like that) (this appears to be intended behavior per the tests)

[ParkerM]

A test attempting to replicate the failure can be found here (sorry about all the reference noise above).

On second thought, I think the test failure here is just due to missing test fixture data (it returns 404 instead of 403). The same test passes with jdk8u212-b04, which also returns a 403 using the curl command mentioned above.

[ParkerM]

It appears that this may be related to the way AWS handles (or does not handle) HEAD requests. If I include the --cookie option like so:

  curl -v -L --cookie --head "https://api.adoptopenjdk.net/v3/binary/version/jdk-11.0.6%2B10/linux/x64/jdk/hotspot/normal/adoptopenjdk?project=jdk"

Then the API -> GitHub -> AWS redirect goes through, then proceeds to vomit binary into my terminal until I have to force close it.

You can see the same behavior by hitting the GitHub release URL (with -L) or the AWS redirect URL (without -L) directly.

[karianna]

Ah yes, our API does require a user agent - CC @johnoliver for more context

[johnoliver]

It was originally put in to allow Gradle to download our binaries (see elastic/elasticsearch#55125 and https://github.com/elastic/elasticsearch/pull/55127/files @james-crowley may be able to give more context there) and yes this was to be a work around for the fact that aws did not allow HEAD requests and Gradle required them to work.

In general I am not a fan of our current HEAD behaviour as (from my reading of it) HEAD should return the same as GET, but to work around the aws and gradle issues HEAD (with the gradle user agent) will return a response while GET will redirect you. As this is non standard we felt it was better to limit the effect to gradle only. Unfortunately adding the functionality highlights HEAD in the swagger docs, I could not find a way to hide the HEAD request from the documentation which is not ideal.

[james-crowley]

@johnoliver Thanks for tagging me in this issue. @bmuskalla like @johnoliver mentioned this implementation of checking if the user-agent is Gradle was due to AWS not allowing HEAD requests.

In my digging around I did find this issue/comment, gradle/gradle#5322 (comment). I was excited to see that Gradle worked with GitHub to support HEAD requests. When trying to fetch the AdoptOpenJDK file, GitHub redirects to AWS and Gradle tries to do a HEAD request on AWS but AWS does not support HEAD requests. Thus downloading the file fails and errors out when building with Gradle.

I agree with @johnoliver, this implementation in the API is not ideal. @bmuskalla is there anything on Gradle’s side of things that can be implemented to fix this bug? Also let me know if I can help with anything. Glad to see AdoptOpenJDK support coming to Gradle!

[karianna]

This has been resolved