GET urls often fail to meet the relevant spec standards

[microsoft]

GET urls often fail to meet the relevant spec standards

[esatterwhite]

I've always been of the thought to construct query strings in a way to made it as easy on both the client and server as possible. In the context of the points above, Using a well known, url safe separator, like __. and repeating keys over comma separated values

?$filter=name eq 'david'&$orderBy=hireDate desc

would become

?name__eq=david&orderBy=-hireDate

assuming the filter type/name is always the last bit of a filter, this also works for nested or relational values

?user__address__state__startswith=W&orderBy=-hireDate

If, for example you wanted an age range of 18 - 25 with compound ordering just include the same key more than once

?age__gte=18&age__lte=25&orderBy=-hireDate&orderBy=age

URL safe, readable, easy to parse, easy to construct for clients, and compliant with relevant http specs.

[dret]

On 2016-07-19 22:42, James Cox wrote:

In this example, the success of this request is contingent on the HTTP
client replacing || with |%20|. Are you sure that's what you mean here?

i'd assume that proper URI escaping is a requirement for any of the
examples given. showing examples with escaping is a bit hard to read,
so i think showing them unescaped makes sense. but yes, there should be
very clear language about the fact that URI escaping is absolutely
necessary.

and of course there's the wonderful issue of IRIs. those at least should
be mentioned and explained, in case anybody needs to use non-ASCII
characters in URIs.

[imajes]

@dret: Yes, it can make it harder to read, which to me implies that it's a poorer design choice, as it muddies what is considered a canonical resource. Should these requests be made with a JSON payload, describing the arguments? How should this request, for example, be made when PII is stored? What is the appropriate Header name to use in that case? Why are $ being used to determine keys, when the HTTP spec clearly indicates how query string keys should operate? etc etc.

[cleemullins]

@imajes We made a deliberate choice not to escape the URL's in the examples as it made such an impact on readability. Are you up for writing a section describing the need to escape the URLs?

The use of $ and spaces, such as the "$filter=foo gt 99" has it's origin in OData, which is a fairly well known and standard family of patterns for building APIs over HTTP. We could certainly debate the points of OData here, but it is broadly used across our ecosystem, has good tooling support, and is standardized.

Switching to JSON payloads describing arguments would mean not using GET requests anymore and are instead using POST (with a payload) to do GET operations. This pattern was very commonly used years ago with SOAP over HTTP, and I don't think anyone wants to revisit that...

The other big advantage (to me, and others) is that by having the query as arguments it's bookmarkable. Being able to easily cut/paste/bookmark these is quite developer friendly.

[cleemullins]

@dret Can you explain what you mean by this:

and of course there's the wonderful issue of IRIs. those at least should
be mentioned and explained, in case anybody needs to use non-ASCII
characters in URIs.

I think I'm missing something obvious, and would like to learn more.

I know Unicode well (although it's not clear if anybody actually understands the total complexity of Unicode) , and know IRI's and URI's fairly well, but an clearly missing something. I'm guessing you're not going to recommend punycoding our URI's... :)

[dret]

On 2016-07-23 13:59, Chris Mullins wrote:

@dret https://github.com/dret Can you explain what you mean by this:
and of course there's the wonderful issue of IRIs. those at least should
be mentioned and explained, in case anybody needs to use non-ASCII
characters in URIs.

you're saying you intentionally did not encode the URIs in examples. i
think that's risky as people may use those literally, and it would be
good to make it as clear as possible that those need to be encoded
before being used in protocols.

if people start thinking about query-like things in URIs and there's
non-ASCII characters in there ("find all cities named Zürich"), then
those queries not only need to be URI-encoded, but also IRI-encoded.

for now it seems that there's no guidance about that, and it seems that
in particular when supporting query-style URIs a la OData, that's
something that needs to be addressed.

[jmealo]

@esatterwhite: PostgREST features a mostly functionally equivalent API that doesn't use spaces (for comparisons sake).

[esatterwhite]

Ah interesting. I have a couple issues with that. Periods have meaning in most written and programming languages. Which can lead to confusion or security holes. Also, the periods are in the value. This means you have to parse values to infer meaning. foo=eq.10.50 and foo=not.eq.10.50. Parsing that, even visually is a little complicated. Is that just an invalid query? Does 50 get ignored? Should it? Or maybe it should mean between 10 and 50? Or think in terms of sentences. foo=eq.hi.thanks,bill. If the filters are a part of they key, that complexity and confusion mostly goes away. foo__eq=hi.thanks,bill. Grammatically terrible, but at least I don't need to jump through a hoop to figure it out where the data starts and stops.

[O1O1O1O]

@esatterwhite

URL safe, readable, easy to parse, easy to construct for clients, and compliant with relevant http specs.

But it doesn't really handle complex queries such as the example (priority eq 1 or city eq 'Redmond') and price gt 100

A quick search turned up RQL which can handle all these and more: https://github.com/kriszyp/rql

[esatterwhite]

@O1O1O1O That is pretty neat. But there is some odd syntax there that doesn't sit too well with me

(foo=3|foo=bar)&price=lt=10

this has price=lt=10 which seems incorrect and gives me pause every time I look at it.
It still think it would be better written as

(foo__eq=3|foo__eq=bar)&price__lt=10

[O1O1O1O]

I'm inclined to agree, although they do say the price=(lt=10) is just an abbreviation for the parenthetical version - which I think it would be better to stick with always.

Plus the LHS part seems rather abusive of parameter parsing since I guess most frameworks would give you a parameter called (foo with value 3|foo=bar). For sure it's intended to be used with some library code that just parses it all for you and builds some syntax tree that is sensical.

So long as we can go with custom query string parsing I'd go with a hybrid something like this:

(priority__eq(1)|city__eq('Redmond'))&price__gt(100)

Or to not depend on any tricky use of | or & and depending on manual query string parsing:

query=and(or(priority__eq(1),city__eq('Redmond')),price__gt(100))

[esatterwhite]

For my tastes, I don't see to much value in not leveraging as much of the standard convention of <key>=<value>.

also with this syntax

(priority__eq(1)|city__eq('Redmond'))

the parens are much more ambiguous between grouping of statements and function calls.

A lot of the things we've gone back and forth on are why graphql was conceived. 🤷‍♂ The use cases for querystrings in urls just don't meet modern application needs anymore. Sure, we can Make it work, but should we have to is the real question.

maybe this discussion is moot