Provide total count of query results

[pyeve]

Provide total count of query results

[nicolaiarocci]

Need to figure out a good way to return the total item count, as _items is just a list of documents (and as you say won't be accurate when paginated). Maybe by using some X- custom header... or adding an additional [metadata] item to the payload (not liking this one much, as we already have both _items and _links). Will do some brainstorming on this - thanks!

[nicolaiarocci]

I came out with 2 possible solutions for this, both REST-compliant:

1. Use a custom X-Pagination-Total-Count header which would simply return the total number of records for the query, possibly paired with X-Pagination-Current-Page, X-Pagination-Per-Page.

2. Rename the _links dictionary to _metadata and add new keys to it: total_count, per_page, etc. This would break backward compatibility but I wouldn't worry much about it because 1) we're still in alpha after all and 2) the change is trivial.

I'm tending towards the second option because it would also allow for future expansion, like adding more metadata as needed. Opinions?

[flaper87]

I'd suggest using link headers. A good example is GH's ussage[0]. Here's the RFC[1]

[0] http://developer.github.com/v3/#pagination
[1] http://tools.ietf.org/html/rfc5988

[nicolaiarocci]

@flaper87 that would be an alternative way to handle pagination links themselves, which are already supported even though with a slightly different format (see HATEOAS). I explored that rfc a few days ago but it does not seem to provide any directive about handling the actual query record count, which is what this ticket is really about.

[mario]

If we must have this, I'd prefer X-Pagination-Total-Count approach.

[flaper87]

@nicolaiarocci But, is it really needed? I mean, having the last page should be enough, right?

What about adding page to the link dictionary and adding first and last? The idea is to keep the API as consistent as possible.

_NOTE:_ I'm not sure the page key is actually needed once you have the link to the last page, it might be, though.

"_links": {
    "self": {
        "href": "localhost:5000/people/",
        "title": "people"
    },
    "parent": {
        "href": "localhost:5000",
        "title": "home"
    },
    "next": {
        "page": 2,
        "href": "localhost:5000/people/?page=2",
        "title": "next page"
    },
    "last": {
        "page": 100,
        "href": "localhost:5000/people/?page=100",
        "title": "last page"
    }
}

[nicolaiarocci]

That's an interesting approach indeed. Let's see what's the OP use case if any.

[nicolaiarocci]

The idea is to keep the API as consistent as possible.

Precisely, that's why I'm so torn on this one. I find that your suggestion is as consistent as it can get. Thanks!

[jdm]

I had a couple use cases in mind - sometimes I'd like to display one page of results at a time, and have the usual "1 .. 4 [5] 6 .. 98" link display. Other times I want to be able to load all results progressively, and display some kind of useful status to user (X of Y results).

[mario]

Afaik having the last page would solve this problem :)

[nicolaiarocci]

@jdm thought so. I guess I'll go ahead and implement @flaper87 solution (minus the page as it should not be needed). In any case, we can always add a X-Total-Count header.