Expose Pagination to client consumers

[kadamwhite]

Expose Pagination to client consumers

[kadamwhite]

Tagging @carldanley @tkellen @bmac @tbranyen @rmccue @iros for your thoughts, since I've either talked to you about this project, or heard you mentioned by people I've talked to about it

[kadamwhite]

Also @tlovett1 @tollmanz

[carldanley]

@kadamwhite Excellent writeup. All of your points are great. While I was reading through it, I really felt like the following would be a decent solution:

• provide a .head() method that allows you to retrieve the header
• provide a .raw() method that returns the raw request object, if needed (includes option 1)
• Instead of automatically augmenting data based on assumptions, expose an augmentation layer for the developer to either use built in transformations (pagination being 1 of those) OR write their own augmentation to transform the data based on what they need.

This would solve most of what you need and allow the developer to make decisions about pagination as needed.

I need more time to digest this idea but these are my initial thoughts, at the very least. If you expose an augmentation layer, you can introduce new stuff to enhance the library as needed. Meanwhile, you're allowing the developer to write their own methods; which I think results in a very powerful API.

[kadamwhite]

Thanks for the quick response, @carldanley. Some thoughts:

• .head() exists, as it is one of the request methods available on WPRequest. As such, getting the headers independently is still possible.
• .raw() is interesting, and worth considering regardless of the decision here.
• I believe some sort of default behavior is needed, because the link header is a raw string: You have to use a link parser library like parse-link-header or parse-links to deconstruct that into something useful; I think the value that's returned should be exposed in some way to client consumers, because being able to work intuitively with pagination is going to be critical because requests will be limited to max 100 items at once.

[carldanley]

Playing devil's advocate here:

Couldn't you provide the augmentation layer and a built-in transformation for handling pagination? Then have links automatically consume this transformation to deliver the results you're looking for?

[tlovett1]

I'm a fan of option 2. It seems the most intuitive to me rather than having to deal with checking headers. Pagination needs to be added to collections in the WP API backbone client so I'll definitely be following this closely.

[iros]

This is a great writeup @kadamwhite.

here's my 2 cents:

Considering you're making a network request, using then is not that unreasonable. If anything, it should be clear to the user that this is an async operation and might take time. As such, this approach feels the most natural to me.

wp.posts().then(function(resp) {
  // collection:
  console.log( resp.data );
  // pagination URL:
  console.log( resp.links.next );
});

I do like the properties you established in Option 1, example 2 in terms of expressiveness.
Not sure this helps much, but I think you're on the right track.

[kadamwhite]

@iros, there weren't any examples under Option 1: did you mean Option 2, example 2?

The library uses then, or a node-style callback syntax, to handle the asynchronicity no matter what; these are equivalent in the current build:

// Do a GET
wp.posts().get(function(err, data) {});
wp.posts().then(function(data) {}, function(err) {});

// Do a POST
wp.posts().data({}).post(function(err, data) {});
wp.posts().data({}).post().then(function(err, data) {});

As such, it's not the presence of then that bothers me: it's that in the above, the value with which the promise is resolved is the request payload, and if we converted that into the full request (option 1) then every handler would have to explicitly pluck the body off. Not the end of the world, but that's what I was trying to articulate that I was hoping to avoid with Option 2.

[tbranyen]

@kadamwhite I think augmenting the collection makes the most sense. Simply exposing headers or transforms is useful, but not particularly great for pagination discovery. Also introducing a new method next() is rather confusing since you already have a way to fetch posts.

I think that Option 2 offers all the flexibility that's necessary assuming you also change how wp.posts() works so that passing in a page offset will fetch it:

// Make the first request for the first page with the default limit.
wp.posts().then(function(page1) {
  // Resolve with both the first and second page.
  return Promise.all([
    page1,
    // Also fetch the second page for example purposes.
    wp.posts({ page: page1.paging.next })
  ]);
}).then(pages) {});

This would make it very clear on how you could fetch all pages. Building up that Promise.all array and then chaining with a .then.

/2c

[kadamwhite]

Alright; this is the proposed structure for collection responses:

• collection (array)
  • paging (object)
    • total (int; total # of records)
    • totalPages (int; total # of pages)
    • links (object)
      • next (string; URL to next collection)
      • prev (string; URL to previous collection)

I'm also planning to pre-bind paging.next and paging.prev properties, which would be WPRequest objects bound directly to the URLs exposed through paging.links.(prev|next). The reasoning for this is to make it easy to immediately trigger a request for the next or previous collection without changing the behavior of the base wp.* methods (i.e., the parameter suggestion @tbranyen brought up), and to make a clear distinction between exposing the raw URL in paging.links.(next|prev) (which is all we have for links right now) vs a user-facing object in paging.(next|prev).

[kadamwhite]

Initial implementation largely handled in #77, #79 and #80.

[kadamwhite]

Closing as this is officially released in #82 (0.3.0); we may come in and refactor this to make pagination work more as a chained output transformation, rather than part of the core WPRequest functionality, but the public API for pagination info (at least for what the WP API currently provides) is implemented as documented above.