Giter Site home page Giter Site logo

matterhorn's People

Contributors

blakechambers avatar blasterpal avatar nakort avatar

Watchers

avatar avatar avatar

Forkers

nakort blasterpal

matterhorn's Issues

Request Specs/Implementation - Updating resources

Using JSON API as guide from http://jsonapi.org/format/#crud-updating to

  • Uses PATCH method. PATCH /posts/1
  • Missing attributes do not affect resource, only attributes sent to server will change.
  • Implement normal nested attributes for relations for 0.1.0
  • Ensure duplicates are not created when PATCHing relationships
  • Support Rails PUT
  • Errors http://jsonapi.org/format/#errors & Matterhorn errors as specified athttps://github.com/blakechambers/matterhorn/blob/master/ROADMAP.md

Note

Resource Error Handling

'Errors. Errors returned from server will be keyed with a top level "errors" object. Each error should provide a "title" per spec for any Matterhorn errors. Later versions of the api should also provide "detail" and "code".'

Example returned error:

{:errors=>
  [{:title=>"resource_error", :detail=>"author: can't be blank", :status=>422},
   {:title=>"resource_error", :detail=>"body: can't be blank", :status=>422},
   {:title=>"resource_error", :detail=>"title: can't be blank", :status=>422}]}

Multi-id Resource GET

  • Get collection scoped to ids provided /posts/1,2,3,4,5
    Example:
    /posts/5519bc4368616e50aa070000,5519bc4368616e50aa090000,5519bc4368616e50aa0b0000,5519bc4368616e50aa0d0000.json
  • Get relation scoped to ids provided /posts/1,2,3,4,5/vote
  • This PR/Issue also adds #25 - Nested relations GET

Ordering

Ordering

While json api does support sorting, it's version is more specific to providing something flexible for all types of sorting and ordering.

While this is nice, in a production system (and certainly for public apis) locking down the ordering to some named that can be managed server-side is a little more realistic. This becomes more important when using named indices in the database, especially with larger datasets.

That said, we should consider a lighter weight ordering syntax, that can match 1-to-1 with any indexed sets you want to provide.

Here's an example syntax of what's being proposed:

class Article
  include Mongoid::Document
  include Mongoid::Timestamps

  field :title
  field :number_of_comments

  belongs_to :author
end

class ArticlesController < ApplicationController
  include Matterhorn::Ordering

  allow_order 'oldest', :created_at.asc
  allow_order 'recent', :created_at.desc

  # first ordered by number_of_comments, then created_at
  allow_order 'most_comments', :number_of_comments.desc, :created_at.desc

  default_order 'oldest'
end

Controller methods

  • allow_order - creates a named order. In the above example should pass directly to mongoids sorting on collections. Reserve the keyword 'none', in the future we can add order=none to override default_order
  • default_order - specifies which order will be chosen when the order param is not requested

Serialization

Order should be merged into the request env and passed down the chain until it gets interpreted during serialization.

Links

When using ordering, Links should be added for the different named orders. For example, the above controller should add links as such:

GET /articles.json

{
  "orders": {
    "oldest" : "http://example.org/articles?order=oldest",
    "recent" : "http://example.org/articles?order=recent",
  },
  ...

This should allow sorting to be implemented and used alongside ordering in future versions.

Link inclusion at top-level is included regardless if foreign keys exist in `data`

Per: #9

" Each link class should decide if it can be included or not.
Inclusions and links may or may not have a way to display a URI template in the header, so it should be up to the link class to decide the format. In the above example, the links to author and comments only work if the author_id and initial_comments_ids are available in the serialized article. It's not a requirement to have those. "

Does not appear to be working as stated above:

I removed :author_id from Post serializer and the author is still linked in links in top level.

raw_body=
  {"links"=>
    {"author"=>"http://example.org/users/{posts.author_id}",
     "votes"=>"http://example.org/posts/{posts._id}/vote",
     "comments"=>"http://example.org/posts/{posts.}/comments",
     "initial_comments"=>"http://example.org/comments/{posts.initial_comments_ids}"},
   "data"=>
    {"_id"=>"551c4f8268616e0e1f200000",
     "type"=>"post",
     "links"=>
      {"author"=>{"linkage"=>{"id"=>"551c4f8268616e0e1f1f0000", "type"=>"users"}, "related"=>"http://example.org/users/551c4f8268616e0e1f1f0000"},
       "votes"=>{"linkage"=>{"id"=>"551c4f8268616e0e1f200000", "type"=>"votes"}, "related"=>"http://example.org/posts/551c4f8268616e0e1f200000/vote"},
       "comments"=>{"linkage"=>{"id"=>"551c4f8268616e0e1f200000", "type"=>"comments"}, "related"=>"http://example.org/posts/551c4f8268616e0e1f200000/comments"},
       "initial_comments"=>{"linkage"=>{"id"=>"1,2,3", "type"=>"comments"}, "related"=>"http://example.org/comments/1,2,3"},
       "self"=>"http://example.org/posts/551c4f8268616e0e1f200000"},
     "_type"=>"post",
     "body"=>"Et quibusdam ut sit est aut."}},

Remove Top-level relation links will no longer contain model relation links.

see http://jsonapi.org/format/#document-structure-top-level-links

Given the following routes and models:

#./config/routes.rb
resources :articles
resources :authors

#./app/models
class Author
  include Mongoid::Document
end

class Article
  include Mongoid::Document

  belongs_to :author
end

author  = Author.create
article = Article.create author: author

Currently, the API would provide:

GET `http://example.com/articles/1`

{
  "links": {
    "self": "http://example.com/articles/1",
    "author" : "http://example.com/authors/{articles.author_id}"
  },
  "data": {
    "type": "articles",
    "id": "1",
    "links": {
      "author": {
        "related" : "http://example.com/authors/1",
        "linkage" : {
          "type" : "authors",
          "id"   : 1
        }
      }
    }
  }
}

Instead it should provide:

GET `http://example.com/articles/1`

{
  "links": {
    "self": "http://example.com/articles/1"
  },
  "data": {
    "type": "articles",
    "id": "1",
    "links": {
      "author": {
        "related" : "http://example.com/authors/1",
        "linkage" : {
          "type" : "authors",
          "id"   : 1
        }
      }
    }
  }
}

As it states in the docs, "The top-level links object MAY self links, related links, or pagination links". Currently we provide a fourth item, which is the current resources links (provided as url templates). That needs to be removed and the the logic to build the templates should be removed.

Relationship related urls should not keep params when paging

Discovered this while refactoring some tests, see the resource/index_spec, specifically the commented sections.

it "should allow a page param" do
  request_params.merge! offset: "1"
  perform_request!

  expect(data).to provide(ordered_posts[1..-1])
end

But this test fails with the following output:

Failure/Error: expect(data).to provide(ordered_posts[0..0])
       Actual and Expected do not match.
       Actual   [{"id"=>"554abf18426c61146a770000", "links"=>{"author"=>{"linkage"=>{"id"=>"554abf18426c61146a760000", "type"=>"users"}, "related"=>"http://example.org/users/554abf18426c61146a760000?limit=1"}, "vote"=>{"linkage"=>{"post_id"=>"554abf18426c61146a770000", "type"=>"votes"}, "related"=>"http://example.org/posts/554abf18426c61146a770000/vote?limit=1"}, "comments"=>{"linkage"=>{"post_id"=>"554abf18426c61146a770000", "type"=>"comments"}, "related"=>"http://example.org/posts/554abf18426c61146a770000/comments?limit=1"}, "self"=>"http://example.org/posts/554abf18426c61146a770000?limit=1"}, "type"=>"posts", "author_id"=>"554abf18426c61146a760000", "body"=>"body", "initial_comments_ids"=>nil}]
       Expected [{"id"=>"554abf18426c61146a770000", "links"=>{"author"=>{"linkage"=>{"id"=>"554abf18426c61146a760000", "type"=>"users"}, "related"=>"http://example.org/users/554abf18426c61146a760000"}, "vote"=>{"linkage"=>{"post_id"=>"554abf18426c61146a770000", "type"=>"votes"}, "related"=>"http://example.org/posts/554abf18426c61146a770000/vote"}, "comments"=>{"linkage"=>{"post_id"=>"554abf18426c61146a770000", "type"=>"comments"}, "related"=>"http://example.org/posts/554abf18426c61146a770000/comments"}, "self"=>"http://example.org/posts/554abf18426c61146a770000"}, "type"=>"posts", "author_id"=>"554abf18426c61146a760000", "body"=>"body", "initial_comments_ids"=>nil}]

Params are getting passed into the link building for relationships, which shouldn't happen.

BaseSerializer "type" should be plural

Strictly test "type" field always returns a pluralized name. Currently BaseSerializer returns type as a singular either sometimes or all the time.

Types in links are already pluralized, so we should just ensure that type is pluralized across the board.

link management from the perspective of individual serialized resource

Assuming you have a collection of posts, and your controller lists a private inclusion or link available from the controller:

class PostsController < Matterhorn::Base
  scope = proc do |set_member|
    current_user.votes.all
  end

  add_inclusion :votes, scope: scope
  ...

The responding json (by the json-api spec) should list the votes inclusion. The problem with that is when this resource is serialized by another controller, the underlying resource would have different links. This would break the goal of globally cacheable objects.

We should move the logic to where the current_user object provided at the DSL level from there inclusions could be provided anywhere, model or another class file or confit or sorts. Possibly current_user is method built in rack and exposed as a thread variable.

In controller, something like expose :current_user would expose that variable to the scoped serializer as a configuration. This configuration should not be accessible to all the individual serializers but be usable when constructing the links for each individual object.

Request env

Matterhorn needs a request_env to store it's request logic. Currently there isn't good separation of concerns between the logic for generating the links inclusions and the actual display of those objects.

This is needed prior to finishing the code for inclusions, which currently don't track the request env at all. Inclusions will need to have access to request_env in order to check for authentication visibility... e.g. if you request an inclusion that requires authentication, the inclusion will need to decide how to handle that (raise a 403, ignore request).

This would include:

  • the current_user for the request
  • the primary resource/collection
  • link objects available to the request.
  • inclusions requested.

In your api app, most code bases should setup the following code:

# app/controllers/application_controller.rb
class ApplicationController < Matterhorn::Base
  include AuthenticationCode # this is custom to you're app

  helper_method :current_user
  add_env       :current_user

protected ###################################

  def current_user
    @current_user ||= begin
      User.where(id: session[:user_id]).first
    end
  end

end

Now within requests:

# app/controllers/posts_controller.rb

# matterhorn index methods should handle the index 
# methods for you, this is just to demonstrate that the
#  request_env is available as an instance method.
def index
  request_env[:current_user] # == current_user
end

When the request_env is called it should load each of the methods into a memoized hash. If it is not called, current_user shouldn't be built.

# builder_support
def build_json
  # this should merge controller.request_env into the base object, to be mixed into options for scoped collection/resource.
end

Add support for Fetching relationship status.

see fetching relationships.

A server MUST support fetching relationship data for every relationship URL
provided as a self link as part of a link object.

The docs provide a sample request for a link related.

HTTP/1.1 200 OK
Content-Type: application/vnd.api+json

{
  "links": {
    "self": "/articles/1/links/author",
    "related": "/articles/1/author"
  },
  "data": {
    "type": "people",
    "id": "12"
  }
}

2 strategies worth considering here:

  1. This could be implemented as a single controller that handles link changes across many types of objects.
  2. This would be a set of actions on the specific controller for each resource.

I tend to think the former is probably better, as the controllers won't really need to be overloaded that much. But one concern is validations.

Validations will come from some model and need to be provided through some kind of controller, but the challenge is making those controller actions for validations work in all cases or making those controllers be a first class citizen (generating a file in /app/controllers/ for links management).

It's possible you could do both, by generating a base links management controller, and then in each sub controller provide an interface for customizing those controllers. Something like:

class ArticleLinksController < Matterhorn::LinksController

  def show
    # ... something custom
    super
  end
end

class ArticlesController < Matterorn::Base
  manages_links_with ArticlesLinksController
end

Lastly, routing for this will require some extra attention. I'm not really sure what would be best.

All models require Matterhorn::Inclusions::InclusionSupport

Is this a firm requirement as part of Gem or should we make the gem a little smarter. Presently, nested resources throw an exception when Matterhorn::Inclusions::InclusionSupport is excluded.

1) multi ids resources with_request "GET /#{collection_name}/:id1,:id4/:nested_collection.json" should use multi-ids to scope relationship
     Failure/Error: perform_request!
     NoMethodError:
       undefined method `__inclusion_configs' for #<Mongoid::Criteria:0x007fc328166668>
     # ./lib/matterhorn/serialization/scoped.rb:68:in `merge_inclusions!'
     # ./lib/matterhorn/serialization/scoped.rb:43:in `block in serializable_hash'
     # ./lib/matterhorn/serialization/scoped.rb:42:in `tap'
     # ./lib/matterhorn/serialization/scoped.rb:42:in `serializable_hash'
     # ./lib/matterhorn/serialization/scoped_collection_serializer.rb:9:in `serializable_hash'
     # ./lib/matterhorn/serialization/scoped.rb:48:in `as_json'
     # ./lib/matterhorn/responder.rb:12:in `default_render'
     # ./lib/matterhorn/resources.rb:111:in `block in index'
     # ./lib/matterhorn/resources.rb:213:in `with_scope'
     # ./lib/matterhorn/resources.rb:110:in `index'
     # ./spec/resources_api/multi_id_resources_spec.rb:83:in `block (3 levels) in <top (required)>'

Add basics for implementing app in README/Wiki

We should extract the basic steps to setup an app using Gem and using the basic matchers to test implementing app.

Example:

Add to your Gemfile

gem matterhorn

Generate a basic app based on Rails-api

rails-api new my_json_api_app

Create Matterhorn Controller

class PostsController < Matterhorn::Base
   include Matterhorn::Resources

And so forth

Pagination

See the jsonapi paging spec first. Here's what I'm thinking:

class Article
  include Mongoid::Document
  include Mongoid::Timestamps
end

class Pagination
  def initialize(resource, request_env, page_params, options)
  end

  def next_page_url ; end
  def prev_page_url ; end
  def first_page_url ; end
  # ...
end

class ArticlesController < ApplicationController
  include Matterhorn::Ordering
  include Matterhorn::Paging

  paginates_with Pagination
  pagination_params :page

  default_order :oldest
end

The above controller should add links as such:

GET /articles.json

{
  "links": {
    "self"  : "http://example.org/articles?page=2",
    "next"  : "http://example.org/articles?page=3",
    "prev"  : "http://example.org/articles?page=1",
    "first" : "http://example.org/articles",
  },
  ...

add LinkSet

The codebase at this time has an InclusionSet object which is used to handle the construction of both the global links header, and additionally the included objects.

To better support serialization, we should create a LinkSet which should house a collection of link objects. There should be a few different types of link objects initially:

  • self
  • has_many/has_one
  • belongs_to one/many

Each of these will have different rules for how they are built into links, so each set member will probably have a serializer class for each. Let's talk about an example.

Let's assume you have a a list of articles where each has an author and each user has a vote. Votes are provided as a singleton resource, available as a nested singleton object under each article. Also you want to provide a few comments in the API, let's say the first 3. Let's call these 'initial_comments'.

So, there are 3 different associations available:

  • Author - belongs_to one
  • Vote - has_one
  • Initial Comments - belongs_to_many

note: the vote is a has_one from the API perspective, but internally it will most likely go through a has_many :votes association in the model.

Here's how the json should look:

GET /articles/1.json

{
  "links": {
    "self": "http://example.org/articles/1",
    "initial_comments" : "http://example.org/articles/{article.initial_comment_ids}",
    "vote" : "http://example.org/articles/{article.id}/vote",
    "author" : "http://example.org/people/{article.author_id}",
  },
  "article" : {
    "type": "articles",
    "id": "1",
    "author_id" : "9",
    "initial_comment_ids" : ["2, 3"],
    "links": {
      "self": "http://example.org/articles/1",
      "author": {
        "related": "http://example.org/people/9",
        "linkage": { "type": "people", "id": "9" }
      },
      "vote": {
        "related": "http://example.org/articles/1/vote",
        "linkage": { "type": "vote", "article_id": "1" }
      },
      "initial_comments": {
        "related": "http://example.org/comments/2,3",
        "linkage": [
          { "type": "comments", "id": "2" },
          { "type": "comments", "id": "3" }
        ]
      }
    }
  }
}

Requirements:

  • Each link class should decide if it can be included or not.
  • Inclusions and links may or may not have a way to display a URI template in the header, so it should be up to the link class to decide the format. In the above example, the links to author and comments only work if the author_id and initial_comments_ids are available in the serialized article. It's not a requirement to have those.

This is blocking: pagination and inclusion request action handling

Paginating links and ordering

Add self, next, prev, first links to our current pagination interface,

if there is no next page, next should be null, same for prev.

GET /articles.json

{
  "links": {
    "self"  : "http://example.org/articles?page=2",
    "next"  : "http://example.org/articles?page=3",
    "prev"  : "http://example.org/articles?page=1",
    "first" : "http://example.org/articles",
  },
  "data" : "..."
}

For endpoints that support ordering, links should be added for the different named orders. For example:

    GET /articles.json

    {
      "orders": {
        "oldest" : "http://example.org/articles?order=oldest",
        "recent" : "http://example.org/articles?order=recent",
      }

Add support for `belongs_to_many` relations as links

Given you have:

#./config/routes.rb
resources :articles
resources :tags

#./app/models
class Tag
  include Mongoid::Document
end

class Article
  include Mongoid::Document

  # see http://mongoid.org/en/mongoid/docs/relations.html#has_and_belongs_to_many 
  # more details.  The `inverse_of: nil` should not be required, it's just 
  # to demonstrate that this association doesn't need to be dual sided to work 
  # inside of matterhorn.
  has_and_belongs_to_many :tags, inverse_of: nil
end

tag = Tag.create
article = Article.create tags: [tag]

Let's assume you have an article:

GET `http://example.com/articles/1`

{
  "links": {
    "self": "http://example.com/articles/1"
  },
  "data": {
    "type": "articles",
    "id": "1",
    "links": {
      "tags": {
        "related" : "http://example.com/articles/1/tags",
        "linkage" : [
          {
            "type" : "tags",
            "id"   : 1
          }
        ]
      }
    }
  }
}

Belongs to many should require a nested route in order to display a related link.

Empty belongs to many relationships must provide linkage as an empty array "[]".

Request Specs/Implementation - Creating resources

Using JSON API Spec as guide from sections http://jsonapi.org/format/#crud-creating to http://jsonapi.org/format/#crud-creating-responses-other

  • - POST /posts
  • - Nested resource creation
POST /posts
{
    "title": "Ember Hamster",
    "body" : "blah blah 111",
      "author": 
        { "id": "9" }
      }
    }
}
  • - The response MUST include a Location header identifying the location of the newly created resource. - just return self.link. Timebox on multi-create and see what happens.
  • Errors http://jsonapi.org/format/#errors & Matterhorn errors as specified athttps://github.com/blakechambers/matterhorn/blob/master/ROADMAP.md

The request env should not be provided to base_serializer instances

Currently, the request env is propagated to all serializers by default, which allows access to the url builder, request params, paging, etc. While this is helpful, the concern is that the request env contains params and logic that should not be able to modify a models json representation, e.g. prevent the model from being cached efficiently.

Caching on items inside of data or includes should not be modified by the request env. Scoped Collection and Scoped resource serializers... will still need access to those variables to produce the correct links for things like paging, ordering, filters, etc.

with_request doesn't have access to it let variables

See: https://github.com/blasterpal/matterhorn/blob/show_spec/spec/resources_api/show_spec.rb#L23

Exception

hbeaver~/Dropbox/blasterpal/matterhorn (show_spec)$ be rspec spec/resources_api/show_spec.rb:24
/Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/example_group.rb:634:in `method_missing': undefined local variable or method `resource_id' for RSpec::ExampleGroups::Show:Class (NameError)
        from /Users/hbeaver/Dropbox/blasterpal/matterhorn/spec/resources_api/show_spec.rb:14:in `block in <top (required)>'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/example_group.rb:363:in `module_exec'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/example_group.rb:363:in `subclass'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/example_group.rb:253:in `block in define_example_group_method'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/dsl.rb:43:in `block in expose_example_group_alias'
        from /Users/hbeaver/Dropbox/blasterpal/matterhorn/spec/resources_api/show_spec.rb:3:in `<top (required)>'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/configuration.rb:1226:in `load'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/configuration.rb:1226:in `block in load_spec_files'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/configuration.rb:1224:in `each'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/configuration.rb:1224:in `load_spec_files'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/runner.rb:97:in `setup'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/runner.rb:85:in `run'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/runner.rb:70:in `run'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/lib/rspec/core/runner.rb:38:in `invoke'
        from /Users/hbeaver/.rbenv/versions/2.1.0/lib/ruby/gems/2.1.0/gems/rspec-core-3.2.2/exe/rspec:4:in `<top (required)>'
        from /Users/hbeaver/.rbenv/versions/2.1.0/bin/rspec:23:in `load'
        from /Users/hbeaver/.rbenv/versions/2.1.0/bin/rspec:23:in `<main>'

Add inherited_resources dependency for url / nested resource helpers

See https://github.com/josevalim/inherited_resources/blob/9b1386f0effe9510e132915fa329f353f35db41c/app/controllers/inherited_resources/base.rb#L15-L37

While it might make sense to define some of this logic here, this would be great to use for the initial implementation. Currently, the API constructor doesn't allow for nested params.

We should start by adding a method that sets up most of this functionality on our module action methods.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.