Custom queries and mutations using custom resolvers

[prisma]

Custom queries and mutations using custom resolvers

[cvburgess]

Things we would love to see if we wanted to actually use this for a project:

• Ability to integrate with external API and parse the JSON response
• Ability to integrate with external database

We have a lot of existing RESTful services (some ours, some of partners) that we cannot migrate. Being able to hook into these would make this incredibly useful, allowing us to "transform" older services into shiny new ones that are all linked together.

Like @Thomas-Kuipers said, I think just exposing the ability to connect to a database with a qualified URL would be fantastic and allow for those who have existing data or have a complex setup to get started quickly without trying to migrate GBs or PBs of data.

[Thomas-Kuipers]

We would like to use GraphCool as a proxy to our existing external MySQL database. We have no possibility to migrate our entire database to GraphCool, but we would like to use GraphCool's awesome capabilities to query our own data, without needing to write our own GraphQL server. Ideally, GraphCool would just need a db host and username, and based on the table relations it would be able to figure everything out itself.

[TheoMer]

The ability to incrementBy/decrementBy (x) a numeric field value, for example the likes value on a Post, without having to write two queries to;

1. ascertain the current value, then
2. update the field, after some extra prior client side manipulation

would be nice

[migueloller]

Yet another case for custom queries (and access to the database) is to be able to run a custom algorithm on the data and expose it via a query. For example, if you have a newsfeed that depends on a non-trivial ranking process (more than just filtering) it's not possible to implement with Graphcool right now.

[kabriel]

+1 really need this so we can integrate with our other backend REST APIs. Like others have said, we want one endpoint to Graph.cool and a way to integrate that into our existing services. This would be a huge win!

[sedubois]

Yes custom extension of the GraphQL schema would be awesome 👍

I'm taking care of a legacy app (REST API with underlying Rails code) and it will take a while before the whole thing can be revamped with GraphQL. So a migration path would be needed where the code migration and the data migration can be decoupled.

[onlymejosh]

I'd like to be able to do aggregations on the data essentially what I can achieve in elastic search.

Say I have the following:

Vinyl {
  genre:string
  artists {
    name:string
  }
}

Id like to be able to groupBy genre with a count so I can display it in a facet / search result.

It would return something like:

{
  "data": {
    "allVinyls": [
      {
        "genres": [
          {
            "name": "TIMELINK",
            "count": 10
          },
          {
            "name": "SLOW LIFE",
            "count": 2
          },

        ]
      }
    ]
  }
}

[marktani]

another use case for custom queries:

• highly indivual ordering of queries

[sorenbs]

A lot of good use cases in this thread - thanks everyone!

A few notes:

@migueloller In the case of a complex newsfeed you might consider adopting a fan out approach as popularised by twitter and described in http://highscalability.com/blog/2013/7/8/the-architecture-twitter-uses-to-deal-with-150m-active-users.html

This guarantees stable read performance and can be easily modelled on Graphcool with two models:

1. incoming event. A mutation callback is responsible for writing this to all relevant timelines
2. timeline event. All timeline queries go to this model. In the trivial implementation you simply query chronologically, but you can be smart about precomputing some values that the timeline app can the use in the query

In general @kabriel`s description in https://github.com/graphcool/feature-requests/issues/40#issuecomment-283770482 aligns very well with our vision. We can't commit to a timeline yet, but this is something we are actively working on.

[thenikso]

Yep, this one would allow Graph.cool to act as an "API gateway" so that one could have a type not backed by the graph.cool automatic database thing but by a function instead.

The type could look like (code might be inaccurate):

type MyLegacyApi implements FunctionNode {
  id: ID!
  somethingImportant: String!
}

and the 2 function backing it (one for single element and another for list):

// myLegacyApi get
module.exports = function(params) {
  return fetch('https://mylegacyapi.com/api?id=' + params.id); // id sould already be de-relayed?
}

// allMyLegacyApi get
module.exports = function(params) {
  // Utilities to generate relay like cursors should be provided in a library?
  return fetch('https://mylegacyapi.com/api' + paramsToQueryString(params));
}

// plus mutations and whatnot

Am I making sense?

NOTE: I think having a "context" that easily allow me to access ie: current user, convenience graph.cool graphql querying method ect like in #219 would be best in those functions

[kbrandwijk]

If those two features are connected, I'm worried about the 'getting there' part...

[thenikso]

Configuring an event trigger with a hook to "read" in a request pipeline function might just do the trick.

Also, maybe marking fields in a node that should not be stored in the db but provided in a function could be an idea. Even specifying a specific function for a field could cover both partial and complete type backing.

type MyNode implements Node {
  id: ID!
  name: String
  legacyField: String! @function
  onlyCallingFunctionIfRequested: String @function('myFunction')
}

[sorenbs]

I think even in that case it's a pretty good approach. We will at some point build a scheduling mechanism into the Graphcool platform, but for now it would be fine to use something external. If your are comfortable with aws I would suggest a periodic lambda function that does the work. If you have a lot of work to do, you can have the periodic function enqueue a bunch of work items to kinesis.

I realise this is more work than writing a complex sql query that handles all the ranking though.

[kurt-o-sys]

👍
agree! I'd like to have a maximum distance from a certain point as filter...

[kbrandwijk]

Also, keep in mind that a GraphQL endpoint is usually geared towards a specific front-end, and bringing together all back-ends/microservices/datasources in one endpoint. For that reasons, queries should be as specific as necessary for the frontend, and tailored to it.
Auto-generating very generic queries is more similar to the traditional REST API style, where you have a very generic endpoint for a very specific source, exactly the other way around.

One of the keys to allowing custom queries and mutations is to be able to specify queries on every Type, and being able to enable/disable the auto-generated ones too. For example, if I have a Type Post and a Type PostItems, and I only want to retrieve Postitem nodes as children of a Post, I wouldn't want or need the top-level query allPostItems(...) in my schema, so I would like to disable that one.

The other use case is that my front-end should be as thin as possible, so I don't want to construct queries with ordering and filtering and specific fields. For this, being able to define persisted queries on a Type would be great: https://dev-blog.apollodata.com/persisted-graphql-queries-with-apollo-client-119fd7e6bba5

[thekevinbrown]

@kbrandwijk Indeed, I'll post it over there, thanks!

[marktani]

What you can do at the moment is to define a new Genre model with a key field of type unique enum or string. Then you create new genres, in this cases two genres with unique key "TIMELINK" and "SLOW LIFE" (strings) or TIMELINK and SLOW_LIFE (enums).

You define a many-to-many relation between Vinyl and Genre with fields vinyls and genres.

Then you can query all genre counts with this query:

query {
  allGenres {
    _vinylsMeta {
     count
    }
    key
  }
}

{
  "data": {
    "allGenres": [
      {
        "_vinylsMeta": {
          "count": 2
        },
        "key": "SLOW_LIFE"
      },
      {
        "_vinylsMeta": {
          "count": 10
        },
        "key": "TIMELINK"
      }
    ]
  }
}

or by using aliases:

query {
  slowlife: allGenres(filter: {key: SLOW_LIFE}) {
    _vinylsMeta {
			count
    }
    key
  }
  
  timelink: allGenres(filter: {key: TIMELINK}) {
    _vinylsMeta {
			count
    }
    key
  }
}

{
  "data": {
    "slowlife": [
      {
        "_vinylsMeta": {
          "count": 2
        },
        "key": "SLOW_LIFE"
      }
    ],
    "timelink": [
      {
        "_vinylsMeta": {
          "count": 10
        },
        "key": "TIMELINK"
      }
    ]
  }
}

There probably still needs to be some work done on the client-side to accomplish what you want, but until custom queries become a thing that should get the job done. Hope that helps! :)

[marktani]

Also see #79

[marktani]

A nice addition for these custom operations is the possibility to include information of the currently logged in user as input arguments.

[sedubois]

Regarding my last comment, actually, it might be more feasible to restart from the Postgres DB layer directly, as others mentioned. I use a Postgresql DB so I guess something like Postgraphql could be used to introspect the schema.

At any rate, I think this is quite an important feature also from Graphcool's point of view so that you can attract bigger existing customers. It's not very feasible to require of a production system to get rewritten from scratch at once. A more sensible approach is to do decouple the code and data migration.

I'm planning to start working on this now, and would love to do so using Graphcool. Would you have a status update? 🙂

[migueloller]

@sorenbs, interesting approach. I had read this article previously but didn't think how it could be applied with a custom type system. Based on your recommendation, where would custom computations happen? In the mutation callback of the incoming event? Perhaps with a microservice that runs the necessary computation and creates a timeline event with the required fields needed to do a timeline query?

[migueloller]

Our use case is a little bit different from Twitter because we don't have a specific event (a tweet) that triggers a feed update. For us, the feed simply updates every day (every day there are new restaurants and events to recommend) so we would have to simulate that some how with a custom mutation in Graphcool that we have a worker call every day. This sounds a bit hacky, though.

[philcockfield]

+1 👍 - this would be super powerful.

[marktani]

Yet another great use case:

custom queries can be used to avoid deeply nested json responses and reduce query complexity for nested structures. Instead of querying for this.userInfo.contact.company.seller.id you could define a specific query that combines company and seller. Basically it's a transformmation the output json.

See also this related proposal for the GraphQL spec: graphql/graphql-spec#249

[kbrandwijk]

+1

[jhalborg]

Would love this feature, it would make Graphcool viable for a lot of project in the consultancy I work at, masking obsolete APIs and databases to the client apps moving forward.

Also, a the specific use case that brought me to this issue is integration of a weather API in my React Native app. As of now, I will have to interact with it directly, leading to the app communicating with two APIs instead of one - and also preventing me from using GraphQL and Relay, which makes me sad :-/ ( ;-) )

[kbrandwijk]

@thenikso @jhalborg I think this FR was to extend existing types with custom queries and mutations. Creating Types connected to different datasources is a different feature alltogether in my opinion.

[marktani]

I think the concepts are similar enough that it makes sense to discuss them in this FR. Thanks @thenikso, that looks like a great approach.

[kbrandwijk]

An event pipeline for queries sounds great!

[perrosnk]

+1 That would be awesome

[dberringer]

Is there a targeted release date for the ability to use graphcool to wrap rest apis yet? Next week / Next Month / Next year? As soon as I can do this I'm in. I'm on the edge of my seat. Thanks.

[marktani]

Hey @dberringer, with the new feature Schema Extensions, wrapping REST APIs is already possible. Feel free to join the discussion in the forum to bring up your use case and join the beta 🙂

[dberringer]

Even better. Thanks @marktani!

[thekevinbrown]

Another use case that either I'm blind and didn't see, or would be really helpful is to be able to effectively create views over data.

We have 3 tables, all which have different data, but in a particular screen, they need to be sorted such that they need to be queries and paginated together.

So an example:

type Announcement implements Node {
...etc...
  title: String!
  associatedDate: DateTime!
}

type Event implements Node {
...etc...
  title: String!
  associatedDate: DateTime!
}

type Conversation implements Node {
...etc...
  name: String!
  responseRequiredBy: DateTime!
}

In SQL we could just union the things and give them a unified structure, then ORDER BY whatever we called the date filed. In graph.cool right now we've created another record that relates to all 3 so we can execute this query, but it'd be much nicer to construct this with a view rather than by needing an actual new node that denormalises the data in order to feed what shows in one screen while the rest of the nodes are actually quite distinct.

[kbrandwijk]

@blargity That seems to be actually more related to https://github.com/graphcool/feature-requests/issues/165.

[marktani]

Resolver functions available as of now: https://blog.graph.cool/extend-your-graphcool-api-with-resolvers-ca0f0270bca7 🎉

Thanks a lot for everyone who helped testing it out and provided feedback along the way 🙏