Create GraphQL endpoint, leveraging neo4j graphql library v3

[cncf]

Create GraphQL endpoint, leveraging neo4j graphql library v3

[halcyondude]

(from CNCF slack #landscape-graph channel) https://cloud-native.slack.com/archives/C03BXBYFMQS/p1658122893053639

Apologies for the long response delay. This is more of a link rosetta stone, I'm working on actual docs that explain some of this, but wanted to elaborate on some of the results of the past 4 weeks of research/learning/design.

Start here

• https://principledgraphql.com (in style of 12 factor)
• GraphQL’s SDL language is the source of truth.
• Workshop on GraphQL APIs and neo4j
• https://www.digitalocean.com/community/tutorials/graphql-graphql-sdl

Apollo Federation (v2)

A federated supergraph uses multiple "types" of GraphQL schemas:

---

(source: https://github.com/apollographql/federation/blob/main/docs/source/federated-types/overview.mdx)

graph TB;
  serviceA[Subgraph<br/>schema<br/>A];
  serviceB[Subgraph<br/>schema<br/>B];
  serviceC[Subgraph<br/>schema<br/>C];
  composition[["🛠<br/>Composition "]];
  supergraph{{"Supergraph schema<br/>(A + B + C + routing machinery)"}};
  api(["API schema<br/>(A + B + C)"]);
  serviceA & serviceB & serviceC --> composition;
  composition -- "(Composition succeeds)" --> supergraph;
  supergraph -- "(Remove routing machinery)" --> api;
  class composition tertiary;

• Subgraph schemas. Each subgraph has a distinct schema that indicates which types and fields of your composed supergraph it can resolve.
  • These are the only schemas that your teams define manually.
• Supergraph schema. This schema combines all of the types and fields from your subgraph schemas, plus some federation-specific information that tells your gateway which subgraphs can resolve which fields.
  • This schema is the result of performing composition on your collection of subgraph schemas.
• API schema. This schema is similar to the supergraph schema, but it omits federation-specific types, fields, and directives that are considered "machinery" and are not part of your public API.
  • This is the schema that your gateway exposes to clients, who don't need to know internal implementation details about your graph.

• https://www.apollographql.com/docs/federation/federated-types/overview
• https://www.apollographql.com/docs/federation/federated-types/composition
• https://www.apollographql.com/blog/announcement/backend/announcing-federation-2
• https://www.apollographql.com/docs/federation/
• https://www.apollographql.com/docs/federation/federation-2/new-in-federation-2
• https://github.com/neo4j-graphql/neo4j-graphql-js/blob/dev/docs/apollo-federation.md (note neo4j-graphql-js is no longer a thing, but writeup is good)

Federation 2 Demo App

https://github.com/apollographql/supergraph-demo-fed2

Visualizing and documenting

• Investigating: SDL/GraphQL -> OpenApi: https://github.com/IBM/openapi-to-graphql
• graphql playground w/ neo4j movies db https://movies.neo4j-graphql.com/
• openapi-to-graphql: Translate APIs described by OpenAPI Specifications (OAS) into GraphQL Website
  • https://developer.ibm.com/open/projects/openapi-to-graphql

Visualizing Neo4J Model (Closely Tracks GraphQL…generated from it…)

• https://neo4j.com/product/bloom/
• https://cytoscape.org
• https://neo4j.com/developer/graph-visualization
• https://medium.com/neo4j/visualizing-graphs-in-3d-with-webgl-9adaaff6fe43
• https://github.com/neo4j-contrib/neovis.js
• https://github.com/vasturiano/3d-force-graph

Current state of neo4j + graphql

• https://www.youtube.com/watch?v=sZ-eBznM71M
• Training to match the slides above (they are extensive w/ speaker notes)
  https://www.youtube.com/watch?v=DIFgwgbC-7c
• Slides: https://docs.google.com/presentation/d/1fvEwpvkhSAGXRWcVHPQqNOFgvGT9A8yMZXfyF-gn-Bw/edit?usp=sharing (edited)
• Investigating: https://github.com/apollosolutions/neo4j-subgraph

CI/CD tooling

• schema composition checks as well as generating all the things: https://www.apollographql.com/docs/rover, https://github.com/apollographql/rover

Subscriptions

Apollo Federation 2 doesn’t support subscriptions OOTB, have been checking this out as well: https://github.com/apollosolutions/federation-subscription-tools

[halcyondude]

https://github.com/cncf/landscape-graph/blob/4-graphql-endpoint-v1/db/cncf/cncf.graphql

https://github.com/cncf/landscape-graph/tree/4-graphql-endpoint-v1/db/cncf

@AlexxNica FYI (WIP - but progress all the same :))

[halcyondude]

From what I'm seeing right now, the 4-graphql-endpoint-v1 branch isn't handling any types for the GraphQL schema yet. Do you have any thoughts on how to handle that if using GraphQL as the source of truth?

@AlexxNica re: the branch and graphql types, (a couple weeks ago) a prototype is here:

https://github.com/cncf/landscape-graph/blob/4-graphql-endpoint-v1/db/core/landscape-graph-core-schema.gql

Actively working on something more complete this morning.

I've also found this useful for interactive iteration, in concert with Neo4j Desktop's GraphQL extension..

Domain Graph for VSCode

[halcyondude]

after doing a bit of research, a few things are clear.

• the data model is best described via GraphQL
• data is likely best loaded via GraphQL mutations
• graphql type definitions generated using the Introspector nicely handle constraint and index declarations
• a VERY nice workflow using arrows.app exists.
  • generate data model (as we've done) using arrows.app
  • export as .graphql
  • write the schema --> Neo4j DB using the neo graphql v3 library

Game Changer

https://neo4j.com/docs/graphql-manual/current/type-definitions/interfaces/#_directive_inheritance

Any directives present on an interface or its fields will be "inherited" by any object types implementing it. For example, the type definitions above could be refactored to have the @relationship directive on the actors field in the Production interface instead of on each implementing type as it is currently:

interface Production {
    title: String!
    actors: [Actor!]! @relationship(type: "ACTED_IN", direction: IN, properties: "ActedIn")
}

type Movie implements Production {
    title: String!
    actors: [Actor!]!
    runtime: Int!
}

type Series implements Production {
    title: String!
    actors: [Actor!]!
    episodes: Int!
}

interface ActedIn @relationshipProperties {
    role: String!
}

type Actor {
    name: String!
    actedIn: [Production!]! @relationship(type: "ACTED_IN", direction: OUT, properties: "ActedIn")
}

https://neo4j.com/docs/graphql-manual/current/type-definitions/interfaces/#_overriding

In addition to inheritance, directives can be overridden on a per-implementation basis. Say you had an interface defining some Content, with some basic authorization rules:

interface Content
    @auth(rules: [{ operations: [CREATE, UPDATE, DELETE], allow: { author: { username: "$jwt.sub" } } }]) {
    title: String!
    author: [Author!]! @relationship(type: "HAS_CONTENT", direction: IN)
}

type User {
    username: String!
    content: [Content!]! @relationship(type: "HAS_CONTENT", direction: OUT)
}

type PublicContent implements Content {
    title: String!
    author: [Author!]!
}

type PrivateContent implements Content
    @auth(rules: [{ operations: [CREATE, READ, UPDATE, DELETE], allow: { author: { username: "$jwt.sub" } } }]) {
    title: String!
    author: [Author!]!
}

[AlexxNica]

As we get back on track, here are some notes I had about finding the best way of handling the data model<->graphql<->neo4j workflow:

From what I'm seeing right now, the 4-graphql-endpoint-v1 branch isn't handling any types for the GraphQL schema yet. Do you have any thoughts on how to handle that if using GraphQL as the source of truth?

These are the methods I've come so far after some research:

Cypher as the source of truth

This method changes the flow to be Cypher->Neo4j->GraphQL.

Steps:

1. Use Cypher queries within .cypher files (so we can handle version control, CI/CD) as the actual source of truth.
2. Load that into the database.
3. Introspect and generate the GraphQL schema (check if the schema already exists and only generate if changes were made to the types (otherwise we can be generating schemas way too often–possibly extra load).
4. Generate TypeScript definitions based on the generated schema.
5. We're ready to consume the GraphQL schema and the TypeScript definitions.

Possible drawbacks:

• Is Neo4j capable of generating reliable and ready-to-use GraphQL schemas? (example from introspector limitations)
  • Can be remediated: Do some processing on the generated schema to handle edge cases.
• We don't have a visual representation of the schema.
  • Can be remediated: The visual could be generated by another tool based on the Neo4j DB, GraphQL schema, or the TypeScript definitions.

GraphQL as the source of truth

This method changes the flow to be GraphQL->Cypher->Neo4j. (The Cypher in this case is automatically generated by the Neo4j GraphQL Library.)

Schema:

1. Write the GraphQL schema (either manually or through apps like arrows.app (see note about arrows.app below))
2. Use the Neo4j GraphQL Library to integrate the schema through Neo4j and a GraphQL server.
3. Generate TypeScript definitions based on the schema.
4. We're ready to consume the GraphQL schema and the TypeScript definitions.

Data loading:

1. Write mutations directly to the GraphQL server.

Data processing:

• Use GraphQL custom resolvers.
• Use any other method before the mutation/query.

Visual representation

It might be worth exploring Mermaid as an option to generate visual representations of the schema. GitHub now supports it natively within markdown instances (issues, documents, etc.).

Note about arrows.app

After some research and testing, I've found arrows.app to be unreliable (some bugs caused the graph to lose its current status and saved outdated versions on top of new ones) and hard to use in a collaborative setting (not having integrations or clear ways of openly collaborating).

More context: neo4j-labs/arrows.app#55 (comment)

Example using Mermaid:

flowchart TB
    subgraph Person
    Person_id["id: ID!"]
    Person_name["name: String!"]
    Person_organization["organization: String!"]
    Person_location["location: [Location!]!"]
    end
    subgraph Role
    Role_id["id: ID!"]
    Role_name["name: String!"]
    end
    subgraph Organization
    Organization_id["id: ID!"]
    Organization_name["name: String!"]
    Organization_headquarters["headquarters: [City!]!"]
    end
    subgraph Location
    Location_id["id: ID!"]
    Location_name["name: String!"]
    end
    subgraph GitRepo
    GitRepo_id["id: ID!"]
    GitRepo_name["name: String!"]
    end
    subgraph TocRole
    TocRole_id["id: ID!"]
    TocRole_name["name: String!"]
    end
    subgraph Project
    Project_id["id: ID!"]
    Project_name["name: String!"]
    end
    subgraph ProjectRole
    ProjectRole_id["id: ID!"]
    ProjectRole_name["name: String!"]
    end
    subgraph TAGrp
    TAGrp_id["id: ID!"]
    TAGrp_name["name: String!"]
    end
    subgraph TagRole
    TagRole_id["id: ID!"]
    TagRole_name["name: String!"]
    end
    subgraph City
    City_id["id: ID!"]
    City_name["name: String!"]
    end
    Person_location -->|LOCATED_IN| Location
    Person -->|MAINTAINER_OF| GitRepo
    Person -->|HAS_TOC_ROLE| TocRole
    Person -->|HAS_ROLE| ProjectRole
    ProjectRole -->|SERVED| Project
    Project -->|IN_SCOPE| TAGrp
    TagRole -->|SERVED| TAGrp
    Organization_headquarters -->|HQ_IN| City
    %% Temporary fix to represent bidirectional linking. It seems GitHub doesn't support Mermaid's native bidirectional linking yet.
    Organization <--> node((IS_SUB)) <--> Organization
    Organization -->|EMPLOYED| Person
    Person -->|IS_BOARD| Organization

Edit: forgot to add the source for the Mermaid example. Here it is: https://github.com/nikas-org/collab-coordination/blob/d795485e6c54c895838a663bbfb55b22cbd9e1b2/examples/landscape-graph_mermaid.md

[halcyondude]

At the risk of "marketing," pictures > words sometimes :) Hope this helps!