lotas/contentful-graph

Visual representation of contentful content models in form of graphs

contentful-graph

Visual representation of contentful content models in form of graphs

Can be exported with fields:

or without:

Requirements

graphviz

If you plan to render the graph as image or PDF, make sure you have graphviz installed on your system.

node.js

The library is using async/await functions, so you need the latest 7 or 8 versions.

Installation

Can be installed globally:

npm i -g contentful-graph

Or locally

npm i --save-dev contentful-graph

Running without installation

Import script can be executed with the help of npx

npx contentful-graph

Running import

There are several ways of running the import:

1. running CLI bin/contentful-graph command
2. import API and invoking functions programmatically

For this purpose, you'll need to know your contentful spaceId and token

Token may come in two flavors: Delivery token or Management token

Management token has an advantage of showing one-to-one relationships for models (as pointed by @jelz in #1)

Command-line version

When running the command line version, make sure you export following variables:

CONTENTFUL_SPACE_ID= spaceId
CONTENTFUL_TOKEN= either deliveryToken
CONTENTFUL_MANAGEMENT_TOKEN= or management token
CONTENTFUL_ENVIRONMENT_ID= environment id

or simply in command-line:

CONTENTFUL_ENVIRONMENT_ID=master CONTENTFUL_SPACE_ID=123 CONTENTFUL_MANAGEMENT_TOKEN=token ./bin/contentful-graph

Command-line options

--help (-h) Displays usage information

--dev (-d) Includes developer information (model Id's and field Id's)

--hide-fields (-n) Do not include fields information, only entities

Reading from local file

It is possible to run import against local file with exported model defintions:

./bin/contentful-graph ./my-contentful-model.json

Web version

You can run it locally or host it on any server using contentful-graph-web

docker run -it --rm -p 3000:3000 lotas/contentful-graph-web
# or checkout project locally and
make BUILD && make RUN
# or if you have graphviz in your system simply
npm run dev

Api version

Package exposes following functions:

getContentTypesFromManagementApi(spaceId: string, managementToken: string, environment: string): Promise<Object>

Will import content types from contentful using management api

getContentTypesFromDistributionApi(spaceId: string, apiToken: string, environment: string): Promise<Object>

Will import content types from contentful using distributions api

contentTypesToModelMap(contentTypes: Object): Object

Will enrich existing content types with mapping information (one-to-one, one-to-many)

modelsMapToDot(modelsMap: Object, {hideEntityFields: Boolean, dev: Boolean}): String

Will create dot graph definition out of the model map

You can use them as follows:

const convertApi = require('contentful-graph');

// either with managementToken
const contentTypes = await convertApi.getContentTypesFromManagementApi(spaceId, managementToken);
// or with apiToken, you just need one of the calls
const contentTypes = await convertApi.getContentTypesFromDistributionApi(spaceId, apiToken);

// enrich content types with relationship definitions
const modelsMap = convertApi.contentTypesToModelMap(contentTypes);

// generate dot string that can be passed to graphviz
const dotStr = convertApi.modelsMapToDot(modelsMap, {});

Generating graphs

After you would run the import, you should have something like this:

digraph obj {
  node[shape=record];

  Image [label="{Image |        | <title> Title|<photo> Photo|<imageCaption> Image caption|<imageCredits> Image credits|<categoryId> CategoryId}" shape=Mrecord];
  Asset
  Author [label="{Author |        | <name> Name|<twitterHandle> Twitter handle|<profilePhoto> Profile photo|<biography> Biography|<createdEntries> Created entries}" shape=Mrecord];
  Category [label="{Category |        | <categoryId> Category Id|<categoryName> Category name}" shape=Mrecord];
  PhotoGallery [label="{Photo Gallery |        | <title> Title|<slug> Slug|<author> Author|<coverImage> Cover Image|<description> Description|<images> Images|<tags> Tags|<date> Date|<location> Location}" shape=Mrecord];

  Image:photo -> Asset [dir=forward];
  Image:categoryId -> Category [dir=forward];
  Author:profilePhoto -> Asset [dir=forward];
  Author:createdEntries -> Image [dir=forward,label="1..*"];
  PhotoGallery:author -> Author [dir=forward];
  PhotoGallery:coverImage -> Asset [dir=forward];
  PhotoGallery:images -> Image [dir=forward,label="1..*"];
}

To render this to image or PDF, pipe it to the dot command:

./import.js | dot -Tpng > model.png
./import.js | dot -Tpdf > model.pdf

Where ./import is the script that produces dot output

Links

Contentful Management API

Contentful Delivery API

GraphViz Fiddle

contentful-graph's Issues

space.getEnvironment is not a function error

Hey lotas,

I'm having a Contentful user running into the attached error after following the instructions on the plugin page
space.GetEnvironment is not a function.

Thank you!

Update: Was able to resolve

Doesn't map out all relations

Not sure how I should capture a good repro for this, but I noticed that many but not all relations are mapped out by this utility. Some of our content types just end up without relationships with any other content-types (but they for sure have relationships with other types).

How do you suggest we proceed?

How can I give a better repro without sharing the entire data for the contentful space?

throws TypeError if multiple reference field contains validation for non-existent content type

Consider the example of a content type which has a multiple reference field and that field has a validation to only accept entries of certain content types with the ids: series, season, video. The model has since changed and series has been deleted and replaced with show. The validation on that multiple reference field is now invalid.

When I use this tool in the above mentioned situation I get

(node:31538) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): TypeError: Cannot read property 'name' of undefined
(node:31538) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

I think this tool could deal with that invalid reference by either labeling it as invalid (with the benefit of alerting the user to this error) or by excluding it quietly.

Happy to provide a PR with a fix.

Update dependencies to "latest" version

Almost all dependencies are outdated.

I would also upgrade to the supported version of Node.JS.

Supported minimal version of node would be 10.*
Supported minimal version of npm would be 6.14.*

General issues

Hi there,

Jakub from Contentful here. Thank you for contributing to our open source ecosystem. There are some issues with https://github.com/lotas/contentful-graph/blob/master/src/get-relations.js. Allow me to list them to help you with making your project better:

1. You check only for field.items.validations. It works only with multiple references fields (array of links). For singular links you'd have to check field.validations.
2. There is a problem with field.validations: they are not available in CDA (contentful npm module). You'd have to use CMA (contentful-management npm module) response for the /content_types endpoint call.
3. You always map over linkContentType validation (validation.linkContentType.map). It can fail for some other validations (e.g. array size validation) where linkContentType is undefined.
4. The other thing is that linkContentType can be either an array or a string. If it's a string, it's an ID of a singular linked content type.

Please let me know if there's something I could help with.

J.

Make it an npm module

Heyo. 👋

I'm really digging this module. Do you have plans to make it a proper npm module?

What do the different colors mean?

Thanks for this great tool.

I am new to contentful and I am not sure what the colors mean in the graph generated by contentful-graph.

What do the red, green, and yellow edges represent?

Thanks

Use content type id's as dot node id's and escape labels

Hey there,

great tool, we love it here at Contentful and use it a lot internally :)

I discovered a few problems with more complex data models, just want to list them here, hoping to find time to file a pull request on my own.

The actual problem:

When having complex content type names with special chars like Layout > Image, the id of the dot-node or the label might be invalid:

digraph obj {
  "Layout > Image" [label="{Layout > Image ... }"];
  "Layout > Text" [label="{Layout > Text ... }"];

  "Layout > Image" -> "Layout > Text"
}

(This actually works but in more complex tests, this broke the graphviz rendering)

I'd love to see output like this, it might also be more readable:

digraph obj {
  layout_image [label="{Layout \> Image ... }"];
  layout_text [label="{Layout \> Text ... }"];

  layout_image -> layout_text
}

1. Instead of content type name I took the content type id as dot node id
2. I escaped the label text to avoid problems with special chars.

Thanks a lot,
Benedikt

Output to diagrams.net

It would be super useful to support the file format of diagrams.net, which would allow editing the graphs after exporting them from Contentful.

Hyphens and spaces in element names lead to syntax error with dot

I'm getting this error message
Error: <stdin>: syntax error in line 31 near '-'
when running
CONTENTFUL_SPACE_ID=OMITTED CONTENTFUL_MANAGEMENT_TOKEN=OMITTED node_modules/contentful-graph/import.js | dot -Tpdf > model.pdf

import.js seems to run without issues, line #31 in the output looks like this:
UICOPYView:strings -> UI-COPY-String [dir=forward,label="0..*"];

So I'm wondering is the - character in the element name UI-COPY-String causing these issues? Should it be somehow escaped for dot to read correctly?

RichText link does not reflect all configurations #19

The fix introduced in PR #21 for issue #19 does not reflect all possible states.

Missing / not working:

• If there are no restrictions, it just fails / breaks
• If there are no links / assets to entries allowed by disabling linking to any asset / entry, it will assume that everything is allowed

Getting data over distribution api does not work anymore

#25 introduced an issue with the distribution API. The Contentful client changed the behavior.

Output is missing edges for richtext embedded entries

The graph is really useful, thank you very much for this!

We noticed that embedded entry references within richtext fields aren't picked up yet and hence don't become edges in the graph.

By the way, is this repo still active? Would you also accept PRs?

I'm getting status 403

I'm trying to get my schema as usually do. But without updating anything I'm getting a bad token result, it stopped working today

  var error = new Error();
              ^

AccessTokenInvalid: {
  "status": 403,
  "statusText": "Forbidden",
  "message": "The access token you sent could not be found or is invalid.",
  "details": {},
  "request": {
    "url": "/spaces/jq8d65xjulgq",
    "headers": {
      "Accept": "application/json, text/plain, */*",
      "Content-Type": "application/vnd.contentful.management.v1+json",
      "X-Contentful-User-Agent": "sdk contentful-management.js/7.54.2; platform node.js/v16.0.0; os macOS/v16.0.0;",
      "Authorization": "Bearer ...BaWpw",
      "user-agent": "node.js/v16.0.0",
      "Accept-Encoding": "gzip"
    },
    "method": "get"
  },
  "requestId": "d95706be-c9b6-4cdb-9fc9-d7c1f6c91b11"
}

I have my env variables already set up

Could you help me w this?

No 'latest' release

Due to this issue I cannot install pachage like @latest

npm ERR! 404  'contentul-graph@latest' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)