commercetools/commercetools-docs-kit

Tools and components for developing Documentation websites 🛠

commercetools-docs-kit's Issues

Writing Style: Override Date Formatting Rule

When writing Dates, we prefer an "international long format" like "2020 January 20" over the USA convention coded into the Google Style Guide we base on.

Additionally, formats with the Months spelled out should always be preferred over numeric-only formats to make sure the date is free of any ambiguities. So writing "December 12th 2012" and writing "2012-12-12", "2012 / 12 / 12" , "12.12.2012", "12.12.12" should all be warnings. The warning text should be separate, so we need two separate Rules

• Prefer spelling out the month in dates
• Prefer international date format

(Time not handled here, that's a separate decision)

add google analytics plugin and make the id configurable per site

default should be an empty or no id configured in a way that does not track but does not throw errors either.

Investigate analyzing bundle size

https://www.gatsbyjs.org/packages/gatsby-plugin-webpack-bundle-analyser-v2/?=bundle
https://github.com/siddharthkp/bundlesize

Setup Sentry and React boundaries

https://reactjs.org/docs/error-boundaries.html
https://docs.sentry.io/error-reporting/quickstart/?platform=browsernpm

Investigate how to dedupe emotion styles

I noticed that in the production build some styles are "duplicated" all over the page, for example for links.

This seems to be a "known behavior" in Emotion: emotion-js/emotion#1061

We should investigate how to reduce the duplication. They mention having a "common" emotion ancestor, but I'm not sure exactly what that means.

Implement API Type Design

Should take advantage of design system, spacings and other tools from the root theme.

Clarify software license for open source packages

@nkuehn what is your idea for licensing the packages here?

Tracker: Images Plugin should wrap all images in <figure> independent of title or not

To be able to consistently format we want to suggest / request an improvement to the gatsby images plugin that allows to configure that it always wraps the images in a <figure> tag independently of whether there will be a <figcaption>. this allows for more consistent styling across the two use cases.

Add Cookie Consent Footer (external script)

See existing implementation (done in gatsby-ssr.js and excluding certain evironments)

example with commercetools ID:

import React from 'react';
import RootWrapper from './root-wrapper';
export const wrapRootElement = RootWrapper;

export function onRenderBody({ setPostBodyComponents }) {
  if (process.env.NODE_ENV === 'production' && !process.cwd().endsWith('/test'))
    setPostBodyComponents([
      <script
        key="pqr"
        type="text/javascript"
        src="https://cdn.cookielaw.org/langswitch/99bb6fae-765a-4fc5-9b6e-8c89a353335d.js"
        charSet="UTF-8"
        onLoad="function OptanonWrapper() {};"
      />
    ]);
}

Maybe we can detect preview builds using ZEIT variables: https://zeit.co/docs/v2/more/now-for-github/

Top Menu

• implemented in the kit (part of the theme)
• pulls its content / links from a central URL e.g. https://docs.commercetools.com/menu.json
• that URL is provided by a new "home" website in the commercetools-docs repo. A netlify redirect places it into the root.

theme: main container always has scrollbar

The main container y overflow is not set to auto, so when on windows or having a mouse attached the scrollbar is visible independently of whether it's needed.

From a windows user perspective that's perceived as a bug because it deviates from the 99% convention how websites behave.

Exploration: Expose "ramldoc" RAML types on GraphQL

Based on the RMF based file generator by @jenschude, let's expose the "RAML 1.0 Datatype" files on the GatsbyJS GraphQL API.

It can be implemented as an additional functionality in the transformer-raml branch that we started already.

• still "listening" on File Node creations, still filtering for .raml via mime type.
• If the first line indicates it's a Data Type fragment, go into this new piece of code
• Load the file content using a YAML parser , e.g. js-yaml or yaml
• apply the minimum necessary transformations to safely get it into GraphQL
  • arrays to objects to get a stable GraphQL schema (see / take from legacy plugin)
  • remove the brackets from(annotationsInBrackets)
• omit some of the fancier transformations, e.g. constants etc.
• Provide as a RamlDataType node on GraphQL, the parent node being the file read from.

what do you think?

Make `websites/site-template` an actual gatsby starter

(updated, doesn't have to be an NPM package)

• change name to gatsbyJS convention: gatsby-starter-commercetools-docs
• create a README
• (if we want to be a product: setup script inspired by https://github.com/EmaSuriano/gatsby-starter-mate/blob/master/bin/setup.js )

Not doing:

• register in the public starters list

resources:

• https://www.gatsbyjs.org/docs/creating-a-starter/
• https://medium.com/@emasuriano/how-to-create-a-gatsby-starter-e7d53083a880

Make MDX transclusion possible for topic based writing

Technical Writing works a lot with a concept named "Topic Based Writing" which is a fancy name for re-using content across many places. MDX has native support for transclusion and being able to use it would make this a much more interesting tool for large-scale usage.

MDX documentation: https://mdxjs.com/getting-started#documents

If done it would be very likely that the centralized content lives in a separate package.

example usage

import TopicBasedWritingConcept from "../topics/concepts/topic-based-writing.mdx"

<TopicBasedWritingConcept />

It did not work out of the box.

Square favicon / site icon without (anti)aliasing artifacts

The current SVG orginal of the site icon is not square, which the manfest plugin complains about for good reason. The resulting icons have the right part of the CT cube cut off.

Let's get a properly shaped SVG and ask a design wizard to create readily rendered variants for the very small resolutions (16x, 32x) that have smooth lines.

Component: Multi Language code example from external files

update: here's a branch with experiments in an actual MDX file: https://github.com/commercetools/commercetools-docs-kit/blob/extended-code-syntax-brainstorming/websites/docs-smoke-test/src/content/code-samples/multi-language-code.mdx

(This builds on top of #64 )

Since markdown does not allow to express multi-language code blocks examples that have a language switcher need to be pulled from external files. The implementation may also support multi-language code blocks that hold the actual code in the markdown document, but that is not the design we optimize for.

pseudo-MDX based on the pattern developed for single language code in #64 :

<MultiCode title="Creating a Cart">
   <CartCreateExampleJs />
   <CartCreateExampleJs highlightLines="5-10" />
</MultiCode>

This tickets stands for the first iteration, which must not yet support a data layout that has all the languages next to each other in a folder and allows to pull all languages via a wildcard import.

The languages end up in the dropdown on the right of the title.

It must be possible to pass a title

Child Elements are always single code examples that are rendered without their title bar and copy button.

Fix ARIA role for links

New api test website

Create new api test website that is based on the new api theme. Should contain eventual (empty) pages and their navigations

API docs: URL parameter switcher ("Region switcher")

For go-live we will only show the placeholder {region} in the base URL part of API and other docs, but it's much more useful to have user specific state that holds their default region and the components rendering API hosts and MC urls dynamically pick that up.

Required: A design and a naming strategy for the browser storage.

There must not be a default region, default is showing a placeholder or selector

Types not rendered as links in Import API docs site

Description

Types that are defined on a page somewhere on the website should be linked. This affect various property types and response representations on the Import API site.

Expectation
Types links are queried from graphql nodes generated gatsby-transformer-mdx-introspection tool, so the links are expected to be available in graphql.

Current behaviour

In the current case the links are not present in graphql. So types are rendered as a string.

Proposed solution

The next step is to look into the gatsby-transformer-mdx-introspection to implement the fix.

Rename design tokens colors to provide more meaningful context for usage

As the list of design tokens keeps growing, the current token names (especially for colors) are not super useful anymore.

I'd like to go over them (when I have some time) and provide more meaningful names.

Example of "good" names:

commercetools-docs-kit/packages/ui-kit/src/design-system.js

Lines 58 to 68 in d62f3b5

	borderRadiusForBetaFlag: customProperties.borderRadius4,
	borderRadiusForSearchDialog: customProperties.borderRadius6,
	borderRadiusForSearchInput: customProperties.borderRadius6,
	borderRadiusForCodeBlock: customProperties.borderRadius6,
	borderRadiusForTooltip: customProperties.borderRadius4,
	borderRadiusForContentNotification: customProperties.borderRadius6,
	borderRadiusForImageFrame: customProperties.borderRadius6,
	borderRadiusForBlockquote: customProperties.borderRadius6,
	shadowForBetaFlag: customProperties.shadow7,
	shadowForSearchDialog: customProperties.shadow2,
	shadowForFooter: customProperties.shadow6,

Possible to declare all allowed mdx frontmatter as explicit schema ?

(absolute nice-to-have just for robustness)

Currently, if a site's content never uses a frontmatter key that is queried by our implementation, the site build fails because the queried field is not on the graphQL schema.

The "beta" frontmatter is such a case.

I don't know whether it is possible at all to change the graphQL surface of other plugins (gatsbyjs is shielding plugins from each other pretty good), so this might well simply not work.

Smoke Test Sites should not have a valid google analytics ID configured

The smoke test sites currently have a valid google analytics ID configured, which is not desirable because that way they will track traffic into the GA project of the actual docs.

If not configuring one is not desired for testing purposes we can choose to configure a dummy string.

Evaluate keeping the main nav's scroll position

From a usability perspective it's desired that the main nav does not change its scroll position when navigating to a different page in the same site (= microsite with same nav).

It requires some technical investigation into how GatsbyJS behaves when navigating concerning component state.

Flash of unstyled content (regression)

The current master has a flash of unstyled content (tested on a windows 10 machine in Firefox and Chrome).

The vertical spacing both in the left hand nav and the body content is initially higher and gets reduced after page load.

Proposal: Allow or encourage absolute imports in MDX content

To make the way content is written easier to refactor (i.e. rename or move around in the content hierarchy) it would be helpful to allow authors to make import statements absolute.

e.g. when including an image or other MDX or code example file, instead of

import OptimisticConcurrencyDefinition from "../../topics/definitions/optimistic-concurrency.mdx"

writing

import OptimisticConcurrencyDefinition from "topics/definitions/optimistic-concurrency.mdx"

would yield less problem when e.g. moving the file that includes the content.

Root should be the src directory of the current package.

Gatsby supports this explicitly but it has to be configured:
https://www.gatsbyjs.org/docs/add-custom-webpack-config/#absolute-imports

Default to not showing the on-site search box if the whole site is configured "Noindex"

If a whole microsite is configured to be excluded from search engines, its content will not be indexed in the on-site search, too.
To avoid user confusion, such sites should not have the on-site search box because such a box suggests that the topic area you are currently reading is indexed.

Independently, we can consider configuring whether a top menu is shown or not. In total, it's about a "Private Beta" mode that has the following effects:

• beta on site level
• noindex on site level
• no search box
• no top menu (optional but we can hardwire it as a convention, too)

scrollbar overlaps index nav (windows test)

I just took a look at home and found the positioning of the main content scrollbar less than ideal. It feels visually buggy to have it not at the right edge of the viewport and it overlaps the index nav content.

this shot is taken in Firefox on windows 10, but also in Chrome on Windows 10. Maybe "emulatable" on a Mac, too on an external Monitor with a conventional mouse attached.

@Shecki can you make your opinion?

Automate RAML specs generation

Current Process

1. User sets up a Gatsby website which based on the theme @commercetools-docs/gatsby-theme-api-docs. See more information on Gatsby theme on their website here.
2. User clones RMF Codegen repo
3. User runs ./gradlew shadow to generate jar file.
4. User needs to understand location of jar file so the next command can work (I just copy over the file to wherever I want to run the next command)
5. User runs another command to generate RAML files as follows

rmf-codegen % time ./rmf-gen.sh generate -o <gatsby-theme-api-specs-path> -t ramldoc <raml-api-path>

6. Startup Gatsby site

Problem

Given that the bulk of intended users don't have software development background, the process can get easily complicated and therefore confusing. Setting up the actual doc site already has several steps.

Proposed Automated Process

1. Setup actual site
2. Provide path to RAML specs directory that is to be generated
3. Startup Gatsby site

Possible Development Steps to Automation

1. Standard requirement for some sort of config file that will contain the path of the RAML api to be generated.
2. Pre-build process

• Jar file is downloaded (needs to be published somewhere)
• Script that runs jar file is called with path from step 1 passed to it.
• RAML files are generated in api-specs directory.

3. After build is complete generated RAML files should be available i the application.

Component: Single Language code example from file

update: here's a branch with experiments in an actual MDX file: https://github.com/commercetools/commercetools-docs-kit/blob/extended-code-syntax-brainstorming/websites/docs-smoke-test/src/content/code-samples/multi-language-code.mdx

Alternatively to writing example code in markdown as fenced code blocks an author must be able to store the example code in a separate standalone file and pull it into any MDX page. This provides the advantage that the code examples can be own fully valid software projects that are automatically tested.

This issue covers the case of a single-language example like fenced code blocks.

We decided to leverage MDX import statements and custom webpack loaders, which is how gatsbyJS is generally built internally, too. Webpack can be configured cleanly by a plugin like the theme (see reference link below)

Any import statement loading from a path containing /code-examples/ (plus maybe further restrictions like whitelisting packages if possible) is managed by a custom loader that returns a function component that is wrapping our standard Code component. It's detecting the language by file name and injects the content.

The following code is pseudo-MDX

import CartCreateExampleJs from '@commercetools-docs/code-examples/javascript/cart/create.js'
import CartCreateExampleJava from '../../code-examples/java/cart/create.java'

<CartCreateExampleJs title="Creating a Cart in Javascript" />
<CartCreateExampleJava highlightLines="5-10" title="Creating a Cart in Java" />

Not necessary

We don't need to support cases that cannot be derived from the file name extension (e.g. individual terminal cli commands that are not part of a shell script)

References:

• https://www.gatsbyjs.org/docs/add-custom-webpack-config/
• how it's currently written in markdown (Liquid templating using a parameterized template import): https://github.com/commercetools/docs.commercetools.com/blob/master/sdk/jvm-sdk.md (private repository)
• https://prismjs.com/plugins/file-highlight/ (not nice because it does an HTTP call at runtime from the browser)

New API Theme

A theme that extends @commercetools-docs/gatsby-theme-docs. The new name gatsby-theme-api-docs

Production build should fail if a Type or Resource is missing or not found

• Look into error logging using Gatsby tools
• it's better if all errors are collected so build will only fail after that.
• consider e2e test

Consider automatically using the path prefix as the `websiteName` (e.g. in sentry)

The advantage of the path prefix is

• that it's more likely to not collide (it won't on all content hosted under the main docs domain where the sites come together)
• saves another separate ID for the same thing.

CON: one might want to change the one but not the other later on, but do we want to encourage the names and path to be different?

Idea: Shift Docsearch config levels down

The Docsearch UI shows the first three levels of the level configuration that was put into the docsearch github configs repo.

At the moment the first level is the Page title (= H1), the second is the H2 structure, the third is the H3 structure.

It has two issues:

• levels two and three are often identical
• little to no indication what general area of the website the hit lives in.

If we want to stick to the UX idea of docsearch that it's a little like a filtered subset of the complete information hierarchy, I propose to move the usage of the hierarchy one level up:

1. Part of the site (API reference, Tutorials, Custom Applications, Import API, Mechant Center documentation)
2. H1 (page Title)
3. H2 (Section) (and optionally: Flatten H3 into the third level, too to keep allowing deep matches - we have duplicate hits in the current config, too)

You would not be able to do the kind of super-deep deeplink matches we currently have (e.g. searching for a specific update action or type name and jumping there but we could

---

ALTERNATIVE

Alternatively we could first try the feature that enables faceting by attributes. I.e. sticking with the current hierarchy that links deep into the details and allowing the user to filter the results to a part of the documentation.

In general I think the hierarchy in the results must be in sync with what a breadcrumb would be to help users build and retain a mental map of the site structure.

FYI @Shecki not urgent but a think you can take into your breadcrumb etc. considerations

Investigate why imports in mdx are not working

MDX supports importing mdx and react components as react components:

https://mdxjs.com/getting-started#documents

Somehow it currently does not work in our setup.

Ref: #85

Move hardwired API names from api-docs theme to per-site config

gatsby-theme-api-docs configures the legacy raml parser with our default standards, which is good. But one option should be configured on site level, that's the includeApis whitelist.

https://github.com/commercetools/commercetools-docs-kit/blob/master/packages/gatsby-theme-api-docs/gatsby-config.js#L17

Instrument all Navigation Links with GA events

To be able to better understand how the internal navigational features of the website are actually used it's necessary to first instrument the respective links with Google Analytics Events.

Page navigation path analysis is built-in in GA and the GatsbyJS plugin takes care of tracking page changes (independently whether via react or as actual reloads).

But events need to be attached manually. The GatsbyJS google analytics plugin provides a wrapper function that allows robust event tracking.

Reading the relatively small source of the GA plugin explains most of what's going on:
https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-plugin-google-analytics/src/index.js

Outbound Links are also tracked as Events, but the code is blocking in that case and it has its own Action category and name.

GA docs here: https://developers.google.com/analytics/devguides/collection/analyticsjs/field-reference#events

• Event Categories could be MainNav, TopNav, IndexNav, SearchResults etc. .
• Event Type could be click (analogous to the actual event that happened in the DOM, but can be any string.
• Event Label is typically the Text of the specific thing clicked.

Other hierarchies are possible, too (e.g. Category "Nav" , Type "Click", Label "Main -> Getting Started -> Setting Up your IDE")

It must be chosen carefully what to track and what not because spamming the event tracking does not help either.

A visual heatmap cannot be generated from this type of data, but tracking the navigation structure logically has other advantages (robust against responsive design differences and varying menu lengths, scolling position of the menu etc)

bug(theme): automated graphql scheme generation affected by other plugins in sites

If a site uses the theme but also uses other plugins and these plugins are creating graphQL surface, the generated scheme file is updated in the theme plugin source code without the theme actually being affected.
If that modified file is accidentally committed it contains scheme that has nothing to do with the theme.

overall impression: since the theme is not generating any graphQL surface itself by creating nodes I think it does not need to have any sort of persisted schema definition.

Beta flag on chapter and (optional) site level

Beta can also be declared for a full chapter or a whole site.

Chapter level it would be declared in the navigation YAML.

site: to be decided.

Implement API Resource Design

Take advantage of design system, spaces and other tools from the root theme

Populate empty pages on api test website with **api types**.

Options for gatsby-transformer-raml

gatsby-transformer-raml should be configurable with the following options:

        {
          includeApis: ['import', 'import-storage-api', 'test'],
          movePropertiesToTop: [
            'id',
            'version',
            'key',
            'createdAt',
            'createdBy',
            'lastModifiedAt',
            'lastModifiedBy',
          ],
          movePropertiesToBottom: ['custom'],
        },

Image should only display the title but not the alt as a caption.

Pseudo-spec:

• if none is provided, neither alt nor a caption is generated (fallback to the file name OK but not ideal)
• if only alt is provided, it's used as alt but not as caption
• if only title is provided, it's used as caption and also as alt (better to have some accessibility than nothing)
• if both are provided alt is the alt and title the caption.

In any case, the html "title" attribute does not have to be set if

& are properly used.

Algolia Docsearch box in top bar

• whether a given site has a search box should be configurable site-wide
  • necessary configuration (no secrets, these are openly visible in the website afterwards): apiKey and indexName. We can also choose to simply allow mixing any allowed docsearch config into a default config.
• We can choose to implement the / keyboard shortcut and taking over the current selection in a separate step. The docsearch widget has an own bindKeyboardShortcuts setting but it's badly implemented and worked only on US keyboards the last time we tried.

Keep in mind that this is the special free Algolia docsearch  offering, not the Algolia product sold for money. 

The generic Algolia has an own set of React components, but docsearch has a pre-built widget for the docsearch offering that has to be used on docsearch sites.

This is the Docsearch widget: https://github.com/algolia/docsearch

Here's how Facebook's docosaurus includes it in React.js:  https://github.com/facebook/Docusaurus/blob/master/packages/docusaurus-theme-search-algolia/src/theme/SearchBar/index.js - not officially open source but you can take it as an inspiration. 
 
This GatsbyJS documentation is therefore not applicable to this case: https://www.gatsbyjs.org/docs/adding-search-with-algolia/

Configurable search engine include/exclude (site-wide with page override)

To be able to execute go-lives of new products and websites in a structured way it needs to be possible to configure the search indexing on a site-wide level and override it on a per-page level.

The site wide default should be configurable in the site's gatsby-config.js , per-page overrides should be MDX frontmatter (or use react helmet in pure JS pages)

Whether the implementation uses robots.txt or the HTML meta attributes or a combination is open to implementation as long as overriding works in both directions (override a page to be visible in an otherwise excluded site, override a page to be hidden in an otherwise indexed site).

References:

• https://support.google.com/webmasters/answer/93710?hl=en

Explore Breadcrumbs: Build Prototype

Remove ShortTitle from example page frontmatter

n/t

provided styled components for definition list HTML

In MDX / MD authors are allowed to write HTML definition lists. should have spacing and indentation as designed for the "Subscription Heath Status" example content.

Populate empty pages on api test website with **api resources**.

Broken Link Checker: Warn or throw errors on insecure links (`http://*`)

To gain more independence of Netlify's analysis features it would help if the broken link checker would warn on unencrypted URLs. It's checking all source links of images, scripts etc, too so that's a relatively broad coverage.