qlik-oss/core-website

Source code for the core.qlik.com website.

Qlik Core Website

This repo contains sources to the Qlik Core end-user documentation.

Documentation is primarily represented as markdown files and images. Markdown files are organized in the same way as the generated site is organized.

MkDocs is used to generate and build the site. Currently the mkdocs-material theme is used but this shall be changes to a theme matches Qlik branding.

For the generated site, visit core.qlik.com.

Prerequisites

To develop and contribute to the Qlik Core documentation, the following must be available on the local machine:

• Docker - Docker for Windows or Docker for Mac
• Bash - On Windows, Git Bash included in Git for Windows is recommended

Serving the Site Locally

To generate and serve the site locally on a developer machine, use the serve.sh Bash script, in the repository root:

./serve.sh

This makes the generated site available locally.

Once the site is running locally, modifications to sources can be made and the local web server automatically updates. No re-build step is needed.

Building the Site

To build the site for deployment to some external web server, use the build.sh Bash script, in the repository root:

./build.sh

This builds the site contents in the site/ folder in the repository root. This folder is an entry in .gitignore to avoid undesired pending repo changes.

Linting the Documentation

For linting the documentation in this repository we use lint-condo, which contains a set of linters for linting a markdown file. The linting is part of the Circle CI pipeline, but can also be run locally

./lint.sh

The linters that are enabled is configured in .lint-condo.yaml.

core-website's Issues

Describe some fundamental logging aspects in the Core recipe

In the Core recipe, it can be valuable to see examples of how minimum logging levels can be set for each service, together with some shorter information on in which trouble-shooting scenarios it can be worth to modify logging levels.

Outhaul: Exemplify usage

Expected outcome:

• Problem definition; what problem does Outhaul aim to solve?
• Typical use case and usage flow.

Documented in README.md

Doc distribution: RabbitMQ integration

Discuss with engine how QIX engine integrates with rabbitMQ

Investigate how it works now:

• Reload events sent correctly
• Reload events received and acted upon
• Command options (feature flags)
• Qvf compatibility (file format)

Document the first Hello Engine/Data/Visualization examples

Should follow the structure of the "web-platform" examples.
This is about splitting the "web-platform" example.

Investigate/decide which website generation tool to use for Frontira page

• Must support markdown
• Will be used to generate the Frontira dev page

Update custom analytics use-case documentation

The documentation should clearly describe the use-case actually implemented and what scenario that is targeted.

Example of things that needs updating:

• Which deployment is targeted?
• Is dev-ops our target audience?

Deploy generated doc site to web server

• Possibly use the "tools" AWS instance to host the doc site.
• At least, master branch shall always be deployed.
• Look at possibilities to deploy all branches so that they can be reached, e.g. like http:///branch- or something.

Document how License Service logs

Also include information on how to set the minimum log level output by the service.

MongoDB connector in NodeJS with the new GRPC protocol

Document the GRPC protocol

Create link from Core recipe info to the core repo

For navigation.

Generate Frontira site on CCI

Build and generate the site static content on CCI for internal consumption and viewing.
Store the site as artifacts on CCI

Logging

#20

#21

#24

Document Qlik Elastic deliverable categories and corresponding test structure

Add definition of core, and link to the core repo

Add in the readme an initial description of the core recepie, and link to the core repo.

Create a web platform recipie

The recipie will be based on the current qlik elastiq tutorial.

The .MD files should be moved from there to here and modified to work as a recipe for the web platform.

This recipe will still be more basic than the others and be a good starting point for developers new to frontira.

Investigate options for auto-generating QIX API docs

We need an QIX API reference for Engine, we either have to link into Sense help or create a mechanism for generating a markdown file that lives with this documentation (for each Engine version).

Halyard.js integration test against a QIX engine docker image

Define what the logging recipe should contain

The logging format and levels will be presented in separate markdown which will be applicable for all Frontira services. This is a specification/contract and not a recipe.

The individual log levels/format for each service is documented on each service.

What should the logging recipe include? Tips and trix? Any specific scenario that needs documentation?

This task is to define the logging recipe, or if it is not needed at all.

Documentation source and ToC

This issue outlines several documentation topics that we see as needed before 1.0 can be released.

This repo (qlikea/info) will contain the Frontira end-user documentation source files. All end-user documentation that currently recides in other repositories should be consolidates and moved here.

Table of contents

To be filled in after sync with DocLoc.

Documentation source

• All Frontira components should have their end-user documentation here. They host their developer documentation themselves.
• All web platform components (e.g. enigma.js, halyard.js) host their own documentation, and should be linked to from here.

Needs syncing

• Auto-generated QIX API documentation (refer to Sense Help?)
• Auto-published "public" Engine image
• Documentation of important INI_INT's (-S <key>=<value>) and how they work/augment Engine
• Documentation of features more related to Linux/Docker (JWT, rules engine, TTL, GRPC data-loading)

Doc distribution: Sync with Qix engine testers

Help engine with getting test coverage regarding rabbitMQ integration
(create/reload/destroy)

• Possible to write unit test?
• Investigate integration test?

Set up Circle CI for mongodb connector

Start it and run the example, fail the build if the example exits with code != 0.

Document the Outhaul usage

Document "local" deployment of the Core recipe

Set up Circle CI for postgres connector

Start it and run the example, fail the build if the example exits with code != 0.

API lockdown

This is a placeholder task for the research and documentation activities related to API lock-down. Basically, we need to figure out what QIX Engine will support in time for GA.

The expected outcome is a new task describing what needs to be documented in this area.

Document log format in the logging recipe

Document the Hello Visualization example

Investigate options for generating flows and diagrams

When using mkdocs and markdown files, we need a way to generate diagrams in one central way.

Create a top-level recipe README.md

This README shall contain a summary of all recipes, and link to each separate .md file.

Write a landing/introduction text for the Frontira product

When landing on the Frontira documentation site, there must be a text that clearly describes the purpose of the Frontira product, it's intended use and target audience. It must describe the power and uniqueness of QIX Engine and describe what benefits come with building a solution on top of it. It must catch developer's interest in the very few first sentences.

Generate Frontira site using MkDocs

Move Mira end-user documentation to info repo

Move the end-user documentation of Mira to the info repo.
Keep developer-related information in the mira repo. This information may need some modifications when split up to make sense. This task involves all work needed in the info and mira repos.

Move License Service end-user info to this repo

Engine-doc: Add engine documentation for logging

We need to document the following for engine:

• Log format
• Log level
• Categories

This should be provided in a separate markdown in the services folder.

Document JWT and Section access

Note this task is pending #47.

Documentation should reside within info/docs/recipes/authorization.md with the following ToC:

• JWT
  • Introduction to JWT
  • JWT format (describes the structure and fields of our JWTs)
  • Configuration (describes how to configure QIX Engine to use JWT, including secrets, signing algorithm, etc)
  • Validation (describes how QIX Engine validates the JWT and what happens if validation fails)
  • Section access (describes what section access is and how it's used together with JWTs, should include a simple example)

Collect/research required logging fields

Contribute back to service contract repo in internal github org.

Research other conventions in and outside R&D.

Create reload recipe

Create a recipe that outlines how to load data using halyard and a custom connector to postgres

Write the Getting Started section

Give the user information on what first steps to take to get familiar with QIX Engine and the supporting libraries. Point to our tutorials. Point to our full use case.

Find inspiration by looking at other examples, e.g. Pluralsight and others.

License-service: Document how to toggle log levels + additional log fields

Investigate/define logging levels

When to use info/debug, warn, error levels etc. with a couple of examples.

Contribute back to service contract repo in internal github org.

Research other conventions in and outside R&D.

Document how to adjust log levels

Show with example (e.g. given a scenario where one Engine goes missing, turn on debugging in Mira/Engine and check how many Engines Mira finds).

Add Google Analytics to docs

We should use GA to get some data from the usage of our docs

Investigation :: JWT and section access

This is a placeholder task for the research activities that needs to be done before being able to do the actual documentation. We need to gather material around:

• JWT structure
  • Which fields are supported?
  • Which signing algorithms can be used?
• Engine configuration
  • How do we configure engine to use/validate JWT?
• How does engine validate JWT?
• Section access
  • Existing Qlik Sense documentation
  • How does section access relate to fields in JWT?

Newspaper use case

Introduce a use case describing how one could approach scaling relatively static data models to a large audience.

Doc distribution: Frontira documentation

Investigate and document different scenarios regarding doc reload/distribution

• Different file systems (distributed / local)
• Create/remove doc (API vs system)
• Qix message payload documentation
• Document rabbitMQ setup and monitoring

Add support for linting the Markdown doc

• Pick a good tool to lint the Markdown
• Integrate as step in CCI
• Make possible to run locally with low effort

Investigate Engine roadmap for doc distribution

Confidence level in new distribution features?
Messaging queue solution, when, what?
Endpoints to "reopen" doc?
Different strategies on view nodes in regards to responding to an reload event?

Add system design docs to Custom Analytics UI

Engine-doc: Monitoring/scaling

In https://confluence/display/QE/Monitoring+and+scaling you will find the following (expandable) headers with information regarding QIX Engine specifics:

• QIX engine memory management
• QIX engine CPU utilization and scaling over cores
• Linear scaling of QIX engine resources
• Frequency analysis of warnings to determine RAM saturation
• Scaling up versus scaling out

This information should be included in the qix-engine/README.md service documentation.