kata-containers/documentation

Kata Containers version 1.x documentation (for version 2.x see https://github.com/kata-containers/kata-containers).

Kata Containers

Welcome to Kata Containers!

This repository is the home of the Kata Containers code for the 2.0 and newer releases.

If you want to learn about Kata Containers, visit the main Kata Containers website.

Introduction

Kata Containers is an open source project and community working to build a standard implementation of lightweight Virtual Machines (VMs) that feel and perform like containers, but provide the workload isolation and security advantages of VMs.

License

The code is licensed under the Apache 2.0 license. See the license file for further details.

Platform support

Kata Containers currently runs on 64-bit systems supporting the following technologies:

Architecture	Virtualization technology
x86_64, amd64	Intel VT-x, AMD SVM
aarch64 ("arm64")	ARM Hyp
ppc64le	IBM Power
s390x	IBM Z & LinuxONE SIE

Hardware requirements

The Kata Containers runtime provides a command to determine if your host system is capable of running and creating a Kata Container:

$ kata-runtime check

Notes:

• This command runs a number of checks including connecting to the network to determine if a newer release of Kata Containers is available on GitHub. If you do not wish this to check to run, add the --no-network-checks option.

• By default, only a brief success / failure message is printed. If more details are needed, the --verbose flag can be used to display the list of all the checks performed.

• If the command is run as the root user additional checks are run (including checking if another incompatible hypervisor is running). When running as root, network checks are automatically disabled.

Getting started

See the installation documentation.

Documentation

See the official documentation including:

• Installation guides
• Developer guide
• Design documents
  • Architecture overview
  • Architecture 3.0 overview

Configuration

Kata Containers uses a single configuration file which contains a number of sections for various parts of the Kata Containers system including the runtime, the agent and the hypervisor.

Hypervisors

See the hypervisors document and the Hypervisor specific configuration details.

Community

To learn more about the project, its community and governance, see the community repository. This is the first place to go if you wish to contribute to the project.

Getting help

See the community section for ways to contact us.

Raising issues

Please raise an issue in this repository.

Note: If you are reporting a security issue, please follow the vulnerability reporting process

Developers

See the developer guide.

Components

Main components

The table below lists the core parts of the project:

Component	Type	Description
runtime	core	Main component run by a container manager and providing a containerd shimv2 runtime implementation.
runtime-rs	core	The Rust version runtime.
agent	core	Management process running inside the virtual machine / POD that sets up the container environment.
dragonball	core	An optional built-in VMM brings out-of-the-box Kata Containers experience with optimizations on container workloads
documentation	documentation	Documentation common to all components (such as design and install documentation).
tests	tests	Excludes unit tests which live with the main code.

Additional components

The table below lists the remaining parts of the project:

Component	Type	Description
packaging	infrastructure	Scripts and metadata for producing packaged binaries (components, hypervisors, kernel and rootfs).
kernel	kernel	Linux kernel used by the hypervisor to boot the guest image. Patches are stored here.
osbuilder	infrastructure	Tool to create "mini O/S" rootfs and initrd images and kernel for the hypervisor.
kata-debug	infrastructure	Utility tool to gather Kata Containers debug information from Kubernetes clusters.
agent-ctl	utility	Tool that provides low-level access for testing the agent.
kata-ctl	utility	Tool that provides advanced commands and debug facilities.
trace-forwarder	utility	Agent tracing helper.
runk	utility	Standard OCI container runtime based on the agent.
ci	CI	Continuous Integration configuration files and scripts.
ocp-ci	CI	Continuous Integration configuration for the OpenShift pipelines.
katacontainers.io	Source for the katacontainers.io site.
Webhook	utility	Example of a simple admission controller webhook to annotate pods with the Kata runtime class

Packaging and releases

Kata Containers is now available natively for most distributions.

General tests

See the tests documentation.

Metrics tests

See the metrics documentation.

Glossary of Terms

See the glossary of terms related to Kata Containers.

documentation's Issues

Add fedora installation guide

Add fedora installation guide for kata containers packages

Document SIGUSR1 in developer guide

Explain how to obtain a backtrace by sending SIGUSR1 to a kata component.

qemu-img dependency

In the "Create an image" section,
in the "Build the image" sub section
I hit the "qemu-img" not installed dependency. It probably needs to be called out upfront in the "Assumptions" section up-top.

Fix minor typos in the development guide

On reading the development guide, I noticed a few changes
that would enhance the readability of the guide. These are just
minor changes.

Create licensing strategy document

We need a brief document explaining that the project uses SPDX license identifiers.

Add mention of collect script in dev guide

Now that kata-containers/runtime#93 has landed, add details of it to the developer guide.

Create installation documentation

Once kata-containers/packaging#1 is resolved and we have OBS packages, we need instructions for all distros we produce packages for.

Remove mention of Clear Containers in dev guide

The dev guide references Clear Containers packages.

Once we have packaged components, we need to update the dev guide to reference the Kata packages instead.

Improve agent debug enable in dev guide

The dev guide currently only enables agent debug if no kernel params have been specified in the config file. We can do better than than.

Add ubuntu installation guide

Add ubuntu installation guide for kata containers packages

Add limitations document

We need to create an equivalent of the Clear Containers limitations document.

Every limitation listed MUST be accompanied by a reference to an issue number.

Update dev guide to make runtime use qemu-lite

The runtime build now defaults to using vanilla qemu. Update the developer guide to reset that value back to qemu-lite for the time being.

Related: #68.

Embellish the readme

The README is somewhat sparse. Add some brief explanatory text and a few pointers.

working scenarios document

The working scenarios are discussed in kata-containers/runtime#31 as part of the design requirement of Kata Containers.

Add CoC doc

Add missing CODE_OF_CONDUCT.md document.

Add Developer guide

The developer guide in the wiki needs to move "in-tree".

install: Ubuntu docs does not work.

There is a typo in the installation in ubuntu docs.

Add an upgrading doc

Add a doc explaining how to "upgrade" from Clear Containers. At this stage, that will actually just be "install kata packages" 😄

Add code of conduct doc

Add the standard code of conduct document.

Add information how to use kata + cri (contaienrd) + k8s.

Include details how to create an entrusted pod.

Add SLES installation guide

Add design documents to documentation repo

create a directory in the repo for design document

add requirements document

dev docs, need updating for initrd and qemu?

I think https://github.com/kata-containers/documentation/blob/master/Developer-Guide.md#build-and-install-the-kata-containers-runtime may be out of date due to a couple of recent changes:

• the toml config file now has a line for an initrd.img, which is not built and installed by this doc - so kata-runtime fails to load the toml (as the file is not found, which is an error) (a quick fix is to just hand comment out the initrd img line in the toml)
• the toml config file now referes to the system qemu (not the lite version), but on some distros (my Ubuntu machine for instance), the system qemu is old (2.5.0), and does not have the necessary nvdimm options, so errors out. (a quick fix is to update the toml by hand to point back at the lite qemu)

Create an architecture document

We need to create an ARCHITECTURE.md that covers the overall system.

This document could, if necessary, reference architecture documents found in each components repository. These would essentially document just themselves. The intent is to define once and reference as many times as needed.

For example, the ARCHITECTURE.md created in this repository may say something like:

For further details on the Kata shim implementation, refer to the shim architecture document

Component repositories

The individual README.md files in each code repository must contain a reference to this central architecture document.

If a component repository has a separate ARCHITECTURE.md this:

• Must be referenced in the components README.md.
• Must itself reference the central ARCHITECTURE.md document.

For an example of what we should be aiming for, see the Clear Containers architecture document:

• https://github.com/clearcontainers/runtime/blob/master/docs/architecture/architecture.md

Diagrams

All diagrams must:

• Be stored along with their corresponding "source" files (for example, if a .png file is referenced, the .svg file that was used to generate the .png must also be stored to allow the diagrams to be updated by all.
• Use freely available open formats for their sources.

explain how to build and setup initrd image in dev guide

People are asking how to use it in the kata-dev channel. Let's document it in the dev guide for other's reference.

Add doc for assets

Add a new document covering how and when we will update assets (image, kernel, and hypervisor [for qemu-lite]). Doc can be based on:

• https://github.com/clearcontainers/runtime/blob/master/docs/assets-update.md

Add runtime library API design

Fix release checklist link

The "release checklist" in the Releases.md should link to https://github.com/kata-containers/documentation/blob/master/Release-Checklist.md, but doesn't ;)

Need to update the prereqs for Kata Dev env setup

While trying to setup the Dev Environment for Kata I came across some dependencies (e.g make, gcc) which need to be installed before proceeding with the dev env setup. We can also incorporate links to relevant documents (e.g. Golang install steps)

Use the correct rootfs directory name when copying the kata-agent binary

When adding a custom agent to the image the following command fails.
sudo install -o root -g root -m 0550 -t rootfs/bin ../../agent/kata-agent

Require doc team sign-off for docs

We should require an "ack" from the documentation team before merging any doc changes on PRs.

verify and add documentation for network aggregation use case

verify and add documentation for network slice use case

verify and add documentation for GPU passthrough use case

Example md @ https://egernst.github.io/posts/Clear-Containers-and-graphics-passthrough

Should verify minimal capabilities required and make use of simple openCL test (hash cat).

Prevent multiple addition of kata runtimes in the docker configuration

If kata-runtime is already added as a runtime to kata-containers.conf
then you need not add it again. This is helpful when we redo some
of the earlier steps mentioned in the dev-guide and end up updating
docker config again resulting in multiple "--add-runtime kata-runtime=.."
which further results in failure in docker restart.

Ideally Should Be:
[Service]
Type=simple
ExecStart=
ExecStart=/usr/bin/dockerd -D --default-runtime runc --add-runtime kata-runtime=/usr/local/bin/kata-runtime

But Can Be:
[Service]
Type=simple
ExecStart=
ExecStart=/usr/bin/dockerd -D --default-runtime runc --add-runtime kata-runtime=/usr/local/bin/kata-runtime --add-runtime kata-runtime=/usr/local/bin/kata-runtime

Result:
sudo systemctl restart docker
systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/docker.service.d
└─kata-containers.conf
Active: failed (Result: exit-code) since Mon 2018-04-09 21:22:08 IST; 536ms ago
Docs: https://docs.docker.com
Process: 21784 ExecStart=/usr/bin/dockerd -D --default-runtime runc --add-runtime kata-runtime=/usr/local/bin/kata-runtime --add-runtime kata-runtime=/usr/l
Main PID: 21784 (code=exited, status=1/FAILURE)
CPU: 59ms

Apr 09 21:22:08 localhost.localdomain dockerd[21784]: -v, --version Print version information
Apr 09 21:22:08 localhost.localdomain dockerd[21784]: and quit
Apr 09 21:22:08 localhost.localdomain dockerd[21784]: Run 'dockerd COMMAND --help' for more information on a command.
Apr 09 21:22:08 localhost.localdomain dockerd[21784]: , Code: 125
Apr 09 21:22:08 localhost.localdomain systemd[1]: docker.service: Service hold-off time over, scheduling restart.
Apr 09 21:22:08 localhost.localdomain systemd[1]: Stopped Docker Application Container Engine.
Apr 09 21:22:08 localhost.localdomain systemd[1]: docker.service: Start request repeated too quickly.
Apr 09 21:22:08 localhost.localdomain systemd[1]: Failed to start Docker Application Container Engine.
Apr 09 21:22:08 localhost.localdomain systemd[1]: docker.service: Unit entered failed state.
Apr 09 21:22:08 localhost.localdomain systemd[1]: docker.service: Failed with result 'exit-code'.

Fix runtime link in dev guide

The "Build and install a Kata Containers runtime" link should be "Build and install the Kata Containers runtime" to match the section name in the developer guide.

Update devguide for initrd

Now that kata-containers/runtime#98 has landed, we need to update the dev guide for initrd handling.

Add RHEL installation guide

verify and document SPDK use case

Update dev guide for OBS installation packages

Add details to the developer guide explaining that the recommended way to install Kata is using the OBS binary packages.

Devs can then override particular components by building them from source.

Add Release Process doc

We need an equivalent of:

• https://github.com/clearcontainers/runtime/wiki/Release-Process
• https://github.com/clearcontainers/runtime/wiki/Release-Checklist

Explain how to setup a debug console

Whilst we wait for kata-containers/agent#169 to finish simmering, we need to document in the developer guide how to manually setup a debug console for debugging purposes.

Add further details on Containerd CRI Plugin in the arch doc

Update the architecture guide with more details on the Containerd CRI Plugin.

Update dev guide to explain how to build a custom kernel

Once kata-containers/packaging#1 lands, we should have some tooling to make it easy to create a custom kernel based on the base Kata kernel configuration.

Install Guest Kernel Images section needs location of clear-container vmlinux/vmlinuz

The "Install Guest Kernel Images" section of the document is missing a pointer to the location wherefrom the user should fetch the "vmlinux.container" and "vmlinuz.container" images.

Enable pullapprove

Require 2 acks from the doc team, for parity with the other repos.

Update dev guide for vanilla qemu

Update https://github.com/kata-containers/documentation/blob/master/Developer-Guide.md to explain that qemu-lite is no longer needed.

Add CentOS installation guide

add minimum contributing guidelines