linjun007/flink-on-k8s-operator

Kubernetes operator for managing the lifecycle of Apache Flink applications.

Kubernetes Operator for Apache Flink

This is not an officially supported Google product.

Kubernetes Operator for Apache Flink is a control plane for running Apache Flink on Kubernetes.

Project Status

Alpha

The operator is under active development, backward compatibility of the APIs is not guaranteed for alpha releases.

Prerequisites

• Version >= 1.9 of Kubernetes
• Version >= 1.7 of Apache Flink

Overview

The Kubernetes Operator for Apache Flink extends the vocabulary (e.g., Pod, Service, etc) of the Kubernetes language with custom resource definition FlinkCluster and runs a controller Pod to keep watching the custom resources. Once a FlinkCluster custom resource is created and detected by the controller, the controller creates the underlying Kubernetes resources (e.g., JobManager Pod) based on the spec of the custom resource. With the operator installed in a cluster, users can then talk to the cluster through the Kubernetes API and Flink custom resources to manage their Flink clusters and jobs.

The operator supports creating both Flink job cluster and Flink session cluster through one custom resource FlinkCluster, depending on whether a job spec is provided. See samples: Flink job cluster, Flink session cluster.

Installation

The operator is still under active development, there is not Helm chart available yet, please follow the Developer Guide to build the operator and deploy it to your Kubernetes cluster.

Documentation

• Developer Guide
• FlinkCluster Custom Resource Definition
• Test environment setup for streaming applications
• Use remote job JAR in GCS
• Use GCS as remote storage with custom Flink images

Contributing

Please check CONTRIBUTING.md and the Developer Guide out.