cdk8s-team/cdk8s

Define Kubernetes native apps and abstractions using object-oriented programming

cdk8s's Issues

[Suggestion] allow to set the Python version for python-app

Is your feature request related to a problem? Please describe.
When initializing a new python-app cdk8s automatically sets the Python version for the pipenv to 3.7. As I'm running Ubuntu 18.04 which comes with Python 3.6 by default I end up with a lot of warnings but the execution of cdk8s works just fine.

Describe the solution you'd like
As I didn't encounter any problems running cdk8s with Python 3.6 I suppose it is compatible with Python versions lower than 3.7. If that's true it would be nice to be able to set the desired Python version when executing cdk8s init python-app. Something like cdk8s init python-app --python-version 3.6 would be very nice.

Import custom resources

Support for resources that depend on things like Tiller, istio, etc

Release to all package managers in parallel

At the moment our release workflow is sequential. This takes time and also if one target fails, all of them fail. It shouldn't be too hard to harvest the dist/ directory as a build artifact and then create a job that releases to each package manager.

Mailing List - No access

The mailing list link leads to a Google Group that I have no authorization to access.
Isn't this supposed to be public?

Thanks in advance.

Map references and dependencies between resources and charts

helps users with generation and debugging

Docker assets

Similarly to AWS CDK, cdk8s should provide a way to bundle docker images in constructs to support end-to-end "gitops" experience.

Imagine your cdk8s app includes both the k8s definitions and your application code (rooted by a Dockerfile).

Then you, commit a change to any of these and your cluster is updated.

[Suggestion] cdk8s.yaml config file should support object version overrides

Is your feature request related to a problem? Please describe.

Right now, with the CLI you can specify specific versions of k8s objects. For example,

cdk8s import k8s --language typescript --include io.k8s.api.extensions.v1beta1.Deployment

To specify the latest k8s version, except to include the v1beta1 version of Deployment

Describe the solution you'd like

This should be possible in the cd8ks.yaml config.

Describe alternatives you've considered

I think there's a couple ways to do this:

app: node index.js
language: typescript
imports:
  - [email protected] --include  io.k8s.api.extensions.v1beta1.Deployment

app: node index.js
language: typescript
imports:
  - [email protected]
include:
  -  io.k8s.api.extensions.v1beta1.Deployment

app: node index.js
language: typescript
imports:
  - [email protected]
overrides:
  - import: [email protected]
    override: io.k8s.api.extensions.v1beta1.Deployment

Additional context

Of these, I'd go with the 2nd or 3rd. The 2nd simply mimics the current functionality, while the third extends it.

I think in the future, when, say, the knative API objects are supported, then 3 would be the best way to do this. That being said, 2 solves the problem now, and 3 can be implemented when needed.

I'd choose (2).

[Suggestion] change —app to —cmd for cdk8s synth

Is your feature request related to a problem? Please describe.
Since most people are creating cdk8s “apps”, it’s a little confusing that the “app” parameter for cdk8s synth takes in an executable and a file rather than the app name.

Describe the solution you'd like
We could change the parameter name to cmd to more clearly convey the intent: that this is the cmd used to run the synthesis.

Describe alternatives you've considered
We could leave it how it is since the templates make this pretty clear, but for those not using templates it’s still confusing. We could also just add better docs to the command line, but I don’t think this name is so important that it warrants keeping the name over adding docs.

cdk8s.json

The CLI should be able to read its configuration from a file at the project root directory in combination with command line switches.

Publish Constructs as Helm Chart

Auto-generation of helm charts

[Suggestion] fix ordering in YAML output

Is your feature request related to a problem? Please describe.
Synthesized YAML shows up out of order from standard K8s format. It technically works, but for a user familiar with K8s YAML formatting can be confusing to read.

example:

spec:
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: 8080
  selector:
    app: myappnginx37F28063
kind: Service
apiVersion: v1
metadata:
  name: myapp.nginx.service.5f289726

Describe the solution you'd like
Follow K8s YAML formatting conventions

1. apiVersion
2. kind
3. metadata
4. spec
5. etc...

example:

apiVersion: v1
kind: Service
metadata:
  name: myapp.nginx.service.5f289726
spec:
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: 8080
  selector:
    app: myappnginx37F28063

There may be a styling guide somewhere but I can't find it - looking at K8s documentation

[BUG] Autogenerated resource names not properly formatted

Auto-generated resource names follow the format this.object.uniqueid. This results in output that does not conform with K8s naming validation.

Input TS

new Deployment(this, 'deployment', {
      spec: {
        replicas: 1,
        selector: {
          matchLabels: label
        },
        template: {
          metadata: { labels: label },
          spec: {
            containers: [
              {
                name: this.node.uniqueId,
                image: options.image,
                ports: [ { containerPort } ]
              }
            ]
          }
        }
      }
    });

Output YAML

spec:
  replicas: 1
  selector:
    matchLabels:
      app: myappnginx37F28063
  template:
    metadata:
      labels:
        app: myappnginx37F28063
    spec:
      containers:
        - name: myappnginx37F28063
          image: nginx
          ports:
            - containerPort: 8080
kind: Deployment
apiVersion: apps/v1
metadata:
  name: myapp.nginx.deployment.f97e0de6

Error

Error from server (Invalid): error when creating "myapp.k8s.yaml": Deployment.apps "myapp.nginx.deployment.f97e0de6" is invalid: spec.template.spec.containers[0].name: Invalid value: "myappnginx37F28063": a DNS-1123 label must consist of lower case alphanumeric characters or '-', and must start and end with an alphanumeric character (e.g. 'my-name',  or '123-abc', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?')

Publish constructs as custom resources

Implement a generic Kubernetes Operator which will allow publishing constructs as custom k8s resources.

[hypothetical] README

You can use cdk8s to implement custom Kubernetes resources (CRDs) which are backed by a generic operator.

Let's take the WebService example and use cdk8s to publish it as a CRD.

CRD manifest

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: webservices.example.com
spec:
  group: example.com
  names:
    kind: WebService
    plural: webservices
    singular: webservice
  scope: Namespaced
  versions:
    - name: v1
      schema:
        openAPIV3Schema:
          properties:
            spec:
              properties:
                containerPort:
                  type: integer
                image:
                  type: string
                port:
                  type: integer
                replicas:
                  type: integer
              required:
                - image
              type: object
          required:
            - spec
          type: object
      served: true
      storage: true

Here's how users will use this custom resource in a manifest (equivalent to this example):

apiVersion: v1
group: com.example
kind: WebService
metadata:
  name: hello
spec:
  image: paulbouwer/hello-kubernetes:1.7
  replicas: 2
---
apiVersion: v1
group: com.example
kind: WebService
metadata:
  name: ghost
spec:
  image: ghost 
  containerPort: 2368

Here's a sketch for how you would expose the WebService construct through a CDK8s app:

import { WebService, WebServiceOptions } from './web-service';
import { App, Stdio } from 'cdk8s';

const app = new App({ outdir: Stdio.stdout }); // <-- sketch

const objects = Stdio.readYamlObjects();

// we might be able to offer some sugar for this
for (const obj of objects) {
  if (obj.kind !== 'WebService' || obj.group !== 'com.example') {
    throw new Error(`unsupported object kind ${obj.group}.${obj.kind}`);
  }

  new WebService(app, obj.metadata.name, obj.spec);
}

app.synth();

When this CDK8s app is executed:

node index.js

Here's what happens:

1. Read STDIN and parse it as Kubernetes manifest (multiple objects)
2. Define a new WebService instance for each object defined in the manifest
3. Pass spec of each object to the WebServiceOptions parameter of the WebService constructor.
4. Synthesize the result to STDOUT

Operator Deployment

To serve this resource, we need to deploy an operator to our k8s cluster that will subscribe to changes for k8s objects of this kind, will invoke our CDK app with the updated manifest and apply the result to the cluster.

NOTE: we will need #178 to be implemented so that the operator can apply with --prune. Otherwise, there is no way to delete resources.

Since cdk8s app can be written in any programming language and users need to be in control of the environment used during synthesis, we recommend to package your CDK app into a docker image which will be deployed as a side car container with the operator image.

Therefore, to deploy a CDK-based custom resource to a cluster, we will need:

1. The CRD
2. A Deployment with two containers - the generic operator and the CDK app's sidecar image.
3. These containers will be connected through an http server.

Sidecar Protocol

The sidecar container will start a CDK synthesizer server which exposes a local http endpoint used by the operator to trigger synthesis. The protocol will be a simple CGI-like? server where the request body is the updated YAML manifest and the response is the synthesized output. The server will spawn the CDK app (above) for every request, pass the request body to STDIN and send back STDOUT as the response body.

Alternatives considered:

• Operator spawns the CDK app within the same container: The main downside is that it will require that the runtime environment in that image will be suitable for both the CDK app (in any language) and the operator.
• Operator spawns the CDK app in a "docker in docker" setup. The main downside is that this will require privileged execution and will be more complicated from an image management perspective. Using a sidecar has the benefit of delegating all image management and spawning to k8s.

[BUG] Images do not show on desktop under cdk8s.io features

Describe the bug
On the features section of the website, images show next to the feature descriptions on mobile, but not on the desktop.

To Reproduce
Open https://cdk8s.io/#Features in a desktop browser.

Expected behavior
Images render. Images need to be updated, also.

Additional context
Could be a Hugo problem. Need to look at HTML/CSS source code.

[Suggestion] Create python examples, similar to the typescript examples

Is your feature request related to a problem? Please describe.

Python developers who want to use cdk8s only really have the "intro to python": https://github.com/awslabs/cdk8s/blob/master/docs/intro-python.md

There should be more complex examples for python.

Describe the solution you'd like

I would like each of the current 4 examples in examples/ to be copied to Python as well.

Describe alternatives you've considered

Alternatively, these examples could be different from the typescript examples, so we have more of a variety of examples.

Additional context

[Suggestion] documentation on updating to use the latest release

Is your feature request related to a problem? Please describe.
We need clear instructions on updating your CDK to the latest release, or a simpler method. Today you have to:

• update package.json file to the latest release version ("@awslabs/cdk8s": "^0.10.0")
• run yarn install, yarn build,yarn synth

Describe the solution you'd like
Single shot CLI based solution to use the latest build.

Allow specifying module name when importing

Proposed syntax: [module=]import.

For example, the following cdk8s.yaml file includes a bunch of imports:

app: node index.js
language: typescript
imports:
  - [email protected]
  - https://raw.githubusercontent.com/jenkinsci/kubernetes-operator/master/deploy/crds/jenkins.io_jenkins_crd.yaml
  - mattermost=mattermost_crd.yaml

• The first import (k8s API) will use the default k8s module name.
• The second CRD will import to jenkins (which is the resource kind).
• The resource kind of the Mattermost CRD is actually ClusterInstallation. This means that the default module name will be clusterinstallation so we here we want to provide an alternative and we prefix the import source with mattermost= which tells the CLI to use this module name instead.

cdk8s synth

Add a cdk8s synth command which will provide a uniform experience for executing the cdk application.

Real world examples

• pure K8s app example
• stateless AWS app example
• stateful AWS app example

add to /examples

Generate Jsii-compatible interfaces from api spec

Getting started link is broken

Getting started is broken

[Suggestion] cdk8s documentation should follow k8s documentation style guide

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Documentation for cdk8s can be written by many people, with different styles of writing. In the CONTRIBUTING.md guide, we should suggest that contributors use the k8s style guide found here

Describe the solution you'd like
A clear and concise description of what you want to happen.

A quick sentence and/or paragraph in CONTRIBUTING.md that points to the k8s documentation style guide.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Rename `cdk8s generate` to `cdk8s import`

We decided to use the "import" terminology instead of "generate"

Improve messaging about cluster agnostic-ness

We should improve the message that cdk8s works with any Kubernetes cluster.

Ideas:

1. Show logos of popular k8s providers at the top of the readme.
2. Demo gifs for using cdk8s with multiple providers

[Suggestion] A hook in aws-eks for cdk8s Chart

Is your feature request related to a problem? Please describe.

We would like to abstract the details of provisioning AWS infrastructure (EKS, etc.) from our customers (data scientists). They provide a Spark docker container with the application code and we run it for them. We are leveraging a new capability in Apache Spark so that we can run the job on EKS. We have tested the solution using Cluster.addResource() of the aws-eks module and would love to use cdk8s in order to provide some reusable constructs for our customers.

Describe the solution you'd like
Here's a sample of what we are thinking of building.

main.ts:

import cdk = require('@aws-cdk/core');
import k8s = require('cdk8s');
import eks = require('@aws-cdk/aws-eks');
import { SparkEksJob } from './spark-eks-job';

class SparkPiChart extends k8s.Chart  {
  constructor(scope: cdk.Construct, ns: string) {
    super(scope, ns);

    // my spark job image
    new SparkEksJob(this, 'SparkPi', {
      image: 'mysparkimage',
      command: [
         "/bin/sh",
          "-c",
          "/opt/spark/bin/spark-submit",
          "/opt/spark/examples/jars/spark-examples_2.11-2.4.4.jar"
       ]
    });
  }
}

const jobApp = new k8s.App();
const sparkPi = new SparkPiChart(app, 'spark-pi-job');

// This will be its own construct, but leaving this here for simplicity
const cluster = new eks.Cluster(this, 'job-cluster');

// add the chart 
cluster.addChart(sparkPi);

// synthesize
cdk.app.synth();

spark-eks-job.ts:

import { Construct } from '@aws-cdk/core';
import { App, Chart } from 'cdk8s';

// imported constructs
import { Job } from './imports/k8s';

export interface SparkEksJobOptions {
  /** 
    * The Docker image to use for this service.
  */
  readonly image: string;

  /**
   * Command to run
   */
  readonly command: Array<string>;
}

export class SparkEksJob extends Construct {
  constructor(scope: Construct, ns: string, options: SparkEksJobOptions) {
    super(scope, ns);
    
    // the rest of my k8s Job definition
    new Job(this, 'SparkJob', {
      ...
    });
  }
}

Describe alternatives you've considered
AWS CDK + Helm charts

I can work with you on example implementation/testing.

[BUG] release scripts run for forks

Describe the bug
The release scripts currently run with commits to master on forked repositories, causing an error because the secrets are not available.

To Reproduce
Fork the repo, wait for an upstream commit to master and rebase, or commit directly to master.

Expected behavior
Release script should not run on forks

Golang Support

Tracking issue for Golang support in cdk8s. Please +1 this issue to let us know you care (no need to comment).

Remaining work:

• cdk8s-team/cdk8s-cli#26
• cdk8s-team/cdk8s-cli#27
• #637

[Suggestion] support imported `.yaml` with mixed CRD and normal resource definitions

Is your feature request related to a problem? Please describe.

When importing resources, we generally expect them to be either CRDs or k8s. We should support a file that has both CRDs and standard resource definitions in them.

For example, take https://operatorhub.io/operator/composable-operator (can also be found here: https://operatorhub.io/api/operator?packageName=composable-operator)

This contains Service (from k8s), a CRD named Composable, and a KafkaSource (from api version sources.eventing.knative.dev/v1alpha1).

Describe the solution you'd like
A clear and concise description of what you want to happen.

They should be able to import this file, and have 3 imports/:

service.ts (they shouldn't need all of k8s imported)
composable.ts (on of the CRDs)
kafkasource.ts (a standard resource from another api version)

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

This should unblock some work on https://github.com/awslabs/cdk8s/issues/72

This issue might be similar to https://github.com/awslabs/cdk8s/issues/71 but I think is distinct enough to have its own issue.

[BUG] I should be able to import a CRD from an insecure server

Describe the bug
A clear and concise description of what the bug is.

I want to import my CRD from an http URL.

To Reproduce
Steps to reproduce the issue.

cdk8s import http://example.com/myCRD.yml

Expected behavior
A clear and concise description of what you expected to happen.

I expect this to fetch from that server. I get a Error: can't find file http://example.com/myCRD.yml error)

Additional context
Add any other context about the problem here.

Fix is right around here:

https://github.com/awslabs/cdk8s/blob/edf2e8ce62a588b099be75a625b2323427d364d9/packages/cdk8s-cli/lib/import/crd.ts#L87-L97

---

This can be a good first issue for someone wanting to contribute! If no one picks it up for a while, I'll take it.

Elasticsearch L2 Research

[cdk8s-plus] Ingress L2

Java support

Tracking issue for Java support in cdk8s. Please +1 this issue to let us know you care (no need to comment).

How to use multi-line in ConfigMap?

apiVersion: v1
data:
  kubernetes.conf: |-
    <match fluent.**>
      @type null
    </match>

kind: ConfigMap
metadata:
  name: fluentd-config
  namespace: kube-system

new ConfigMap(this, name, {
    metadata: {
        name: 'fluentd-config',
        namespace: 'kube-system'
    },
    data: ???
});

How to change multi-line to cdk8s?

Documentation on using the CDK8s CLI

• cdk8s init
• cdk8s generate
• changing the default version of the K8s API
• K8s custom resource definitions

[BUG] CRD-importing should support templates with multiple resources

Describe the bug

A template with multiple API Objects in it throws an error when trying to import.

To Reproduce

Example:

Save this as template.yaml (example from: https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/)

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: crontabs.stable.example.com
spec:
  group: stable.example.com
  versions:
    - name: v1
      served: true
      storage: true
  scope: Namespaced
  names:
    plural: crontabs
    singular: crontab
    kind: CronTab
    shortNames:
    - ct
  preserveUnknownFields: false
  validation:
    openAPIV3Schema:
      type: object
      properties:
        spec:
          type: object
          properties:
            cronSpec:
              type: string
            image:
              type: string
            replicas:
              type: integer
---
.
.
.
(same as above resource)

cdk8s import template.yaml throws:

name: 'YAMLSemanticError',
message: 'Source contains multiple documents; please use YAML.parseAllDocuments()',

Expected behavior

This should be supported by cdk8s as k8s allows for a collection of APIObjects in a CRD.

https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/

Additional context

Seems like the error is pointing to here:

https://github.com/awslabs/cdk8s/blob/master/packages/cdk8s-cli/lib/import/crd.ts#L29,L35

Should just be using parseAllDocuments() and then likely changing some return types and whatever needs be based on using CustomResourceDefinitionManifest[] instead of just CustomResourceDefinitionManifest

[cdk8s-plus] ConfigMap L2

[Feature request]Is it ready for ServiceAccount, ClusterRole, ClusterRoleBinding, DaemonSet?

new ServiceAccount(this, 'fluentd-serviceAccount', {
    metadata: { 
        name, 
        namespace: "kube-system"
    },
});

npm run synth

found nothing output...

"cdk8s gen --language X": support generating API objects in all target languages

The requirement is that cdk8s gen will be able to generate code in all languages supported by jsii.

Proposed usage:

cdk8s gen --language java|typescript|python|csharp [--outdir .gen]

synth/deploy CLI

apply.sh

.NET support

Tracking issue for .NET support in cdk8s. Please +1 this issue to let us know you care (no need to comment).

RFC: Should we recommend to commit imports?

The current guidance and project templates recommend that imports are not checked in to source control and basically generated upon build.

This brings up a few issues:

1. In some languages (such as python or JavaScript), there isn’t actually a build step.
2. If these files are not checked in, users need the cdk8s CLI to be installed in their CI/CD environment and so they can run “import”
3. In the early stages of the project, the APIs are still in flux and every version can introduce breaking changes. If users commit these files to their repos, they are less likely to be broken if we change the generated out. On the other hand, if the framework changes in a breaking way, the generated checked in code itself may break..

I am leaning towards changing our guidance (at least in python) to recommend that the “imports/“ directory be committed to source control, so it becomes an intrinsic part of the project source code. I have a feeling it will work better for users and will simplify integration with existing tools.

Vend the cdk8s-cli through standard package managers (homebrew etc)

It is awkward to need to install node for non-JS languages. We should vend the CLI through homebrew and other popular package managers so their installation will be more idiomatic to each platform.

Dependencies and order between resources and charts

It is not uncommon to require that resources will be applied to a k8s cluster in a specific order. For example, if a resource is placed in a namespace, the Namespace resource must be applied before it.

Constructs have a concept of dependencies (Node.of(this).addDependency(that)), and cdk8s should respect dependencies between resources when the chart's manifest is synthesized so that the resources are ordered in the manifest based on their topological order.

Furthermore, if a dependency is added between two charts or resources from two different charts, we need to somehow codify the order between the charts. To do that we can employ a convention in which charts have a numeric ordinal prefix (i.e. 000-chartname.yaml). This seems like a practice used by other projects (example) and offers a simple way for users to apply these charts through a simple kubectl apply -f DIR.

Reference documentation website

Generate cdk8s reference documentation and include in website

Improve website design

• Logo
• Graphics
• Layout & stuff

Build in jsii

[BUG] "Getting Started" link on cdk8s.io doesn't go anywhere

Describe the bug
A clear and concise description of what the bug is.

On https://cdk8s.io/ the first link there "Get Started with cdk8s" links to https://cdk8s.io/getting-started/ which doesn't exist.

To Reproduce
Steps to reproduce the issue.

Open https://cdk8s.io/ and click the button.

Expected behavior
A clear and concise description of what you expected to happen.

There should be a landing page for "Getting started" and users can then choose whether to use Typescript or Python.

Additional context
Add any other context about the problem here.

Great first issue!

[Suggestion] allow to skip the empty directory check when initializing

Is your feature request related to a problem? Please describe.
On several occasions when executing cdk8s init I encountered the following error message: Cannot initialize a project in a non-empty directory. This is completely fine in most cases but e.g. when there's nothing else than a .git folder or anything cdk8s unrelated in the directory it would be very nice to skip this check.

Describe the solution you'd like
My proposal would be to add a generic --force or a more specific --skip-empty-check flag to the init command to allow skipping the is-directory-empty-check.

Describe alternatives you've considered
Right now I move all files out of the directory, execute cdk8s init and move the files back.

[Suggestion] allow cdk8s synth to print to stdout

Is your feature request related to a problem? Please describe.
Currently when using cdk8s with tools that can read K8s manifests from stdin I first need to execute cdk8s synth and afterwards cat dist/*.

Describe the solution you'd like
It would be nice to have the option to make cdk8s synth print directly to stdout. This could be achieved my some kind of flag, e.g. cdk8s synth --print-to-stdout | kubectl apply -f

Import Helm Charts