luizalabs/teresa

Open source tool to deploy apps to Kubernetes clusters

Teresa

Teresa is an extremely simple platform as a service that runs on top of Kubernetes. It uses a client-server model: the client sends high level commands (create application, deploy, etc.) to the server, which translates them to the Kubernetes API.

Client Installation

Download (recommended)

This is the best way to get the latest release.

• Access https://github.com/luizalabs/teresa/releases
• Download the latest release for your OS. Eg: teresa-linux-amd64
• Rename the download file to teresa. Eg: mv teresa-linux-amd64 teresa
• Make it an executable. Eg: chmod +x teresa
• Move it to the bin folder. Eg: sudo mv teresa /usr/bin

Then you're good to go 🙂 ! teresa should now be available to use on your terminal.

Homebrew

Run the following in your command-line:

$ brew tap luizalabs/teresa-cli
$ brew install teresa

Snap

Run the following in your command-line:

$ sudo snap install teresa-cli

Server Installation

Server requirements:

• Kubernetes cluster (>= 1.9)

• database backend to store users and teams (SQLite or MySQL)

• storage for build artifacts (AWS S3 or minio)

• rsa keys for token signing

• (optional) TLS encryption key and certificate

The recommended installation method uses the helm package manager, for instance to install using S3 and MySQL (recommended):

$ openssl genrsa -out teresa.rsa
$ export TERESA_RSA_PRIVATE=`base64 -w0 teresa.rsa`
$ openssl rsa -in teresa.rsa -pubout > teresa.rsa.pub
$ export TERESA_RSA_PUBLIC=`base64 -w0 teresa.rsa.pub`
$ helm repo add luizalabs http://helm.k8s.magazineluiza.com
$ helm install luizalabs/teresa \
    --namespace teresa \
    --set rsa.private=$TERESA_RSA_PRIVATE \
    --set rsa.public=$TERESA_RSA_PUBLIC \
    --set aws.key.access=xxxxxxxx \
    --set aws.key.secret=xxxxxxxx \
    --set aws.region=us-east-1 \
    --set aws.s3.bucket=teresa \
    --set db.name=teresa \
    --set db.hostname=dbhostname \
    --set db.username=teresa \
    --set db.password=xxxxxxxx \
    --set rbac.enabled=true

Look here for more information about helm options.

You need to create an admin user to perform user and team management:

$ export POD_NAME=$(kubectl get pods -n teresa -l "app=teresa" -o jsonpath="{.items[0].metadata.name}")
$ kubectl exec $POD_NAME -it -n teresa -- ./teresa-server create-super-user --email [email protected] --password xxxxxxxx

QuickStart

Read the first sections of the FAQ.

teresa's Issues

Implement a metrics collector process

We should be able to collect a timeseries for cpu, mem, etc. from all apps in order to alert the user of possible anomalies, for instance significant increases in resource consumption.

Command team list shows wrong info

When the user belongs to two teams only the last label is shown:

$ teresa team list
Teams:
  - team 2 (member)
    contact: [email protected]
  - team 2 (member)
    contact: [email protected]

How to specify tls.crt and tls.key

Hi, I have a self-signed ssl certificate. How should I specify the tls.crt and tls.key when install this chart?

Support app removal

Setup a CONTRIBUTING.md

Contributing files are a great way to help people to get started contributing in a project.

I have opened some issues today, but I felt a little worried about how they should be like.

I think that setting up a contributing file may help.

There are some examples I found that are nice, and you use as inspiration:

• This contributing from Nafaya, a github staff engineer
• Contributing from luizalabs/dev-guide (in pt-BR)
• Examples from github help doc

deploy command

It would be nice if the command teresa deploy ask to me what environment I want to run the command.

Create miniteresa for local development

Preferably based on minikube.

Support ClusterIP and NodePort services

For example for UDP services.

Support internal elb

.teresaignore doesn't works like .gitignore

Scenario

# .gitignore
composer.phar
vendor/

# .teresaignore
composer.phar
vendor/

When I run git status the folder vendor/ are not listed on my shell output, but when I do a deploy on Teresa, I see this message:

!     WARNING: Your Composer vendor dir is part of your Git repository.
       This directory should not be under version control; only your
       'composer.json' and 'composer.lock' files should be added, which
       will let Composer handle installation of dependencies on deploy.
       To suppress this notice, first remove the folder from your index
       by running 'git rm -r --cached vendor/'.
       Next, edit your project's '.gitignore' file and add the folder
       '/vendor/' to the list.
       For more info, refer to the Composer FAQ: http://bit.ly/1rlCSZU

So, Teresa are including the folder vendor on tarball for deploy.

---

Changing .teresaignore to this:

composer.phar
vendor

vendor folder are not include and the buildpack install my dependencies.

Rollback don't update envs in namespace annotations

Client times out but deploy continues in background

Mainly with java applications.

Check `cpu` and `memory` flags on `app create` command

To prevent Invalid Limit for cpu and memory limits smaller than max-cpu and max-memory

Deployment rollback

We should automatically rollback to the latest successful deployment when the health check detects
a failure.

[PROPOSAL] List deploys in reversed order

Currently, the last revision is shown at bottom of the table.
I think in most cases, we only need to know which was the last one, and it would be more easy to see if it is displayed first. Example:

+----------+---------+-----+------------------------------------------+
| REVISION | CURRENT | AGE |               DESCRIPTION                |
+----------+---------+-----+------------------------------------------+
| 22       | true    | 4d  | 48f4553 Bump version: 1.118.0 → 1.119.0  |
+----------+---------+-----+------------------------------------------+
| 21       | false   | 5d  | v1.118.0                                 |
+----------+---------+-----+------------------------------------------+
| 20       | false   | 10d | 5290ab57 Bump version: 1.116.0 → 1.117.0 |
+----------+---------+-----+------------------------------------------+
| 19       | false   | 12d | 226868ea Bump version: 1.115.1 → 1.116.0 |
+----------+---------+-----+------------------------------------------+
| 18       | false   | 12d | c00332b Bump version: 1.115.0 → 1.115.1  |
+----------+---------+-----+------------------------------------------+
| 17       | false   | 19d | c2d00239 Bump version: 1.114.2 → 1.115.0 |
+----------+---------+-----+------------------------------------------+

Sort env vars.

It would be nice if teresa app info myapp return env vars sorted.

Implement versioning based on linker -X flag

Related cli issue.

Support ssl for secure services

For https it should be done on the elb.

Support non web process types

Today only the web process is started. Suggestion: multi-container pods.

[PROPOSAL] Show current revision together with revision number

Currently, there is a column CURRENT with true/false values, which only one will be true.

I think would be nicer and easier to find which revision is the current if its displayed together with revision number.

Today:

+----------+---------+-----+------------------------------------------+
| REVISION | CURRENT | AGE |               DESCRIPTION                |
+----------+---------+-----+------------------------------------------+
| 17       | false   | 19d | c2d00239 Bump version: 1.114.2 → 1.115.0 |
+----------+---------+-----+------------------------------------------+
| 18       | false   | 12d | c00332b Bump version: 1.115.0 → 1.115.1  |
+----------+---------+-----+------------------------------------------+
| 19       | true    | 12d | 226868ea Bump version: 1.115.1 → 1.116.0 |
+----------+---------+-----+------------------------------------------+
| 20       | false   | 10d | 5290ab57 Bump version: 1.116.0 → 1.117.0 |
+----------+---------+-----+------------------------------------------+
| 21       | false   | 5d  | v1.118.0                                 |
+----------+---------+-----+------------------------------------------+
| 22       | false   | 4d  | 48f4553 Bump version: 1.118.0 → 1.119.0  |
+----------+---------+-----+------------------------------------------+

+--------------+-----+------------------------------------------+
| REVISION     | AGE |               DESCRIPTION                |
+--------------+-----+------------------------------------------+
| 17           | 19d | c2d00239 Bump version: 1.114.2 → 1.115.0 |
+--------------+-----+------------------------------------------+
| 18           | 12d | c00332b Bump version: 1.115.0 → 1.115.1  |
+--------------+-----+------------------------------------------+
| 19 (current) | 12d | 226868ea Bump version: 1.115.1 → 1.116.0 |
+--------------+-----+------------------------------------------+
| 20           | 10d | 5290ab57 Bump version: 1.116.0 → 1.117.0 |
+--------------+-----+------------------------------------------+
| 21           | 5d  | v1.118.0                                 |
+--------------+-----+------------------------------------------+
| 22           | 4d  | 48f4553 Bump version: 1.118.0 → 1.119.0  |
+--------------+-----+------------------------------------------+

what do you think?

Welcome app pods fail after setting env var

Create an app and set any env var.

Support monitoring hooks

So apps are automatically monitored. Requisites:

• decoupled from teresa itself
• monitoring system agnostic

Teresaignore is broken on Windows

The input files end up being encoded like this:

-rw-rw-rw- 0/0             896 2017-08-24 08:04 src\\routes.js.orig
-rw-rw-rw- 0/0             949 2017-08-24 08:04 src\\server.js
-rw-rw-rw- 0/0            3343 2016-03-12 10:44 swagger\\dist\\404.html
-rw-rw-rw- 0/0              18 2016-01-19 13:05 swagger\\dist\\CNAME
-rw-rw-rw- 0/0           34174 2015-10-28 19:28 swagger\\dist\\bower_components\\ace-builds\\src-noconflict\\ext-language_tools.js
-rw-rw-rw- 0/0            9992 2015-10-28 19:28 swagger\\dist\\bower_components\\ace-builds\\src-noconflict\\ext-searchbox.js
-rw-rw-rw- 0/0           12066 2015-10-28 19:28 swagger\\dist\\bower_components\\ace-builds\\src-noconflict\\ext-settings_menu.js
-rw-rw-rw- 0/0           24444 2015-10-28 19:28 swagger\\dist\\bower_components\\ace-builds\\src-noconflict\\keybinding-emacs.js

instead of:

-rw-rw-rw- 0/0             825 2017-08-31 08:25 /src/routes.js
-rw-rw-rw- 0/0             896 2017-08-24 08:04 /src/routes.js.orig
-rw-rw-rw- 0/0             949 2017-08-24 08:04 /src/server.js
-rw-rw-rw- 0/0            3343 2016-03-12 10:44 /swagger/dist/404.html
-rw-rw-rw- 0/0              18 2016-01-19 13:05 /swagger/dist/CNAME
-rw-rw-rw- 0/0           34174 2015-10-28 19:28 /swagger/dist/bower_components/ace-builds/src-noconflict/ext-language_tools.js
-rw-rw-rw- 0/0            9992 2015-10-28 19:28 /swagger/dist/bower_components/ace-builds/src-noconflict/ext-searchbox.js

Implement a cleanup process

We need for example a scheduler that periodically cleans up old build pods.

[HELM] Update chart version

On the repository the available version it's 0.7.0 while the current version is 0.15.0

Define DaaS implementation strategy

To implement #374, we have a few options:

1. Teresa using helm to manage resources through teresa resource create|delete ...
   • Positive:
     • Leverage maturity and features that already exists in helm, as well as all the charts
     • Quicker implementation
   • Negative:
     • Requires helm/tiller for teresa resource ... to work and it must be a specific version
     • Can't customize everything in teresa resource as we wish
2. Implement everything in Teresa
   • Positive:
     • Fully customizable
     • Avoid adding another dependency
   • Negative:
     • Longer implementation
     • Will have to rewrite it afterwards - as we'll most likely end up with less quality code than teresa have overall
     • Less features, less databases supported

Configure DNS for new apps

It can help
https://github.com/kubernetes-incubator/external-dns

Support Google Cloud Storage

Teresa API could support gRPC

gRPC could help on clients in differents languages, performance, bi-direction streams, etc.

Support the creation of databases as a service

Basically we should be able to do:

teresa resource create mysql --set foo1=bar1 --set foo2=bar2 ...

where the variables are optional customizations (disk size, mysql version, etc). We should avoid to hardcode resource templates and instead use a secure transport such as https to get them.

Tasks

• Create the templates and the transport infra (suggestion: use helm as a starting point)
• Create a client or adapt the current storage to get the resources templates
• Create the resource command. Requirements:
  • create an arbitrary number of component resources in a specific order
  • sane template defaults with support for overriding values
  • interactive with user friendly messages
  • rollback all creations on any failure

[PROPOSAL] Show deploy completion date and time

The deploy list command is really useful and I'm already using it sincce v0.8.0 has come.

But I think that showing the date and time the deploy completed may be more useful, mainly in troubleshooting when the EXACT date matters and when there are multiple deploys in the same day.

This is a real output for an app of mine:

+----------+---------+-----+------------------------------------------+
| REVISION | CURRENT | AGE |               DESCRIPTION                |
+----------+---------+-----+------------------------------------------+
| 17       | false   | 19d | c2d00239 Bump version: 1.114.2 → 1.115.0 |
+----------+---------+-----+------------------------------------------+
| 18       | false   | 12d | c00332b Bump version: 1.115.0 → 1.115.1  |
+----------+---------+-----+------------------------------------------+
| 19       | false   | 12d | 226868ea Bump version: 1.115.1 → 1.116.0 |
+----------+---------+-----+------------------------------------------+
| 20       | false   | 10d | 5290ab57 Bump version: 1.116.0 → 1.117.0 |
+----------+---------+-----+------------------------------------------+
| 21       | false   | 5d  | v1.118.0                                 |
+----------+---------+-----+------------------------------------------+
| 22       | true    | 4d  | 48f4553 Bump version: 1.118.0 → 1.119.0  |
+----------+---------+-----+------------------------------------------+

It would be nice if it is something like this (the COMPLETED AT column):

| ---------- | --------- | ---------------------| ------------------------------------------ |  
| REVISION   | CURRENT   | COMPLETED AT         | DESCRIPTION                                |  
| ---------- | --------- | -------------------- | ------------------------------------------ |  
| 17         | false     | 2017-09-01 01:00:00Z | c2d00239 Bump version: 1.114.2 → 1.115.0   |  
| ---------- | --------- | -----                | ------------------------------------------ |  
| 18         | false     | 2017-09-08 01:00:00Z | c00332b Bump version: 1.115.0 → 1.115.1    |  
| ---------- | --------- | -----                | ------------------------------------------ |  
| 19         | false     | 2017-09-08 01:00:00Z | 226868ea Bump version: 1.115.1 → 1.116.0   |  
| ---------- | --------- | -----                | ------------------------------------------ |  
| 20         | false     | 2017-09-10 01:00:00Z | 5290ab57 Bump version: 1.116.0 → 1.117.0   |  
| ---------- | --------- | -----                | ------------------------------------------ |  
| 21         | false     | 2017-09-14 01:00:00Z | v1.118.0                                   |  
| ---------- | --------- | -----                | ------------------------------------------ |  
| 22         | true      | 2017-09-15 01:00:00Z | 48f4553 Bump version: 1.118.0 → 1.119.0    |  
| ---------- | --------- | -------------------- | ------------------------------------------ |  

(I've changed the tabke format a bit, but it's just for formatting purpose)

The example is in iso format, but it may be formatted with local computer time.

Support http loadbalance

App info command doesn't reflect cluster state

The command teresa app info uses stale information from the annotation, it must always read from kubernetes.

Implement `teresa top pods`

Support pre and post build hooks

Useful for migrations for instance.

Teresaignore file breaks deploy on windows

For some reason the tar file has the full path of each file.

Parameter aws.s3.bucket being ignored during helm install while using Minio

When I set the parameter useMinio=true during helm install, I noticed that the parameter aws.s3.bucket is being ignored, and it's using it's default teresa

Improve security by not mounting the default service account

Add field automountServiceAccountToken: false to the podspec as soon as we migrate officially to k8s 1.7.

Setup instructions unclear

• teresa-api version: a21b7d6 (0.3.1?)
• K8s version: 1.5.2

I've followed all steps outlined on the teresa-api readme file, however the teresa-api pod fails with the following error message:

time="2017-04-27T00:43:30Z" level=fatal msg="the server could not find the requested resource (get secrets teresa-keys)"

Even though teresa-keys exist:

$ kubectl get secrets teresa-keys -n teresa
NAME          TYPE      DATA      AGE
teresa-keys   Opaque    2         2d

This is the deployment file:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: teresa
spec:
  replicas: 2
  template:
    metadata:
      labels:
        app: teresa
    spec:
      containers:
      - name: teresa
        image: cmgvieira/teresa:latest
        ports:
        - containerPort: 8080
        env:
          - name: TERESAK8S_HOST
            value: [MY K8S HOST]
          - name: TERESAK8S_USERNAME
            value: [MY K8S USERNAME COPIED FROM ~/.kube/config]
          - name: TERESAK8S_PASSWORD
            value: [MY K8S PASSWORD COPIED FROM ~/.kube/config]
          - name: TERESAK8S_INSECURE
            value: "true"
          - name: TERESAFILESTORAGE_TYPE
            value: s3
          - name: TERESAFILESTORAGE_AWS_KEY
            value: [MY S3 AWS KEY]
          - name: TERESAFILESTORAGE_AWS_SECRET
            value: MY S3 AWS SECRET]
          - name: TERESAFILESTORAGE_AWS_REGION
            value: us-east-1
          - name: TERESAFILESTORAGE_AWS_BUCKET
            value: [MY BUCKET NAME]
          - name: TERESADEPLOY_REVISION_HISTORY_LIMIT
            value: "5"
          - name: TERESADEPLOY_START_TIMEOUT
            value: 10m
          - name: TERESADEPLOY_FINISH_TIMEOUT
            value: 30m
          - name: NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace

Unless I'm missing something from the readme, I think it should be updated with clearer instructions.

Remove hardcoded timeouts

Mainly from the build process.

Use gRPC instead of go-swagger

Macro vision checklist:

• Login ( #185 )
• create user ( #192 #200 )
• delete user ( #190 )
• create team ( #189 )
• team list ( #197 )
• team add-user ( #193 )
• app create ( #207 )
• app env-set ( #213 )
• app env-unset ( #213 )
• app info (#208)
• app list ( #244 )
• app logs ( #205 )
• deploy ( #211 )

---

Extras:

• set-password command ( #185 )
• [server] create super user command ( #198 )
• TLS (some PRs of series)
• Release command phase ( #226 )
• Health Check ( #230 )
• Version by tag (git) ( #229 )
• minio slug storage ( #236 )

---

Refactorings:

• cli configs ( #188 )
• bash completion ( #187 )
• delete build pods after work ( #211 )
• standardize errors msgs format and server logs (server and client) ( #214 #218 )

---

Fun Stuff

• Unit Tests
• Travis CI (run unit tests against some golang versions ) ( #194 )

Server closes connection after approximately 60 seconds of inactivity

The logs endpoint for instance works by streaming content and needs a persistent connection.

Merge with teresa-cli

Mainly to avoid sync overhead.

Replace go-swagger code generation

Using go-swagger for code generation is causing more pain than gain (immature, opaque, etc.)

Support kubernetes token authentication

Use env var TERESAK8S_TOKEN instead of TERESAK8S_USERNAME and TERESAK8S_PASSWORD.

Support the creation of databases and caches as a service

`app info` command still crash with apps without `HPA`

Shell

As a teresa user
I want to access the shell from a pod
To run cli commands from my app

Deployment of docker images

Skip the building process completely.

Enable cross-zone load balancing for new services

Despite the fact that we try to maintain approximately equivalent numbers of instances in each enabled availability zone, cross-zone load balancing offers one more layer of protection.