deis/deis

Deis v1, the CoreOS and Docker PaaS: Your PaaS. Your Rules.

	This repository (deis/deis) is no longer developed or maintained. The Deis v1 PaaS based on CoreOS Container Linux and Fleet has been replaced by Deis Workflow which is based on Kubernetes.

Deis v1 PaaS

Deis (pronounced DAY-iss) is an open source PaaS that makes it easy to deploy and manage applications on your own servers. Deis builds upon Docker and CoreOS to provide a lightweight PaaS with a Heroku-inspired workflow.

New to Deis? Learn more about Deis Concepts, Architecture and how to Deploy an Application.

Installing Deis v1

Deis is a set of Docker containers that can be deployed anywhere including public cloud, private cloud, bare metal or your workstation. Decide where you'd like to deploy Deis, then follow the provider-specific documentation for provisioning.

Trying out Deis? Please follow the documentation on getting set up with Vagrant. Upgrading from a previous Deis release? See Upgrading Deis for additional information.

Troubleshooting

See the Troubleshooting Deis documentation for assistance with common issues.

Contributing

Interested in contributing to Deis? Check out our Open Roadmap and Planning Process or jump right into hacking on Deis and testing your Deis cluster.

License

Copyright 2013, 2014 Engine Yard, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

deis's Issues

Buildbot Code Coverage

We need code coverage reports on Buildbot.

Buildpack integration should be caching artifacts

Right now artifacts are not being cached using the Buildstep integration. If the deploy pull a lot of dependencies, it can be terribly slow. This should be achieved fairly easily using bind mounts to a cache directory in the bare git repo.

Refactor client/ dir as a django psuedo-app

Mostly so we can use ``./manage.py test client" and have a single test harness. This may involve renaming its deis package and/or moving its contents up one level.

Rotate command

Formation.rotate should allow cycling of nodes and containers. Rotating containers by 2 would cause 2 new containers to be created followed by the 2 oldest containers being removed. Later this infrastructure can be used to implement rolling, zero-downtime deploys.

Application SSL support

The controller currently supports HTTP and HTTPS cluster-wide by installing an SSL cert on the load balancer fronting Deis. We need to provide a way of distributing SSL keys to the router for applications.

Sphinx doc publishing workflow

We need a workflow for publishing auto-generated Sphinx docs to the <deis.io> website. We can start with something manual and automate it over time.

Users should be able to share Formations

As of now only the user who created the Formation can use the API to operate on it. This includes listing the formation, scaling it, git push'ing to it, etc. We need a way to share those permissions with some granular access controls.

Setup CI and test infrastructure for opdemand/deis-cookbook

We should have Foodcritic, ChefSpec and MiniTest (or equivalents) running through Travis CI. We'll need to spend some time designing and writing the tests.

Setup foodcritic for opdemand/deis-cookbook

Need to run foodcritic and update accordingly.

Provide feedback during long-running CLI commands

We need to provide users with some feedback during long running CLI commands like scaling nodes.

Integrate CLI docstrings with Sphinx

Need to write a client.rst that wraps the methods and their docstrings in a way that produces a command-line client reference. We can add the cheat sheet at the top.

Log aggregation

Each node needs to send its logs to the controller to facilitate a deis logs CLI command, that provides real-time log tailing across the formation.

Create "command-line cheat sheet" documentation

Django SECRET_KEY must be generated securely during controller provisioning

Right now the Django SECRET_KEY is a hardcoded attribute. We should allow it to be set via attribute, but default to a null attribute which auto-gens a unique key during provisioning.

Docstrings for API module

We need a first round of docstrings for Sphinx. This is also a good chance to do some basic code cleanup and reformatting.

Sphinx doc CSS and styling

We need the Sphinx docs styled appropriately for the <deis.io> website.

Deis controller doesn't have permission to delete Chef objects

A provisioned Deis controller has permission to bootstrap new systems, creating their Node and Client records on the Chef Server. However after the controller terminates the systems, it's not allowed to delete the Node and Client records it created through the bootstrap. The API calls result in a 403 error and therefore an error during Formation.destroy().

The workaround for now is to place the controller node in the Chef Server's admin group. However, there must be a more granular way grant permission to delete Node and Client records for objects that were bootstrapped by the controller. Looking for a better solution...

Use buildstep image on runtime nodes

The dream of using a stripped down runtime image on execution nodes has experienced a setback. When running a Buildpack app on a stock Ubuntu image, I'm playing whack-a-dependency with system libraries -- and losing badly.

For now we'll have to exec the containers on the same image they were built with. This means bundling the Buildstep image in the optimized AMI and making it part of the Chef recipe.

Reset south migrations

Before the open source release we should reset south migrations.

Travis CI Integration

We should explore whether or not a Travis CI integration makes sense.

CLI list commands only return 10 results

Django Rest Framework is using pagination to restrict results to 10. This makes for strange behavior like:

$ deis containers:scale web=12
$ deis containers
=== web: `gunicorn -b 0.0.0.0:$PORT app:app`
web.1 up 2013-07-27T20:37:57.994Z
web.2 up 2013-07-27T21:54:53.238Z
web.3 up 2013-07-27T21:58:06.082Z
web.4 up 2013-07-27T21:58:06.095Z
web.5 up 2013-07-27T22:02:57.508Z
web.6 up 2013-07-27T22:02:57.525Z
web.7 up 2013-07-27T22:02:57.540Z
web.8 up 2013-07-27T22:02:57.555Z
web.9 up 2013-07-27T22:02:57.572Z
web.10 up 2013-07-27T22:02:57.590Z

This is clearly a bug, but putting it off for now. There's probably a flag we can pass to the HTTP request to prevent this.

Fix Buildpack compile/release and caching

We've seen some issues with Buildstep-included Buildpacks.

• Some Buildpacks fail to build slugs
• Some Buildpacks don't output proper config_vars on release (e.g. PATH, GEM_PATH)
• Some Buildpacks don't cache artificats (e.g. Heroku's Node.js buildpack)

Create "how to contribute to deis" documentation

Create "community structure and code of conduct" documentation

Config update should take multiple values

Right now config:set only takes one value at a time. New config values cause a new Release to be rolled, which results in a converge of the Formation. When updating multiple config values, this can take a painfully long time.

We need config update to take multiple values at once, so there's only a single converge.

Update project README with Getting Started, etc.

We need to update the main project README to include Getting Started code snippets, links out to documentation and other basic information.

Create "How to run and hack on deis" documentation

CLI registration URL requires trailing slash

We need to do a proper URL join to make sure trailing slashes aren't relevant.

API docstring and doc layout improvements

• Finish all API docstrings with internal cross-references
• Reorder the API nav by modules

"pip install deis" should install the client locally

Deis Tutorial

Besides the "quick start" documentation, we need a more comprehensive tutorial on how to use Deis once the controller is up and running.

I'm envisioning a walkthrough that describes using the CLI to deploy and manage real-world formations, while explaining key concepts and providing a deeper look at the CLI.

Maybe something similar to:
https://docs.djangoproject.com/en/dev/intro/tutorial01/

Layer implementation and tests

Formations should be composed of layers, which host nodes of the same type and configuration. Layers (and their nodes) can be scaled up and down independently.

Clean up code to be PEP8-compliant

Clone proper gitosis tag

Right now we're using master by default, we should be using standard release tags.

Containers must not run as "root" user

Containers need to run as a non-root user for LXC best practice.

Node deletion

The API should allow users to delete an individual node by its ID. Deis provisioned the node, so in the event it started misbehaving, we shouldn't force users to go to the cloud provider to kill it.

Move SSH keys to formation

SSH keys are currently attached to the Flavor object. We need to move that to the Formation for security.

0.0.5

Implement and document host hardening measures

A formation's nodes need to be running IPTables and include other system hardening tactics for Ubuntu servers. This will be implemented via Chef recipes.

Fix coverage message in READMEs

It looks like we have an issue with Crate.io's widget. It's coming up "unknown" even though the coverage figures are there.

Publish AMIs with documentation

In order to facilitate fast deployment of controllers and nodes, we need to provide pre-built AMIs that include time-consuming components. AMIs will be based on stock Ubuntu 12.04 LTS and include:

• 3.8 kernel (for proper AUFS support)
• Ruby/Chef clients and dependencies
• Large docker images (e.g. buildstep)

We must provide the script used for bundling to provide complete transparency and allow others to recreate the bundles themselves.

Deprecate unused Django models, settings and apps

Need to deprecate unused models and take another pass at cleaning up any DB schema issues.

Clean up git push workflow

The logic around handling a git push needs some love.

Create "what is the deis release process?" documentation

Pick your release workflow

Deis administrators need the ability to pick their release and have control over rolling out new Deis features. The current plan for this is to have the Deis admins swap the Deis cookbook version in Chef land. This should cause Deis to pull updated tags.

CLI completion and cleanup

We need to complete the supported CLI features and generally cleanup the code to handle contributions.

Web container should be scaled on initial `git push`

When pushing to Heroku, an initial web container is created automatically. This gives you a chance to test out your application without having to scale processes, making for a more streamlined initial workflow.

We need to add that same optimization to Build.push.

Default Providers and Flavors

New users should have Providers and Flavors pre-seeded with sensible defaults. When using the CLI to provide credentials, the API will simply update the default Provider records with the credentials. Users will still be able to add/manage additional sets of credentials.

Re-enable Django admin

We need to re-enable Django admin and configure it correctly according to the new model design.

Formation SSL support

Formations need a way to expose HTTPS and raw SSL sockets.