okteto/okteto

Develop your applications directly in your Kubernetes Cluster

Okteto: A Tool to Develop Applications on Kubernetes

Overview

Kubernetes has made it very easy to deploy applications to the cloud at a higher scale than ever, but development practices have not evolved at the same speed as application deployment patterns.

Today, most developers try to either run parts of the infrastructure locally or just test their integrations directly in the cluster via CI jobs, or the docker build/redeploy cycle. It works, but this workflow is painful and incredibly slow.

okteto accelerates the development workflow of Kubernetes applications. You write your code locally and okteto detects the changes and instantly updates your Kubernetes applications.

How it works

Okteto enables development inside a container, providing a seamless IDE and tool integration as if you were working locally but with the resources of a remote cluster. When you run okteto up your Kubernetes deployment is replaced by a Development Container that contains your development tools (e.g. maven and jdk, or npm, python, go compiler, debuggers, etc). This development container can use any docker image. The development container inherits the same secrets, configmaps, volumes or any other configuration value of the original Kubernetes deployment.

The end result is a remote cluster that is seen by your IDE and tools as a local filesystem/environment. You keep writing code on your local IDE and as soon as you save a file, the change goes to the development container, and your application instantly updates (taking advantage of any hot-reload mechanism you already have). This whole process happens in an instant. No docker images need to be created and no Kubernetes manifests need to be applied to the cluster.

Why Okteto

okteto has several advantages when compared to more traditional development approaches:

• Fast inner loop development: build and run your application using your favorite tools directly from your development container. Native builds are always faster than the docker build/redeploy cycle.
• Realistic development environment: your development container reuses the same variables, secrets, sidecars, volumes as your original Kubernetes deployment. Realistic environments eliminate integration issues.
• Replicability: development containers eliminate the need to install your dependencies locally, everything is pre-configured in your development image.
• Unlimited resources: get access to the hardware and network of your cluster when developing your application.
• Deployment independent: okteto decouples deployment from development. You can deploy your application with kubectl, Helm, a serverless framework, or even a CI pipeline and use okteto up to develop it. This is especially useful for cloud-native applications where deployment pipelines are not trivial.
• Works anywhere: okteto works with any Kubernetes cluster, local or remote. okteto is also available for macOS, Linux, and Windows.

Getting started

All you need to get started is to install the Okteto CLI and have access to a Kubernetes cluster. You can follow our guide for setting up a Kubernetes cluster on AWS here.

The Okteto CLI has two operational modes:

• Okteto Open Source CLI
• Okteto Platform CLI

Okteto Open Source CLI Features

Okteto Open Source requires access to a Kubernetes cluster. It's designed to support Development Containers in any Kubernetes cluster. It doesn't come with features that support multiple developers working on the same cluster. That's the goal of the Okteto Platform

Okteto Open Source supports the following commands:

• okteto context
• okteto up
• okteto down

For reference, our Open Source CLI supports the dev section of the Okteto Manifest.

We have getting started guides for the Open Source mode for the following languages:

• ASP.NET
• Golang
• Java Gradle
• Java Maven
• Node.js
• PHP
• Python
• Ruby

Okteto Platform CLI Features

The Okteto Platform CLI requires installation of the Okteto Helm Chart in your Kubernetes cluster. In this mode, all of the Okteto CLI commands are available (build, deploy, up, down, destroy, etc). The Okteto Platform comes with additional features like:

• User authentication and access control to Kubernetes using your own Identity provider
• Build service for remote container image creation
• Preview environments for every pull request
• Dynamic scaling of environments based on usage
• Secrets manager for your development environments
• Okteto Insights to provide observability on your development environments

And much more! Refer to the Okteto Platform docs to learn more.

Features Comparison

Feature	Okteto Open Source CLI	Okteto Platform CLI
Development Containers	Available	Available
Build Service	Not Available	Available
User Management	Not Available	Available
Access Control	Not Available	Available
Automated Scaling	Not Available	Available
Secrets Management	Not Available	Available
Observability Tools	Not Available	Available
Support	Community Support	Professional Support
Documentation	Open Source Samples	Platform Docs

Useful links

• Getting started
• CLI reference
• Okteto manifest reference
• Okteto Open Source Samples

Releases

Okteto is monthly released into three channels: stable, beta, and dev. By default when Okteto is installed, the stable channel is used. If you need to access features not yet widely available you can install from the beta or dev channel. More information can be found in the release documentation.

Support and Community

Got questions? Have feedback? Join the conversation in our Community Forum. You can also join us in the #okteto Slack channel! If you don't already have a Kubernetes Slack account, sign up here.

Follow @OktetoHQ on Twitter for important announcements.

✨ Contributions

We ❤️ contributions big or small. See our guide on how to get started.

Thanks to all our contributors!

okteto's Issues

CND does not use KUBECONFIG variable

Describe the bug
CND does not use the KUBECONFIG environemnt variable and always looks in the user's home folder for the configuration

To Reproduce

export KUBECONFIG=/home/pau/Desktop/cndtest/test.config
cnd up
    Activating your cloud native development environment...
    Error: stat /home/pau/.kube/config: no such file or directory

Expected behavior
CDN should read from the environment variable first

Desktop (please complete the following information):

• OS: Ubuntu, Mac OS
• All versions?

Support for defining CND commands

To enable this scenario, the sync process should finish before starting the CND environment

'cnd down' crash

Describe the bug
cnd down crashed leaving the following trace:

cnd down
Deactivating your cloud native development environment...
warning: there are 2 cloud native development environments active in your current folder, using 'default/frontend'
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/okteto/cnd/cmd.findDevEnvironment(0xc000adfc00, 0x1, 0x1, 0x3a, 0x0, 0x0, 0x2, 0x2, 0x2)
	/private/tmp/cnd-20190107-22280-8elod0/src/github.com/okteto/cnd/cmd/exec.go:103 +0x502
github.com/okteto/cnd/cmd.executeDown(0x0, 0x0)
	/private/tmp/cnd-20190107-22280-8elod0/src/github.com/okteto/cnd/cmd/down.go:30 +0x84
github.com/okteto/cnd/cmd.Down.func1(0xc000a4a000, 0x2f7f7f8, 0x0, 0x0, 0x0, 0x0)
	/private/tmp/cnd-20190107-22280-8elod0/src/github.com/okteto/cnd/cmd/down.go:20 +0x22
github.com/okteto/cnd/vendor/github.com/spf13/cobra.(*Command).execute(0xc000a4a000, 0x2f7f7f8, 0x0, 0x0, 0xc000a4a000, 0x2f7f7f8)
	/private/tmp/cnd-20190107-22280-8elod0/src/github.com/okteto/cnd/vendor/github.com/spf13/cobra/command.go:762 +0x473
github.com/okteto/cnd/vendor/github.com/spf13/cobra.(*Command).ExecuteC(0x2f56e00, 0xc000adff88, 0x1007500, 0xc00009e058)
	/private/tmp/cnd-20190107-22280-8elod0/src/github.com/okteto/cnd/vendor/github.com/spf13/cobra/command.go:852 +0x2fd
github.com/okteto/cnd/vendor/github.com/spf13/cobra.(*Command).Execute(0x2f56e00, 0x0, 0x0)
	/private/tmp/cnd-20190107-22280-8elod0/src/github.com/okteto/cnd/vendor/github.com/spf13/cobra/command.go:800 +0x2b
github.com/okteto/cnd/cmd.Execute()
	/private/tmp/cnd-20190107-22280-8elod0/src/github.com/okteto/cnd/cmd/root.go:57 +0x2d
main.main()
	/private/tmp/cnd-20190107-22280-8elod0/src/github.com/okteto/cnd/main.go:22 +0x20

Previous to this cnd up was killed because of the error the following error:

rpc error: code = Unknown desc = Error: No such container: 24c661d7152e1914841cbf5ed8c79f0b22d80dede38736367534283b8244fd19

There are no CND environments enabled when the down is launched, maybe this is what causes the 'index out of range' error.

Desktop (please complete the following information):

• Version 0.4.0

Overwrite kubeconfig endpoint value

Is your feature request related to a problem? Please describe.
My KubeAPI is exposed through a jump box.

Describe the solution you'd like
Overwrite my kubeconfig cluster endpoint from okteto up to avoid modifying the original kubeconfig. With this, I can expose the k8 endpoints via a SSH tunnel, and still run my okteto commands locally.

Support for .cndignore

These files won't be synched.
We can just document how synching.net works

Add docs for how to use cnd when working in a team

When working on a development cluster, each team member should have limited access to the cluster to avoid modifying other team member workflows. This is more important if you are working in heterogeneous clusters with staging/production workloads.
We recommend having a namespace for each team member, where she can deploy her dev stack. To give access to a single namespace, you can make use of RoleBindings.
We might also want to document having a wildcard dns/cert for dev environments.

Okteto exec command freezes when there are connectivity issues

Describe the bug
cnd exec command freezes when there are connectivity issues, or when my mac goes to sleep. For example, if I run cnd exec sh, and then I turn off the Wifi, wait a minute, and turn it back on, the sheel freezes, and I can't exit it other than by killing the process.

To Reproduce
Steps to reproduce the behavior (using the vote sample)

1. cnd exec sh
2. Run a never ending command: while [[ 1 ]]; do date && sleep 1; done;
3. Sleep your mac
4. Wake up

Expected behavior
The command should exit cleanly on ctrl+c

Desktop (please complete the following information):

• OS: macos
• Version 0.4.4

Don't allow more than one `cnd up` from the same folder

Don't allow more than one cnd up from the same folder, as this causes issues with the file sync.

Test our windows support

We need to test that cnd works in Windows

Make "name" in cnd.yml optional

By default, it could be cnd-$DEPLOYMENT_NAME

'cnd up' crashed due to conversion issues

Describe the bug
I had cnd up running for a while. At some point, it crashed due to a panic.

Waiting for changes to input files of tasks...
panic: interface conversion: runtime.Object is nil, not *v1.Event

goroutine 66 [running]:
github.com/okteto/cnd/pkg/k8/deployments.GetPodEvents(0x222f800, 0xc000be2180, 0xc0011ad880, 0xc000b79200)
	/Users/ramiro/go/src/github.com/okteto/cnd/pkg/k8/deployments/deployments.go:133 +0x5a3
created by github.com/okteto/cnd/cmd.executeUp
	/Users/ramiro/go/src/github.com/okteto/cnd/cmd/up.go:85 +0x5bf

Desktop (please complete the following information):

• OS: macOS
• 0.4.1

Support for "swap" operator

`cnd down` doesn't kill previous `cnd up` process

Describe the bug
A cnd down on a different terminal doesn't kill the previous cnd up leaving the process hanging and giving the impression that it is still up and running.

To Reproduce
Steps to reproduce the behavior:

1. Open a terminal and do cnd up on your source.
2. Once cnd is enabled, open a different terminal and do cnd down.
3. The first terminal will be still showing this output:

> cnd up
Activating your cloud native development environment...
Linking '/movies-multi/movies-frontend' to ramon/movies/movies-frontend...
Ready! Go to your local IDE and continue coding!

Expected behavior
The process should've died and regained control over the terminal.

'cnd down' warns about 2 existing environments on current folder when there is only 1

I executed cnd down after a single cnd up and go the following output:

$cnd down
Deactivating your cloud native development environment...
warning: there are 2 cloud native development environments active in your current folder, using 'default/movies/movies'
Cloud native development environment deactivated

$cnd list
There are no active cloud native development environments

cnd up fails if there are symlinks in the source folder

Describe the bug
running cnd up con a folder where there are symlinks fails with the error below. It seems like the archiver library we are using to tar doesn't work well with symlinks.

DEBU[0004] cnd-sync init cointainer is now running, sending the tarball
ERRO[0008] failed to tar source folder: walking /Users/ramiro/go/src/bitbucket.org/okteto/cli: /Users/ramiro/go/src/bitbucket.org/okteto/cli/vendor/github.com/docker/docker/integration-cli/fixtures/https/ca.pem: opening: open /Users/ramiro/go/src/bitbucket.org/okteto/cli/vendor/github.com/docker/docker/integration-cli/fixtures/https/ca.pem: no such file or directory

Desktop (please complete the following information):

• macOS
• 0.4.4

Additional context
Add any other context about the problem here (e.g cnd.yml, k8s manifests, logs)

Do cnd binary releases

Error on first `cnd up` asking you to do `cnd up`

Describe the bug
First time cnd up is executed and error message is displayed asking you to do cnd up first. If you execute it again it will normally work.

To Reproduce
Steps to reproduce the behavior:

1. From command line go to the directory where cnd.yml is.
2. Execute cnd up.
3. Error message is displayed.
4. Execute cnd up again.
5. Everything works normally.

Expected behavior
cnd up should properly work the first time is launched.

Screenshots
Output shown:

> cnd up
Activating your cloud native development environment...
Error: cloud native environment is not initialized. Please run 'cnd up' first

Desktop (please complete the following information):

• CND version 0.3.4

Initialize synched folder from tar

This makes init sync faster and allows to create the CND container with a custom command

"rm" command

cnd down doesn't restore the original configuration

Describe the bug
cnd down doesn't restore the original configuration, it's calculating the staleness wrong and skipping the shutdown sequence

cnd down doesn't fully restore original configuration

Running cnd down doesn't stop the CND containers, only remaps the services to the original containers.

However, because cnd deployment has the same labels as the original deployment and services use selectors, services end up sending requests to both the original and the cnd version.

Support for relative paths in cnd.yml for mount:source

CND becomes very unstable on multi-tenant environments

This is similar to #46 . When more than one user is creating CND environments on the same kubernetes cluster, the synchronization process becomes very unstable.

We're currently looking into potential solutions.

Warning: kubectl apply should be used on resource created by either kubectl create --save-config or kubectl apply

Edit the already created service, instead of reading it from the file.

Reconnect port forward if connection breaks

Use filepath.Join instead of path.Join, since it's OS aware

Describe the bug
On Windows, the file paths show using forward slashes, instead of backward ones. While it's technically correct, it might interfere with other tools, and confuse users.

See #58 for more on this.

cnd exec doesn't work with multiple containers per pod

Okteto takes a long time to fail if there's no kubeconfig available

Describe the bug
cnd takes a long time to fail if there's no kubeconfig available. instead of failing immediately, it keeps retrying to activate the deployment for over 5 minutes. We should catch the auth issue and fail quicker.

To Reproduce
Steps to reproduce the behavior:

1. Rename your kubeconfig to something else
2. cnd up
3. The operation will be stuck on activating your environment for a while

The same happens if there's a bad namespace, or a bad deployment name

Expected behavior
I was expecting it to fail fast

Desktop (please complete the following information):

• OS: windows
• latest from master

ERROR: logging before flag.Parse

Describe the bug
When closing cnd up, sometimes the following will be logged:

ERROR: logging before flag.Parse: E0106 23:51:06.973164   14498 portforward.go:316] error copying from local connection to remote stream: read tcp6 [::1]:49193->[::1]:49203: read: connection reset by peer

To Reproduce
Steps to reproduce the behavior:

1. cnd up
2. Wait for it to be half done and then ctrl + c

Desktop (please complete the following information):

• macOS
• Version 0.4.0

Time out the code discovery phase of `cnd create`

Describe the bug
On very large repos, the code analysis we perform as part of cnd create (to detect the main language) can take a very long time. Yesterday, on a friend's repo (an ML-based app full of data and models) it would take minutes. And there wasn't any output while this was happening.

Expected behavior
I think that the create job should always be almost instantaneous. We should maybe limit the number of files we parse, or have a 1s timeout, and then do our best guess based on the information collected.

Desktop (please complete the following information):

• OS: macOS
• Version 0.4.4

Set k8 cluster and namespace in cnd.yml

They should be overridable via command args.

Fail if container to be swapped does not exist

"go build" with cache is still slow with Docker for Mac using CND

This might be due to Docker for Mac filesystem performance issues.

Can't run more than one CND environment at a time

When adding a second CND environment, there is a race condition, and only one of the environments is synchronized.

It seems like there's a race condition on the ksync api, we are into how to mitigate this. For the the time being, please only run one environment at a time.

Consider the OS when calculating syncthing's binary name

Describe the bug
On windows, the syncthing binary name is "syncthing.exe". We assume that it's always "syncthing"

To Reproduce
Steps to reproduce the behavior:

1. Start CND in windows

okteto up
Installing dependencies...
syncthing.exe 16.56 MiB / 16.56 MiB [======================================================================] 100.00% 7s
                                                           ✕  Exec: "C:\\Users\\dado_/.okteto/syncthing": file does not exist

Voting app demo with CND

Full experience demo using Docker for Mac or Minikube and explaining the namespace configuration.

Support to override resource's value

Is your feature request related to a problem? Please describe.
On production, a kubernetes deployment might have limited memory or CPU resources. On dev mode, I'd like to override these, as I might need to run resource-intensive tasks like compile, run tests, etc...

Describe the solution you'd like
Add a resources values to cnd.yml to allow overrides

Automatically create port-forwarders

Is your feature request related to a problem? Please describe.
Most of the time, when I launch a new service, I'll need to connect it. This means that I'm always launching port-forward sessions, or that I'll depend on load balancers or similar stuff.

Describe the solution you'd like
Be able to define a list of localPort:remotePort to automatically create a forwarding session.

Describe alternatives you've considered
I can manually run kubectl port-forward on the pod, but it's manually intensive.

Additional context
I typically launch two port-forwards, one for the service and another for the debugger.

Make the kubernetes namespace accessible in the dev container

Is your feature request related to a problem? Please describe.
A lot of dev scripts need to know the namespace they are running as (e.g. kubeapps uses it on their js code to validate acccess)

Describe the solution you'd like
Inject a "env context" into my dev container with the namespace.

Describe alternatives you've considered
I can add this to my deployment, but I feel that adding a value only consumed by dev on a manifest that's going to go to prod creates noise and potential issues

Add imagePullPolicy: IfNotPresent

For every swapped container, add imagePullPolicy: IfNotPresent container option.

Add support for "attaching" to two different services on the same deployment

Is your feature request related to a problem? Please describe.
In our application, we deploy an API server and the UI server as containers on the same deployment. At the moment it's not possible to use cnd on both containers at the same time.

Describe the solution you'd like
Have the ability to cnd up in multiple projects that are part of the same deployment (different containers).

Describe alternatives you've considered
Just use cnd on one project at a time, but it's too limiting.

Hook up CI

Mount the shared folder as a volume claim

CLI help displayed when command returns an error

2018/10/15 11:06:10 Waiting for the deployment 'cnd-backend' to be deleted...
Error: Kubernetes deployment not deleted after 50s
Usage:
  cnd rm [flags]

Flags:
  -f, --file string   manifest file (default "cnd.yml")
  -h, --help          help for rm

2018/10/15 11:07:00 ERROR: Kubernetes deployment not deleted after 50s

I think that for operational errores, we should log the error, return exit code 1, but not log the CLI help.

Support for "exec" command

Support for environment variables on cnd.yml

Is your feature request related to a problem? Please describe.
When I'm developing a feature, some of my tools require me to inject or override the value of dev environments (e.g. the ENV var in rails).

Describe the solution you'd like
I'd like to be able to set them in cnd.yml

Describe alternatives you've considered
This could be fixed by having a separate deployment.yml for dev, but that defeats the purpose of cnd

Sometimes `cnd up` wont output the container logs

Describe the bug
Sometimes cnd up wont output the container logs even if the logs are been produced (e.g. in GKE stackdriver)

To Reproduce
Steps to reproduce the behavior:

1. cnd up

This doesn't happen every time, seems like a race condition or an error that's not being handled somewhere.

Expected behavior
cnd up should show the logs of the command, or fail with an error.

Desktop (please complete the following information):

• OS: macos
• Version: latest from master

Additional context
This is the cnd.yml I'm using:

swap:
  deployment:
    name: kubeapps-internal-dashboard
    image: okteto/node:11
    command:
    - sh
    - -c
    - yarn install && yarn start
mount:
  source: .
  target: /usr/src/app
scripts:
  hello: echo Your cluster ♥ you

Logs:

➜  dashboard git:(cnd-demo) ✗ /Users/ramiro/go/src/github.com/okteto/cnd/bin/cnd up --loglevel=debug --namespace=kubeapps
Activating your cloud native development environment...
DEBU[0001] dev deployment is based of revision 18
INFO[0001] Updating deployment 'kubeapps/kubeapps-internal-dashboard'...
INFO[0004] Sending tarball...
INFO[0007] Tarball sent.
INFO[0014] Syncthing running on http://127.0.0.1:63352 and tcp://0.0.0.0:63353
Linking '/Users/ramiro/go/src/github.com/kubeapps/kubeapps/dashboard' to kubeapps/kubeapps-internal-dashboard...
Ready! Go to your local IDE and continue coding!

CND stops syncing my files

The connection to cluster is lost and ksync is suddenly shut down, as shown in .ksync/daemon.log:

time="2018-10-13T11:48:55+02:00" level=info msg="update complete" pod=cnd-frontend-5b466d966d-vdv7t spec=cnd-frontend
time="2018-10-13T11:49:19+02:00" level=error msg="lost connection to cluster" ContainerName=content LocalPath=/Users/rlamana/Repositories/okteto/frontend LocalReadOnly=false Name=cnd-frontend Namespace=ramon Pod= Reload=false RemotePath=/usr/src/frontend RemoteReadOnly=false Selector="[cnd=cnd-frontend]"
time="2018-10-13T11:49:19+02:00" level=info msg="signal loss dectected. shutting down"

Mention cluster and namespace on errors

Is your feature request related to a problem? Please describe.
When okteto can't find a deployment, it's not very clear which context it was used:

➜  okteto up                        
Activating your cloud native development environment...
Error: deployments.apps "website" not found

Describe the solution you'd like
It would be useful if the error message included the namespace and cluster / context name it used

This is related to #149

[email protected]