koding/koding

The Simplest Way to Manage Your Entire Dev Infrastructure!

Koding

The Simplest Way to Manage Your Entire Dev Infrastructure!

Koding is a development platform that orchestrates your dev environment. Developers get everything they need to spin up full-stack, project-specific environments in seconds. Share them, update them, and manage infrastructure from a simple interface.

You can try Koding now on koding.com

Quick Start with Docker-Compose

Easiest way to run Koding is to install docker-compose which can be found here. For the rest you can follow these steps:

git clone https://github.com/koding/docker-compose.git koding-docker-compose
cd koding-docker-compose
# Requires docker-compose version >= 1.6
docker-compose up -d

Now you are able to access Koding via port 8090 (e.g. localhost:8090) on your host.

Run Koding on Koding.com

Yes, you can run koding on koding.com by using the provided .koding.yml

For more information about stacks: koding.com/docs

Getting started for Development

You need to install following software packages to run Koding:

• git
• docker
• docker-compose

Start developing

You are now ready to run Koding.

git clone https://github.com/koding/koding.git
cd koding
docker-compose -f docker-compose-init.yml run init
docker-compose up

If you don't have a powerful computer, this may take a while at first, slow computers may take up to 15 minutes before they build the entire system. Please be patient. Once it is up and running, everything will be smooth and very fast.

Now you can navigate to http://localhost:8090 to see your local Koding instance. Enjoy! (If you don't see it, keep waiting, it will show up)

When you edit files on your host computer, they will be visible in the runtime environment. Watchers will automatically restart backend workers, re-compile frontend code. You don't need to do anything for it.

Tips

If you need to execute some commands in runtime environment, here is how you can start a shell in backend service container:

docker-compose exec backend bash

You can follow coffeescript-styleguide that we are relying on.

Running Koding on Local Machine

This is if you don't want to do docker-compose way and install everything locally, (not recommended).

Software Prerequisites

• Go v1.7
• Node.js v0.10
• CoffeeScript v1.8.0
• Supervisor

Start developing

Follow these steps for running the instance:

git clone https://github.com/koding/koding.git /your/koding/path
cd /your/koding/path
node -v # make sure your node version is not greater than `0.10.x`
npm -v # make sure your npm version is 2.15.x
coffee -v # make sure your coffeeScript version must be 1.8
npm install

You should have packages ready for running build specific scripts.

./configure # create necessary config files
./run install # start to install dependencies
./run buildservices # build the services
./run # run all services

As a result, you will have a file watcher watching your backend files (both node, and golang) and restart services when it's necessary. Now open up another terminal and run following commands:

cd /your/koding/path
cd client # move into frontend client folder
npm install # install client dependencies
make # this will run a client watcher for you

Right now you should have 2 different watchers for (1) your backend files, (2)for your frontend client files. Now you can navigate to to see your local Koding instance. Enjoy!

License

This repository is licensed under GNU AGPL V3 Koding Community Edition is licensed under Apache 2.0

Contribute

The main purpose of this repository to continue evolve Koding in order to make it more stable and create the best development experience ever. If you're interested in helping with that, please check our open issues. You can also join the conversation in our slack team!

koding's Issues

User is leaving the collaboration session without a confirmation when he leaves session from sidebar.

http://recordit.co/AAuGD2u1Yv @usirin

Add Your own Credentials

Goals

• Make team stacks independant from the credentials.
• It should ask for credentials when you try to build a stack
• User should be able to select from the shared credentials or create his own credentials

Why
Users need to provide credentials in order to be able to build their stacks. But a team admin may not be willing to share his own credentials with the whole team. This feature will allow users to build a given stack with their own credentials.

Acceptance Criteria

• Is user able to provision a stack by using Vagrant on his own device?
• Is user able to select his own aws keys?

Stack IDE v0.2

Goals
To update stack editor design and flows based on the new prototypes.

Why

Acceptance Criteria

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Check service connectivity

Start workers with supervisor in run script

Local Provisioning Support

Goals

Why

Acceptance Criteria

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Tray Icon

Goals

Why

Acceptance Criteria

• Can you turn on/off VMs?
• Can you mount from Tray Icon?
• Can you work on your local IDE from Tray Icon by opening with Sublime/Eclipse etc?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Docker Specialization

Goals

Why

Acceptance Criteria

Relations
#8161 needs to be done.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Google Cloud Engine Support

Goals

Why

Acceptance Criteria

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Global Hackathon Marketing Flowchart

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Open Source Koding v0.1

Goals
To be able to make Koding source and development flow public to gain more developer resource and gain developer love from the community

Why
Our main focus group are developers and we need more developer resources which can be achieved effectively once we open source Koding.

Acceptance Criteria

• Is Koding repository open-sourced?

Space Race Marketing Campaign Flowchart

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Analytics Dashboard v0.1

Goals

Why

Acceptance Criteria

• Can you see the high priority metrics for every team?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Open Source Koding v0.2

Goals
Having Guidelines, Documents and Settings for open development model

Why
Making system ready for community contributions

Acceptance Criteria

• Did we switch to open development model?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Adding Plain email address as a participant to Private Chat and Accept email replies

Make configure and run script inherit environment variables

• KONFIG_JSON fields should not override corresponding environment variable #7756

Inherit KONFIG_PROJECTROOT in environment variables

Backend Cost Optimizations

Goals
savings from the cost of
• test environments (wercker, travis, test machines)
• development environment
• production environment
• database

Why

Acceptance Criteria

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

'Remove credential' is not working.

http://recordit.co/5f3oylWOeD

'Cannot read property 'Remove' of null' error is displayed on the console when you try to remove a credential. It doesn't matter if it's bootstrapped or not.

error log - http://take.ms/DD5HQ - http://take.ms/mgojK

Koding Deployment Process

Historical Data (Dynamic Google Spreadsheet Image, updated daily)

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Create koding/koding repository on Docker Hub

• Add Dockerfile #8225
• Publish master revision to docker hub

hello test

shall we put a rule to have a comment block for each use of `defer`?

Title says it all. What do you think? cc/ @koding/frontend @cihangir

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Introduce standalone docker-compose distribution

Run development environment with docker-compose

• Container bootstrap script
• docker-compose file
• Remove service container builds in run script

Which client side apps are we using right now?

Since we are in active development these days, we have lots of legacy code moving around in our codebase making it harder to make further refactors since, even though we have a really small set of features, those features are in various places.

I am creating this issue to identify what parts are really being used in which app. Hopefully we can come up with the parts we are not using anymore and remove the remaining parts asap.

Here is the current folder structure

client
├── about
├── account
├── ace
├── activity
├── admin
├── app
├── assets
├── builder
├── dashboard
├── features
├── feeder
├── finder
├── home
├── ide
├── kites
├── landing
├── legal
├── members
├── mocks
├── pricing
├── stack-editor
├── stacks
├── teams
├── test
├── testrunner
└── viewer

Actively used apps are right now:

client
├── app
├── dashboard
├── finder
├── home
├── ide
├── landing
├── stack-editor
├── stacks (parts of it are being used in stack-editor)
└── testrunner

I think we can reduce down all of our apps into these ones, and move all the required logic from other apps under these ones. (move stack related ones into stack-editor, etc..)

I can take the job of creating necessary PRs to make those changes once we identified the files/modules required.

/cc @koding/frontend

docker-compose environment

Goals
Koding should be easy to run and customize via docker containers

Why
Docker container is the easiest solution to run a program. docker-compose will make it easy to setup and run koding for both development and production purposes

Acceptance Criteria

• Does koding run as a container in development environment?
• Does koding development environment can be run with docker compose?
• Does koding run as a standalone docker compose package?
• Is koding container customizable with setting environment variables?

Getting more done in GitHub with ZenHub

Hola! @gokmen has created a ZenHub account for the koding organization. ZenHub is the only project management tool integrated natively in GitHub – created specifically for fast-moving, software-driven teams.

---

How do I use ZenHub?

To get set up with ZenHub, all you have to do is download the browser extension and log in with your GitHub account. Once you do, you’ll get access to ZenHub’s complete feature-set immediately.

What can ZenHub do?

ZenHub adds a series of enhancements directly inside the GitHub UI:

• Real-time, customizable task boards for GitHub issues;
• Multi-Repository burndown charts, estimates, and velocity tracking based on GitHub Milestones;
• Personal to-do lists and task prioritization;
• Time-saving shortcuts – like a quick repo switcher, a “Move issue” button, and much more.

Add ZenHub to GitHub

Still curious? See more ZenHub features or read user reviews. This issue was written by your friendly ZenHub bot, posted by request from @gokmen.

PR Review Process

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Azure Support

Goals

Add support for azure and azurerm terraform providers to terraformer and kloud.

Why

Required support for Azure Cloud in stack templates.

Acceptance Criteria

• it is possible to build an Azure stack.

Staging and Testing

Goals

Why

Acceptance Criteria

• Is the required environment created and ready for testing on Rainforest when a PR is opened?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Smart KD Mount Recommendation

Goals
To improve the mounting UX via a simple kd mount style UX

Why
The settings for mounting can be a bit non-obvious to the end user, Devrim wanted to simplify that UX

Acceptance Criteria

• Does KD recommend you the available options without waiting for you to specify anything?
• Does it check the available space on the local drive?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Tests - Mid-Level List

Goals

Why

Acceptance Criteria

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

"Deployment from Sandbox" Process Flowchart

Tests - High Priority List

Goals

Why

Acceptance Criteria

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Koding Process

KD Open

Goals
To add kdopen functionality to KD

Why
Reducing the separate tooling, and increasing utility of KD

Acceptance Criteria

• Can you type 'kd open' on your local terminal and open your local files?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Farewell Solo

Goals

• Savings from the amount spent on free VMs
• Make a code cleanup
• Only teams focus

Why
Our target audience has changed and code-base have some issues for supporting both solo and teams products together.

Acceptance Criteria

• Is Koding AWS account deleted?
• Is 'Solo dependent' code cleaned up from main repository?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

If a new stack template is applied to the team and old stack template is deleted, members using old stack template cannot reinitialize stack.

'Cannot read property 'on' of null' error is displayed on the console every time that the member tries to reinitialize.

error log - http://take.ms/rSFg7

Supervisor should inherit environment variables

Change Provision button label to `Initialize` in stack editor.

Team Dashboard - Members Tab

Listing

To be able to list users, one should use fetchMembers function

selector = {} # dont use any selector, it is legacy.
options = {} # options is for skip, limit parameters
_globals.currentGroup.fetchMembers(selector, options, console.log.bind(console))

To be able to list roles of a user one should use following API:

accountIds = ["5550f42290fc1e173c70961b"]
_globals.currentGroup.fetchUserRoles(accountIds, console.log.bind(console))

response will one object per user-role 
{
   as: "member"
   targetId: "5550f42290fc1e173c70961b"
}

To be able to list admins of a group

selector = {} # dont use any selector, it is legacy.
options = {} # options is for skip, limit parameters
_globals.currentGroup.fetchAdmins(selector, options, console.log.bind(console))

To be able to list moderators of a group

selector = {} # dont use any selector, it is legacy.
options = {} # options is for skip, limit parameters
_globals.currentGroup.fetchModerators(selector, options, console.log.bind(console))

Blocking user will only be shown to koding group for now

Search

Search will search on email and nickname of the users.

_kd.singletons.search.searchAccounts("cs", {restrictSearchableAttributes:["nick", "email"]}).then(console.log.bind(console))

by default email is not searchable for normal users, to be able to search with email, one should pass restrictSearchableAttributes:["nick", "email"] as option to account search.

• we dont provide searching with first and last name right now
• algolia doesnt support having sort on query params on query time, we need to 4x plan if we really need search ordering.

Moderation

We can update user's role in a specific group with following api:

targetId : "5550f42290fc1e173c70961b"
newRoles:  ["admin", "moderator"]
_globals.currentGroup.changeMemberRoles(targetId,newRoles, console.log.bind(console))

beware that newRoles will override existing ones.

applicable roles are following ones : ['admin','moderator','member','guest']

to be able to kick a member from a group, one should call following api

targeAccounttId : "5550f42290fc1e173c70961b"
_kd.singletons.groupsController.getCurrentGroup().kickMember(targeAccounttId, callback)

Moving towards a new Frontend Builder

Hello,

Following is the builder's requirements to work. I tried my best to describe each step in words, so that we can discuss and come up with better ideas, and mostly because if we know the requirements of it right now we can move to a new one with confidence and ditch bant once and for all.

/cc @koding/frontend

---

• Build Sprites
• Build Styles
• Build Scripts
• Copy Assets
• Copy Third Party

Build Sprites

Sprite Specification

• Each app declares a sprites array in their bant.json. (the default is lib/sprites for each app right now.)

// bant.json
{
  "sprites": [
    "./lib/sprites"
  ]
}

• Each entry in sprites array consists of 2 folders:
  • lib/sprites/1x
  • lib/sprites/2x
• For each file in 1x folder, there should be another file present in the 2x folder with the same name:

# if these two doesn't present at the same time, builder should throw.
lib/sprites/1x/foo.png
lib/sprites/2x/foo.png

Sprite build step

• Builder goes through each app's sprites folders that has been defined in the sprites array in bant.json
• Creates a 1x css spritesheet and saves it under opts.spriteTmpCssOutdir
• Creates a 2x css spritesheet and saves it under opts.spriteTmpCssOutdir
• Creates a 1x sprite image and saves it under opts.spriteImgOutdir
• Creates a 2x sprite image and saves it under opts.spriteImgOutdir
• It watches each sprites folders of each app, and rebuilds sprites for each change event.

Build Styles

Style Specification

• Each app declares a styles array in their bant.json

// bant.json
{
  "styles": [
    "./lib/styl/*.styl",
    "./lib/components/**/styl/*.styl"
  ],
}

• Each entry in styles array can be a glob relative to the root folder of app folder.
• Each entry in styles array corresponds to a file or list of files that can be compiled via stylus compiler.

Style build step

• Copies over KD Framework css file.
• Includes STYLES_COMMONS_GLOB ('app/styl/**/*.styl') (for each app :/)
• Goes through each app's style files that has been defined in the styles array of bant.json
• Creates a single css file for each app under opts.stylesOutdir/<app-name>.css
• If builder is running watch mode for styles:
  • It watches files under STYLES_COMMONS_GLOB
  • When a change event happens with globals, it rebuilds all styles.
  • It watches each file that is described in the styles array of bant.json of each app.
  • When a change event happens under single file, it rebuilds individual app style.
• If builder requires sprites compilation:
  • It looks for each spritesheet file under opts.spriteTmpCssOutdir
  • Includes each spritesheet file into compiled css.

Build Scripts files

• Each app declares a main key which corresponds to a file that is relative to the app root folder.

// bant.json
{
  "main": "lib/index.coffee"
}

• The file in main key will be used as an entry file for bant compilation.

Scripts build step

• It provides a way to require each app with its name with absolute require paths.

# We have 2 files
# client/activity/lib/foo.coffee (A) file under 'activity' app.
# client/ide/lib/bar.coffee (B) file under 'ide' app.

# (B) can require (A) by using absolute paths, and vica-versa.

# client/ide/lib/bar.coffee
# no need to do require '../../../activity/lib/foo'
foo = require 'activity/foo'

• It creates a rewrite map that removes the necessity of using lib for requiring single modules under <app-name>/lib folder.

# no need to write down `require 'activity/lib/foo'`
foo = require 'activity/foo'

• It compiles CoffeeScript.
• It imports CoffeeScript files without the necessity of .coffee extension.

# no need to require like: require './foo.coffee'
foo = require './foo'

• It compiles JSX.
• It compiles Pistachio.
• It minifies output JavaScript file if opts.minifyJs
• It provides below globals to the browser:
  • All app modules under globals.modules
  • Remote api schema under globals.REMOTE_API
  • Ace library path under globals.acePath
  • Ace library config under globals.aceConfig
• It dedupes the kd.js modules. (wtf?)
• It extracts sourcemaps if opts.extractJsSourcemaps option is true.
• If watch mode is enabled:
  • It watches over each required file starting with entry files of the each app.
  • When a change happens in the dependency tree, it rebundles.

Copy Assets

Copy assets builder step

• Copies assets under opts.assetsDir to opts.assetsOutdir with the same folder structure.

Copy Thirdparty

Copy thirdparty builder step.

• Copies files and folders under opts.thirdpartyDir to opts.thirdpartyOutdir with the same folder structure.

---

Here are my suggestions:

1. Remove bant, use webpack

• No maintenance cost for our "own" builder.
• webpack is widely supported by the developers from big companies: facebook, twitter, etc.
• webpack supports awesome debugging tools out of box, hot-reloading
• webpack is the defacto builder in React community.
• Widely support plugins, loaders.
• Loaders for bundling images, style files, etc.
• It automatically supports creating optimized chunks out of builds, and loads them asynchronously.

If we switch to webpack requiring a style file for a component will become like this:

require './fooview.styl'

• This way we can tell webpack to create chunks, and only serve the required js and css files on initial load, and the loading the rest can be deferred.

2. Also @cihangir had some ideas about about uploading sprites to a cdn, and remove that task out of builder as well, I am supporting it but i don't have any clue how it should be implemented, any idea is appreciated here.

Any feedback/help will make our lives as frontend engineers way much easier, cause waiting for 284712894 seconds for a simple js file change shouldn't be a normal practice. 👻

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Video Collaboration Feature Progress

The feature works like the following:

• Client connects to TokBox servers with the generated tokens (Initialization and connecting to TokBox servers).
• There is no request for inviting people to join to video chat, but instead a public channel that you can connect with your video.
• You can switch between other users' videos by clicking their avatar's from the chat window.
• There is an (X) amounts of user limit. (This requires a decision /cc @sinan)
• When the admin/owner is free user, the video chat is limited to 2 people.
• User videos switch automatically when a user is speaking.
• You can Mute/Unmute yourself.
• Session owner can mute/unmute everybody.
• You can Open/Close your camera.
• You can hide/show chat window when you are in video chat.

There will be following tasks and related PR links with them.

The flow

There are 2 endpoints: (This part requires attention from @cihangir and @canthefason)

• /-/video-chat/token is for generating an access token for other TokBox operations.
• /-/video-chat/session is for session generation process from TokBox for them to keep track of the process.

Initialization and connecting to TokBox servers (Tasks are here for reference, those will be redefined)

• Create VideoChatModel by passing the necessary SocialChannel instance.
• VideoChatModel will kick off OpenTokService.
• VideoChatModel will create a new VideoChatView instance and attaches to itself for lazy-access.
• OpenTokService acts a wrapper class around opentok.js.
• VideoChatModel connects to a opentok session.
• VideoChatModel shows video chat via VideoChatView
• VideoChatModel switchs to given user's video.
• VideoChatModel provides an api for mute/unmute yourself.
• VideoChatModel provides an api for mute/unmute others for admins.
• VideoChatModel controls the amount of people can join to video session? (This may be extracted into its own class)

KD SSH

Goals
To enable ssh features of KD for managed machines

Why
SSH isn't supported on managed machines, but many commands in kd use ssh (kd mount, kd cp, etc). Users need ssh on managed machines.

Acceptance Criteria

• Can you SSH into a managed machine?

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

250 TB Week Flowchart

@fatih please check 3rd box VM reconfiguration - let's keep that in mind to get it done once we have time.

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Gitlab Integration v0.1

Goals
Bundle Koding with GitLab so we can reach more users easily

Why we do?
GitLab and Koding have same target audience which providing both services together makes sense.

Acceptance Criteria

• Does Koding get installed with Gitlab?
• Is 'Run This Project' working?
• Did GitLab approve?

Create koding/docker-compose code repository

Digitalocean Support

Goals

Why

Acceptance Criteria

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

Tunnelling Improvements

Goals

Why

Acceptance Criteria

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.