lastbackend/lastbackend

System for containerized apps management. From build to scaling.

The Open-source container management platform

Last.Backend container management platform is the new and modern open-source container management system with service discovery, overlay networks and more..

Table of contents

1. About
2. Key Features
3. Runtime design principles
4. Endpoint Interface
5. Current State
6. How to get started
7. Maintainers
8. Roadmap
9. Community
10. Authors
11. License

---

About container platform

Container platform is an open-source system for automating deployment, scaling, and management of containerized applications. It contains a set of technologies like: container scheduling, service discovery, overlay network, container runtime, container images runtime, load-balancing and more. All of these technologies provided out of the box and don't requiring additional research and setup.

Platform structure:

• REST API server
• State controller
• Resources Scheduler
• Node agent
• Service discovery
• Ingress proxy
• Log collector
• IPVS-based load-balancer
• VxLAN-based overlay network
• Storage runtime driver
• Container runtime
• Container images runtime

Key features

1. Fast application deploying to any server
2. Easy application sharing
3. Easy application management
4. Deploying application by url
5. Deploying scheduling
6. Deploying stateful services.
7. Developer-friendly CLI

Runtime design principles

Our design principles allows us to create extendable and powerful system. We separated runtime into particular package and use interfaces to add ability to extend supported technologies. By default Last.Backend operate with this runtimes:

• CRI - container runtime interface: docker by default
• CII - container image interface: docker by default
• CSI - container storage interface: host directory by default
• CNI - container network interface: vxlan by default
• CPI - container proxy interface: IPVS by default

All these runtimes are documented in runtime section, where are described all methods, types and algorithms.

Endpoint interface

The main endpoint to manage cluster is REST API interface. Our team use swagger for generation API documentation. To create swagger spec, just execute make swagger-spec command in root of repository.

You can use REST API in these options:

• directly with CURL or another apps
• using Last.Backend CLI (located in separate repo lastbackend/cli)
• for building custom go apps - you can use golang client located in pgk/api/client package

Current state

Current version is in public beta stage and include:

• cluster management
• node management
• overlay network based on vxlan
• internal endpoints for pods balancing based on IPVS
• ingress server based on haproxy
• internal discovery server
• services management with basic blue/green deployments
• volumes management

All of these functionality is under active test now, so don't surprised by frequent PRs please.

Join us in Gitter This project has Roadmap, feel free to offer your features.

We are actively searching for contributors! If you want to help our project and to make developers life easier, please read our Contibuting guideliness.

How to get started

If you want to dive into project, the best place to start - is our documentation.

Maintainers

We have separated maintainers page

Note: Last.Backend is under active development stage and our team is working day and night to make it better. Your suggestions, comments and contributions will be very helpful for us!

Roadmap

For details on our planned features and future direction please refer to our roadmap.

Community

Join us on social media:

• Twitter
• Facebook
• Stackshare
• AngelList
• LinkedIn
• [email protected]

Repository owners

The repository is owned by Last.Backend LLC company.

The main contributors are:

• Alexander: https://github.com/undassa
• Konstantin: https://github.com/unloop

---

License

Origin is licensed under the Apache License, Version 2.0.

lastbackend's Issues

Update user package method

Update user package method

Should update user data into kubernetes

User update method

func (c *users) Create(user *du.User) (result *du.User, err error) {
    result = &du.User{}
    err = c.Put().Resource("users").Name(user.Username).Body(user).Do().Into(result)
    return
}

User structure

type User struct {
    ku.TypeMeta
    k.ObjectMeta

        Username string
        Email string
        Pass string
        Salt string
        Created time
        Updated time

    Identities []string
}

Where:

• k - kubernetes api version (current - v1 package)
• ku - kubernetes unversionned api types
• du - deploy it user types

vagrant test box provision failed

Description

While running vagrant up, i have this strange error message:
==> default: Step 9 : ADD ../.. /go/src/github.com/lastbackend/lastbackend
==> default: Forbidden path outside the build context: ../.. ()
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

Version

[$ deploy version]

How To Reproduce

1. vagrant up

any idea what went wrong ?

Create user `signup` handler in daemon

User interaction methods

User interaction method from cli to deployit platform

contains these issues:

Signin method

• #52 - cli method
• #53 - handlers

Signup method

• #54 - cli method
• #55 - handlers

Password recovery method

• #56 - cli method
• #57 - handlers

Password change method

• #58 - cli method
• #59 - handlers

Core issues

• #60 - User package

Select node to deploy on create service page

Select node to deploy on create service page

We need to add ability to select particular node for deploy on create service page.

Version

0.9.9

Create cli user `password recovery` method

A User should be able to execute command deploy password recovery in a terminal.
It should send a user an email with the token you need to use to change a password.
Available flags:

• u username - user username

To make a password recovery, CLI should create an HTTP request:

• type: POST
• url: /user/password/recovery
• data: `json {"username":"username"}``

You can receive:

• 200: json {"status":"ok"}
• 400: json {"error":"credential format invalid"}
• 404: json {"error":"user not found"}

Note: You can use email instead of username in password recovery.

Create Vagrant development environment

Vagrant development environment support

Create a vagrant file based on virtualbox:

• coreos box
• kubernetes under the hood
• build deployit sources
• start deployit daemon
• guest ports forwarding 2967>2967

v0.1.0

How To Reproduce

1. Download Vagrant
2. vagrant up --provision

Create cli user `password change` method

A User should be able to execute command deploy password change in a terminal.
It should change user's password in deploy it system
Available flags:

• t token - recovery token

To make an authentication, CLI should create an HTTP request:
Request:

• type: POST
• url: /user/password/change
• data: `json {"password":"password"}``

You can receive:

• 200: json {"token":"access token"}
• 400: json {"error":"validation error", "description":"validation description"}

Note: password should be at least 8 chars of length.

Create user method in user package

User list package method

Should return list of users from kubernetes

User list method

func (c *users) List(opts k.ListOptions) (result *du.UserList, err error) {
    result = &du.UserList{}
    err = client.Get().
        Resource("users").
        VersionedParams(&opts, k.ParameterCodec).
        Do().
        Into(result)
    return
}

UserList structure

type UserList struct {
    ku.TypeMeta `json:",inline"`
    ku.ListMeta `json:"metadata,omitempty"`
    Items []User `json:"items"`
}

Where:

• k - kubernetes api version (current - v1 package)
• ku - kubernetes unversionned api types
• du - deploy it user types

Create user `password recovery` handler in daemon

Active contributors on website about page

Active contributors section on main website

I would like to add a special section with active contributors on main website.
In about page in a team section.

More detailed prerequisited installation

I'm new at go, but trying to do sth.
I've installed latest go 1.8.3 and it works, but which next requisites do you mean?
Could you provide at least link so i can find them?

Is go etcd client for v3, this: https://github.com/coreos/etcd/tree/master/clientv3 ?

And what about go docker?

I tried to install go etcd client for v3 but got this error:
work/src/github.com/coreos/etcd/clientv3/client.go:236: dialer.DialContext undefined (type *net.Dialer has no field or method DialContext)

and made it this far.

At this point you have this, which does not provide sufficient information for a noob like me:
1.1.1. Prerequisites
Go 1.7 or higher
Go docker client v17.03
Go etcd client for v3

Check github integrations

Set logs messages in to the same format

Description

We need to unify log messages format.

const logLevel = 5
const msg = "package/module/function:"
log.V(logLevel).Debugf("%s> log message", msg)

Version

0.1.0-beta2

fatal error: concurrent map read and map write

{"level":"error","msg":"Task [9ff54b16-7d9e-4e84-a844-894e4bef057d]: container create error Error: No such image: library/nginx:latest","time":"2017-08-15 18:10:12"}
{"level":"error","msg":"Task [2b32d226-d29b-45f3-aa7f-806784f4aa9b]: container create error Error: No such image: library/nginx:latest","time":"2017-08-15 18:10:12"}
{"level":"error","msg":"Task [90d8a2fd-b773-4766-b6f5-a8a35adf051b]: container create error Error: No such image: library/nginx:latest","time":"2017-08-15 18:10:12"}
{"level":"error","msg":"Task [33634bea-e048-4606-9bad-e2e659c6fcc2]: container create error Error: No such image: library/nginx:latest","time":"2017-08-15 18:10:12"}
{"level":"error","msg":"Task [6fe49967-864c-45dd-bf52-1c3e005dd389]: container create error Error: No such image: library/nginx:latest","time":"2017-08-15 18:10:12"}
fatal error: concurrent map read and map write

goroutine 90 [running]:
runtime.throw(0xbaf327, 0x21)
        /usr/local/go/src/runtime/panic.go:596 +0x95 fp=0xc42088e840 sp=0xc42088e820
runtime.mapaccess2_faststr(0xac2c00, 0xc420434f60, 0xc420952900, 0x1c, 0x1, 0x1)
        /usr/local/go/src/runtime/hashmap_fast.go:326 +0x50a fp=0xc42088e8a0 sp=0xc42088e840

Create user `password change` handler in daemon

IRC inactive

It appears like the IRC is currently not actively used. I believe the developers should either be active on it for us to discuss the project with them.

Design and describe new deployit system

• Provide a special concept doc section about deployit
• Review and describe deployit application structure
• Describe deployit REST API protocol

Watch user changes package method

Watch users changes package method

Should connect to kubernetes and watch user changes in realtime

User create method

func (c *users) Watch(opts k.ListOptions) (watch.Interface, error) {
    return c.Get().
        Prefix("watch").
        Resource("users").
        VersionedParams(&opts, k.ParameterCodec).
        Watch()
}

User structure

type User struct {
    ku.TypeMeta
    k.ObjectMeta

        Username string
        Email string
        Pass string
        Salt string
        Created time
        Updated time

    Identities []string
}

Where:

• k - kubernetes api version (current - v1 package)
• ku - kubernetes unversionned api types
• du - deploy it user types

Custom http status texte replacing with go idiomatic http status texte ?

Hey,

i have discovered in the sub pacakge (libs/errors/errors.go) several custom http status texte.
The Golang std lib has all standard http status texte in "net/http/status.go" and helper functions.

Can i replace all unnecessary custom status texte with std status texte and refactor the unique texte ?

Best regards,

Marcel

Network security?

Description

I have not yet had the opportunity to test things yet but is internal (inner cluster) communication properly separated (in a way packets can't be man in the middled)?

nothing happen "vagrant up"

Description

Nothing happen after executing vagrant up

Version

[$ deploy version]

How To Reproduce

1. I tried to clone and directly run the Vagrantfile, fix the NFS problem first in my local environment
2. All package were downloaded and provisioned and bring the node up
3. try to SSH, enter the node, but no lastbackend command.

Optional information

netstat results:

---

Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:8080 0.0.0.0:* LISTEN 2025/hyperkube
tcp 0 0 127.0.0.1:7001 0.0.0.0:* LISTEN 788/etcd2
tcp 0 0 127.0.0.1:4001 0.0.0.0:* LISTEN 788/etcd2
tcp 0 0 127.0.0.1:10248 0.0.0.0:* LISTEN 797/kubelet
tcp 0 0 127.0.0.1:10249 0.0.0.0:* LISTEN 2098/hyperkube
tcp 0 0 127.0.0.1:2379 0.0.0.0:* LISTEN 788/etcd2
tcp 0 0 127.0.0.1:2380 0.0.0.0:* LISTEN 788/etcd2
tcp6 0 0 :::10255 :::* LISTEN 797/kubelet
tcp6 0 0 :::22 :::* LISTEN 1/systemd
tcp6 0 0 :::443 :::* LISTEN 2025/hyperkube
tcp6 0 0 :::4194 :::* LISTEN 797/kubelet
tcp6 0 0 :::10250 :::* LISTEN 797/kubelet
tcp6 0 0 :::10251 :::* LISTEN 1973/hyperkube
tcp6 0 0 :::10252 :::* LISTEN 2074/hyperkube
udp 0 0 0.0.0.0:8472 0.0.0.0:* -
udp 0 0 0.0.0.0:935 0.0.0.0:* -
udp 0 0 10.0.2.15:68 0.0.0.0:* 1432/systemd-networ
udp6 0 0 fe80::582c:2fff:fe9:546 :::* 1432/systemd-networ
udp6 0 0 fe80::a00:27ff:fece:546 :::* 1432/systemd-networ

any help, please!
Thank you.

Only One Public Project?

Describe the bug
I just signed up and went to the public templates section.
There is only one template shown (see screenshot).

To Reproduce
Steps to reproduce the behavior:

1. Go to 'https://cloud.lastbackend.com/templates#public'
2. See error

Expected behavior
I was expecting a lot more, e.g. Wordpress, MongoDB, Bootstrap, PostgreSQL, Etherpad, Apache, CouchDB, SearX, wallabag and many more.

Screenshots

Infrastructure (please complete the following information):

• lastbackend:us-dallas

Some grammar mistakes

Hi, guys, I've visited your site recently and noticed a couple of typos/mistakes.

1. On the start page you have "build your ... in a seconds". I guess, either plural or "a" is redundant here.
2. In the phrase about that you welcome github activity you have a typo "abitious idea".

Just threw a quick glance.

Ansible Installation

Description

I was looking at running the self hosted version of this, but there doesn't appear to be the ansible installation instructions. Are these not completed yet or just missing?

Version

[$ lb version]

Easy datastore management?

Description

Not sure if it's integrated already but it would be great if lastbackend also provides a quick way to setup highly available datastores (mongo, redis, postgres, rabbitmq...). A system like flynn.io has would work.

It's essential for our apps to save data to a persistent store, and this store must be highly available. The system I am hoping for is one with automatic high availability (maybe 1 sync slave, 1 async slave), monitoring and automatic reprovisioning (when a node is permanently down, append a new slave), data security and eventually sharding.

Building not successful.

Description

I can't install LastBackend.

Version

Latest, unsure what version.

Optional information

root@frozola:~# go get github.com/lastbackend/lastbackend
package github.com/lastbackend/lastbackend: no buildable Go source files in /root/work/src/github.com/lastbackend/lastbackend

Create cli user `signin` method

A User should be able to execute command deploy signin in a terminal.
It should authenticate a user in deploy it system with [username|email] and password.
Available flags:

• p password - user password
• u username - user username

To make an authentication, CLI should create an HTTP request:

• type: POST
• url: /user
• data: `json {"username":"username","password":"password"}``

You can receive:

• 200: json {"token":"access token"}
• 401: json {"error":"authentication invalid"}

Note: You can use email instead of username in authentication.

Update Last.Backend installation doc

Update Last.Backend installation doc

After major changes, we need to update our doc with actual info.
Need to create articles about

• deploy docker-machine
• deploy from dockerfile
• build and deploy from sources

Version

v0.9.0

Site typo

There's a little type here. I assume you mean experience.

Confusing licence

LICENSE.md says that project is under Apache License 2.0.
But in files we have:

lastbackend/cmd/api/api.go

Lines 2 to 17 in 941af64

	// Last.Backend LLC CONFIDENTIAL
	// __________________
	//
	// [2014] - [2019] Last.Backend LLC
	// All Rights Reserved.
	//
	// NOTICE: All information contained herein is, and remains
	// the property of Last.Backend LLC and its suppliers,
	// if any. The intellectual and technical concepts contained
	// herein are proprietary to Last.Backend LLC
	// and its suppliers and may be covered by Russian Federation and Foreign Patents,
	// patents in process, and are protected by trade secret or copyright law.
	// Dissemination of this information or reproduction of this material
	// is strictly forbidden unless prior written permission is obtained
	// from Last.Backend LLC.
	//

Please choose under what type of license this project is.

Signup impossible: CORS request failed

I tried to create an account, but signup process failed when your frontend https://app.lastbackend.com/signup call your backend https://api.lastbackend.com/account.

Firefox report a Cross-Origin Request that failed.

Steps to reproduce the behavior:

1. Go to https://app.lastbackend.com/signup
2. Try to create an account
3. See error

Update Vagrantfile to new version of Last.Backend

Update Vagrant file to provision new version of Last.Backend platform

• use coreos box
• install etcd v3 version
• create lastbackend kit image
• create lastbackend dashboard image
• run lastbackend kit image
• run lastbackend dashboard image

Version

0.9.0

Improve cloud CPU specification

The cloud section of the site does not indicate how CPU usage scales. This would be a great addition. Independent CPU/RAM allocation may be cool as well.

Delete user method package

Delete user package method

Should delete user from kubernetes storage

User delete method

func (c *users) Delete(username string) (err error) {
    return c.Delete().Resource("users").Name(username).Do().Error()
}

Where:

• k - kubernetes api version (current - v1 package)
• ku - kubernetes unversionned api types
• du - deploy it user types

Describe package user based on kubernetes resources

User package

User package can do:

• #61 list users
• #62 get user
• #63 create user
• #64 update user
• #65 validate user
• #66 delete user
• #67 watch user

User object is an kubernetes resource.
All users are stored in deployit system kubernetes namespace. But particular user information is stored in user personal namespace in resource user.

User resource contains:

• email
• username
• encrypted password
• salt
• created
• updated

Secrets in the competition comparison

As far as I remember, K8s supports secrets storage, but your competition analysis suggests otherwise.

If I'm mistaken and that's not about "secrets" in the way k8s understands it, you should probably make some clarification.

Create user `signin` handler in daemon

Prepare deployit system for boot

Bootstrap deployit daemon steps:

1. Check kubernetes connection available from config
2. Create if not exists namespace
3. Create if not exists and run if stopped docker registry
4. Create default user
5. Create and setup HTTP server on 2967 port

Note:

• kubernetes client should be singleton and should be available in whole system.

"Talk to us" leads to start page

Describe the bug
When user presses link "Talk to us" down below, the website returns to start page, which doesn't look like correct behavior to me.

To Reproduce
Steps to reproduce the behavior:

1. Open website
2. Scroll down to "Talk to us" and click on the link

Installation instructions do not work

Description

I cloned the repo and followed the instructions. They don't work as there's no makefile in cmd/daemon

Version

Latest from git

How To Reproduce

1. Try to follow your installion instructions from the wiki

Optional information

Create cli user `signup` method

A User should be able to execute command deploy signup in a terminal.
It should create a user in deploy it system.
After executing this command CLI should ask for more information:

• :email - user email
• :username - user username
• :password - user password

Check email for unique

After entering email, deployit should check it for unique:
Request:

• type: GET
• url: /user/check/email
• data: `json {"email":"email"}``

Response:

• 200: json {"status":"ok"}
• 400: json {"error":"not unique"}

Check username for unique

After entering username, deployit should check it for unique.
Request:

• type: GET
• url: /user/check/username
• data: `json {"username":"username"}``

Response:

• 200: json {"status":"ok"}
• 400: json {"error":"not unique"}

Complete registration

After all validations, to make an user singup, CLI should create an HTTP request:
Request:

• type: PUT
• url: /user
• data: `json {"email":"email","username":"username","password":"password"}``

Response:

• 200: json {"token":"access token"}
• 401: json {"error":"authentication invalid"}

Note: You can use email instead of username in authentication.

Create user package method

Create user package method

Should create user and stop it into kubernetes

User create method

func (c *users) Create(user *du.User) (result *du.User, err error) {
    result = &du.User{}
    err = c.Post().Resource("users").Body(user).Do().Into(result)
    return
}

User structure

type User struct {
    ku.TypeMeta
    k.ObjectMeta

        Username string
        Email string
        Pass string
        Salt string
        Created time
        Updated time

    Identities []string
}

Where:

• k - kubernetes api version (current - v1 package)
• ku - kubernetes unversionned api types
• du - deploy it user types

Get user method in user package

User get package method

Should return user from kubernetes by username

User get method

func (c *users) Get(username string) (result *du.User, err error) {
    result = &du.User{}
    err = client.Get().
        Resource("users").
                Name(username).
        Do().
        Into(result)
    return
}

User structure

type User struct {
    ku.TypeMeta
    k.ObjectMeta

        Username string
        Email string
        Pass string
        Salt string
        Created time
        Updated time

    Identities []string
}

Where:

• k - kubernetes api version (current - v1 package)
• ku - kubernetes unversionned api types
• du - deploy it user types

Validate user

Validate user package method

Should get user from storage and validate is password

User validate method

func (c *users) Validate(username, password string) (result *du.User, err error) {
    result = &du.User{}
    err = c.Get().Resource("users").Name(name).Do().Into(result)
        // Provide password validation
    return
}

User structure

type User struct {
    ku.TypeMeta
    k.ObjectMeta

        Username string
        Email string
        Pass string
        Salt string
        Created time
        Updated time

    Identities []string
}

Where:

• k - kubernetes api version (current - v1 package)
• ku - kubernetes unversionned api types
• du - deploy it user types

Add mock database for tests automation

Description

We should have a mock database, based on storage interface for implement tests automation

Version

0.1.0