dataman-cloud/crane

Yet another control plane based on docker built-in swarmkit

Crane

Crane, maintained by dataman-cloud, is a docker control panel based on latest docker release. Besides swarm features, Crane implements some badly needed functionalities by enterprise user, such as private registries authentication, ACL and application DAB(distributed application bundle) sharing. The smart fuzzy search function give user quickly access to the desired page. Crane can help storing registry auth pair, from where you can choose a predefined registry auth pair when deploying a DAB, without the need to docker login when access private image. Crane can also help sharing your private images with your coworkers easily.

Features

• Swarm features: Portal every feature of swarm almost. Crane has highlighted the common swarm functions and improved the user experiences through the friendly frontend.
• Stack Templates Management: User can save a running stack as a template, by which others can deploy repeatly.
• Image Management: The private image owned by user can be publiced to others.
• Fuzzy Search: A in-memory index maintained by the backend serves the function.
• Node Operation: Crane details about a node such as kernel version, docker info, docker images and also containers running on the node.
• Network Management: The overlay network CRUD.
• Registries Authentication Management: You can save your private registry username/password pair to Crane, with which a to-be-deployed stack with restricted image access can attach.
• WebSSH: Command 'docker exec' is the magic behind it.

OS supported

• Ubuntu 12.04 Server
• Ubuntu 14.04 Server
• CentOS 7.X
• MacOS 10.x

Installation

Prerequisites

• docker>=1.12 how to install
• docker-compose>=1.8.0 how to install
• Enable the Docker tcp Socket on port: 2375 how to config
• Start ntp service

Option 1: Stable version in one line

Please read the release/v1.0.6/README.md

Option 2: development workflow from docker build

• build crane image

$ ./bin/build-push-or-up.sh build

• tips to get real host ip based on eth0 interface:

ip addr s eth0 |grep "inet "|awk '{print $2}' |awk -F "/" '{print $1}'

• docker-compose up crame service

$ CRANE_IP=`<your real host ip,such as 192.168.1.x>` ./bin/build-push-or-up.sh up

• remove crane container

$ ./bin/build-push-or-up.sh down

CRANE_IP should be assigned the real host ip address of the running Crane host which is the swarm manager also.

How to use it

Build From Source

Clone crane in GoPath

> mkdir -p ${GOPATH}/src/github.com/Dataman-Cloud
> cd ${GOPATH}/src/github.com/Dataman-Cloud
> git clone https://github.com/Dataman-Cloud/crane.git crane

And make sure you have go (>= 1.6) go into the crane dir

> make

Please click Crane User Guide in Chinese for more details.

Conventions

repo branch

• master: actively moving foward. PR will be merged into this master branch.
• release: Released versions. Tagged commits or hotfix PR will be pushed here. Maintained by the repo owners.

Trouble-shooting

Community

Wechat group: 数人云Crane技术交流群

Contribution

Both pull-requests or issues are welcomed from the community.

License

Crane is available under the Apache 2 license.

crane's Issues

API documents should be embed into project, let user or developer can easy review

need more consider this status. we need api docs

Fail to install crane because of pulling image 'digitallyseamless/nodejs-bower-grunt:5' error

Actual Behavior:

• What is the issue?

Fail to install crane, info like this:

# CRANE_IP=172.16.1.12 ./bin/build-push-or-up.sh up Unable to find image 'digitallyseamless/nodejs-bower-grunt:5' locally 5: Pulling from digitallyseamless/nodejs-bower-grunt 8ad8b3f87b37: Pulling fs layer 751fe39c4d34: Pulling fs layer ae3b77eefc06: Pulling fs layer 7783aac582ec: Waiting 393ad8a32e58: Waiting 63ac16987eeb: Waiting 9feb200db06a: Waiting 3d4b596e06e1: Waiting docker: error pulling image configuration: Get https://dseasb33srnrn.cloudfront.net/registry-v2/docker/registry/v2/blobs/sha256/ec/ecd63549014ec070b1c2f2cd656e9675ef7b3254d558ee43b5e34cbebf65527b/data?Expires=1475053063&Signature=aw9GcF2sKCM-eoQM8uIcbTiBGTqLXyPTR-cqYKlPCfKl3rXGI5IiEq4F9kAqZHqEg2tY3BfdrZE0LHrs6yGpYxmXuund0R1eVF2-A3LbaJ~NlCVeLMtMZ2PQQV9H2GhrOS4yQkHkxHRa2JHQfL5QBuGNkP9AWgdEdNWE2gkz96g_&Key-Pair-Id=APKAJECH5M7VWIS5YZ6Q: EOF.

But there is not problem when docker pull digitallyseamless/nodejs-bower-grunt:5.

# docker pull digitallyseamless/nodejs-bower-grunt:5 5: Pulling from digitallyseamless/nodejs-bower-grunt 8ad8b3f87b37: Pull complete 751fe39c4d34: Pull complete ae3b77eefc06: Pull complete 7783aac582ec: Pull complete 393ad8a32e58: Pull complete 63ac16987eeb: Pull complete 9feb200db06a: Pull complete 3d4b596e06e1: Pull complete Digest: sha256:5c1635e448bacedbdae5dc4fbf5dff71fed1a76d51b012be4a4637bde0de70bd

• What is the expected behavior?

Installation should be successful.

CodePen (or steps to reproduce the issue): *

• CodePen Demo which shows your issue:

  CRANE_IP=172.16.1.12 ./bin/build-push-or-up.sh up

• Details:

  v1.0.5

Angular Versions: *

• Angular Version:
• Angular Material Version:

Additional Information:

• Browser Type: *

• Browser Version: *

• OS:

  Centos 7.2

• Stack Traces:

Crane doesn't support swarm cluster with mulitple managers? and requests should dispatch to new leader when leader rotate

Actual Behavior:

• What is the issue? *

1, Crane doesn't support swarm cluster with multiple manager now.
2, As swarm manager leader rotate, any subsequent requests should dispatch to new leader.

界面存在小bug

Actual Behavior:

• What is the issue? *
  当点击搜索后，再点击切换语言。
  此时搜索界面并没有隐藏，并且导致不能切换语言。
• What is the expected behavior?
  

CodePen (or steps to reproduce the issue): *

• CodePen Demo which shows your issue:
• Details:

Angular Versions: *

• Angular Version:
• Angular Material Version:

Additional Information:

• Browser Type: *
• Browser Version: *
• OS: *
• Stack Traces:

Have to make sure IP Forwarding enabled

Add the above into node-init.sh

refer: http://www.ducea.com/2006/08/01/how-to-enable-ip-forwarding-in-linux/

update api doc

The api doc is deprecated, we'd better to update them. Find a site to serve the api-doc is a good idea.

image not found

ERROR: Error: image library/crane:1.0 not found

ubuntu 12.04 supported

Swarm cluster have been running!
WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use the bundle feature of the Docker experimental build.

More info:
https://docs.docker.com/compose/bundles

Removing rolex_rolex_registry_1
Removing rolex_rolex_1
Starting rolex_rolex_db_1
Recreating 57cef041743f_rolex_rolex_registry_1
Recreating 62545171d5f2_rolex_rolex_1

ERROR: for rolex_db Cannot start service rolex_db: oci runtime error: apparmor failed to apply profile: no such file or directory

ERROR: for rolex_registry Cannot start service rolex_registry: oci runtime error: apparmor failed to apply profile: no such file or directory

ERROR: for rolex Cannot start service rolex: oci runtime error: apparmor failed to apply profile: no such file or directory
ERROR: Encountered errors while bringing up the project.

I need to license registration?

Reference example, from the source to build, running after the show for Activation.
I need to license registration?

remove my image from registry

align the input, DAB text area and buttons

ERROR: NTP is unsynchronised, Please confirm your ntp status before continue

got an error while building registry

when I exec the comaand :CRANE_IP=192.168.59.105 ./bin/build-and-start.sh

I got the following error:
ERROR: for crane_registry Cannot start service crane_registry: oci runtime error: exec: "/entrypoint.sh": permission denied

How about make Crane release package into single image?

For now Crane installation is easy enough by only one line of bash command, but it occupy 4 containers MySQL, Crane Api, Crane frontend and Registry respectively, the total size of images upto 1G, need to improve but here is what I think

1, make all components into one single image/container, this is simple but lack the flexibility if user want make there own version.

2, leave the deployment/installation decision to Crane users, they can choose what DB to use, how to serve static files and how to connect a registry(Crane version or Private registry setup by themselves)

add batch-opration on table-list

Actual Behavior:

• add the feature of batch-opration on table-list*
• In table list , the oprations of “删除”、“扩展”、“停止”、“修改任务数”、“Kill”、“Kill&Scale” need batch.

how to add new user?

we are now use the default user:[email protected],
how to add other user?
I can not find any entry in the crane web page.
thanks

Sometimes there are two more states with a task that is running in the case

Sometimes there are two more states with a task that is running in the case

Demo is dead

I'd like to access the demo but it seems like the demo app is dead.

MacOSX Docker Compose

Expected behavior

get docker compose build up all services

Actual behavior

not starting all services

docker-compose up -d

WARNING: Image for service crane_registry was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Building crane
ERROR: Error parsing reference: "crane:" is not a valid repository/tag

I guess something is wrong with the tag variable of docker images.

Information about the Issue

problems with the front-end if using docker-compose up -d with mac osx sierra

can you provide more details about Mac OSX boostrapping for Mac OSX/docker-machine (ip, volumes,...) ?

Steps to reproduce the behavior

1. cd deploy
2. docker-compose up -d

安装crane面版时报错

安装crane面版时报错。
服务器ubuntu server 14.04.5
执行下面代码进行安装时报错
CRANE_IP=192.168.88.88 VERSION=v1.0.5 ./deploy.sh
Checking the node status
./node-init.sh: 3: set: Illegal option -o errtrace

把node-init.sh第三行注释掉就通过了。

service information displayed error

eg： use images：zookeeper

set env like follow：

ZOO_MY_ID:1
ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888

when the services deployed ；
view the service information， the env show like this：

ZOO_MY_ID 1
ZOO_SERVERS: server.1

did not display the full values!

push image sucessfully,but can not found in the website

push image step by step just like the doc : https://dataman.gitbooks.io/crane/content/get-started/registry.html said

but can not found image in the website :http://my_ip/registry/list/mine

Error when view Terminal of Mysql Project

Actual Behavior:

• What is the issue? *

Version: 1.0.0-e4d31ca

1.Using Image Templates to deploy Mysql project. Json file as default. --> Then the task state is always "failed".

2.Go to the service page, then go to the container page, click "Terminal" tag, Error occur as the screenshot.

• What is the expected behavior?

1.Then the task state should not be "failed".

2."Terminal" tag should not show things like this.

CodePen (or steps to reproduce the issue): *

• CodePen Demo which shows your issue:
• Details:

Angular Versions: *

• Angular Version:
• Angular Material Version:

Additional Information:

• Browser Type: *
• Browser Version: *
• OS: * Centos 7.2
• Stack Traces:

What's the Monitoring solution of Crane platform? cAdvisior or Prometheus ?

Hi, I find the Crane User Guide in Chinese supports real-time monitoring under Cluster Operation Sub-module, however I don't know the specific monitoring solution.

If Crane use monitoring tool , like cAdvisior or Primetheus?
or Crane just use Docker API to collect monitoring data?

https://dataman.gitbooks.io/crane/content/overview/overview.html

crane 安装出错——"crane:" is not a valid repository/tag

Expected behavior

安装成功

Actual behavior

安装过程报错
Building crane
ERROR: Error parsing reference: "crane:" is not a valid repository/tag

or

Building blackmamba
ERROR: Error parsing reference: "blackmamba:" is not a valid repository/tag

Information about the Issue

docker-compose 在build crane和blackmamba时，没有正确读出TAG这个环境变量。
尝试将deploy/docker-compose.yml的两处:${TAG}删掉，问题解决

Steps to reproduce the behavior

1. CentOS7，安装配置好docker相关软件
2. 下载解压crane。执行 CRANE_IP=172.18.24.69 ./bin/build-push-or-up.sh up

Make registry namespace self-decided, instead email prefix

Large the JSON input text size

The DAB JSON input is too small to input.

installation errors

Option 1: Stable version in one line

I got -

Creating network "rolex_default" with the default driver
Pulling rolex_registry (catalog.shurenyun.com/library/registry:rolexv1.0.4)...
Pulling repository catalog.shurenyun.com/library/registry
ERROR: Error: Status 504 trying to pull repository library/registry: "<html><body><h1>504 Gateway Time-out</h1>\nThe server didn't respond in time.\n</body></html>\n\n"

I think the service of catalog.shurenyun.com is down.

Crane on Mac

RT, crane能在docker for mac上运行吗?

can not delete Stack Template

crane default with 4 Stack Template ,but none of them can be delete

Crane should support Sqlite or BoltDB as MySQL need another serivces

Actual Behavior:

• What is the issue? *

Crane should support Sqlite or BoltDB as MySQL need another serivces

One task have both of running tasks

deploy the demo with domain: crane.shurenyun.com

添加Node资源监控和配额管理功能?

Description

可以根据集群可用资源，按项目配置可用份额，以及目前集群可用资源的监控

Feature Request: Add the "Language" droplist to set my favorite frontend language

As the subject, I'd like to setup my favorite frontend language.

How to implement:
We can set the browser cookie to implement it.

Service cannot be explicitly attached to "ingress" network which is a swarm internal network"

使用模版部署时报错：Service cannot be explicitly attached to "ingress" network which is a swarm internal network"

{"code":10001,"data":{},"message":"rpc error: code = 3 desc = Service cannot be explicitly attached to "ingress" network which is a swarm internal network","source":"docker"}

docker 版本 1.12.2
docker-compose 版本 1.8.1
Crane版本是1.0.5

CentOS 7.2

go vet need add to integration testing

Actual Behavior:

• What is the issue? *

found go vet reports many bugs. it should be notice to developer.

Testing Example

$ go vet ./src/utils/config/.../
src/utils/config/config.go:15: struct field tag `env:"CRANE_DOCKER_TLS_VERIFY" envDefault:false,required:"true"` not compatible with reflect.StructTag.Get: bad syntax for struct tag value
src/utils/config/config.go:53: no formatting directive in Fatalf call
exit status 1

[Polling Station] If we store the log history and how to make it if yes

As the subject, Crane is supporting real-time logs. If we add the log history feature? AFAIK, the ElasticSearch or other distributed FS have to be integrated into, which will heavy the crane project much. But the FeatureFlag logHistory can light it maybe.

So we are looking for your advice?

1. Implement it? yes or no
2. how to solve it? ElasticSearch or others?

Curl-to-bash installation needed!

Actual Behavior:

• What is the issue? *

for convenient way, we need help user to easily install crane by bash script.

• What is the expected behavior?

such as curl -sSL http://crane.shurenyun.com/install | bash

this page is not translate in english

Actual Behavior:
missing translate in english

Additional Information:
macos / chrome

deploy service random port

Deploy service cannot specify a random port?

When I update the catalog(or stack template) , I got error: Ban stack template failed

Actual Behavior:

• When I update the catalog(or stack template), which is crane built-in catalog such as 2048, I got error: Ban Stack template failed. attached the snapshot
• The expected behavior should **GOT error: forbid update system default** , because I found the error at backend log
  

Is Debian supported?

Redirected the login page when I click Image/My-Private-Image tab on http://crane.shurenyun.com/

Actual Behavior:

• As the subject
• Go to the My-Private-Image list

CodePen (or steps to reproduce the issue): *

• CodePen Demo which shows your issue:
• Details:

Angular Versions: *

• Angular Version:
• Angular Material Version:

Additional Information:

• Browser Type: *
• Browser Version: *
• OS: *
• Stack Traces:

The default user failed to login

crane部署完后，使用默认的用户名和密码无法登录，提示服务忙，请稍候重试

[tech debt] ICON_DEFAULT looks hard code

vim src/plugins/catalog/model.go +24 goto the ICON_DEFAULT, we'd better make the value configurable or generate the image auto.

Tell user where is the exception in JSON format validation

Our JSON format error reminder func is weak now. As the following img, crane have to show the error line location

bug? ingress网络和自定义网络的device都分配了两个ip

/ # ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
64: eth0@if65: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP
link/ether 02:42:0a:ff:00:0a brd ff:ff:ff:ff:ff:ff
inet 10.255.0.10/16 scope global eth0
valid_lft forever preferred_lft forever
inet 10.255.0.9/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:aff:feff:a/64 scope link
valid_lft forever preferred_lft forever
66: eth1@if67: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
link/ether 02:42:ac:13:00:06 brd ff:ff:ff:ff:ff:ff
inet 172.19.0.6/16 scope global eth1
valid_lft forever preferred_lft forever
inet6 fe80::42:acff:fe13:6/64 scope link
valid_lft forever preferred_lft forever
69: eth2@if70: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1450 qdisc noqueue state UP
link/ether 02:42:ac:10:00:03 brd ff:ff:ff:ff:ff:ff
inet 172.16.0.3/16 scope global eth2
valid_lft forever preferred_lft forever
inet 172.16.0.2/32 scope global eth2
valid_lft forever preferred_lft forever
inet6 fe80::42:acff:fe10:3/64 scope link
valid_lft forever preferred_lft forever

在容器的详情页只能看到一个
网络配置

网络 网关 IP 地址 MAC 地址
ingress 10.255.0.10 02:42:0a:ff:00:0a
test2 172.16.0.3 02:42:ac:10:00:03

New the stash template, don't upload the pic of stash.

**Additional Information**: - `Pic Type: .JPG*` - `Pic Size: 50kb`

registry db store

registry push image,in addition to inform the way,should be combined with training in rotation.