cnrancher/octopus

Lightweight device management system for Kubernetes/k3s

Octopus

Octopus is no longer being actively maintained.

Octopus is a light-weight and cloud-native device management system for Kubernetes and k3s, it does not need to replace any basic components of the Kubernetes cluster. After Octopus deployed, the cluster can have the ability to manage edge devices as custom k8s resources.

• Idea
• Documentation
• Quick-start
• Source code
• License

Idea

Like a real octopus, Octopus consists of the brain and limbs. The brain only needs to deploy one or automatically select a leader in HA mode. It only needs to process relatively concentrated information, such as verifying the existence of nodes and the existence of device models (types). Limbs need to be deployed on each edge node that can be connected to the device, and they communicate with the actual device through the device adaptor (Adaptors). Therefore, Octopus uses a DeviceLink YAML file (a custom-defined k8s object) to configure and manage its device connections.

For more details please refer to the official documentation.

Documentation

• Octopus
  • About Octopus
  • Quick-start guide
  • How to develop
  • How to monitor
  • The state transition of DeviceLink
• Adaptors
  • How it works
  • How to develop
• Contribution
  • How to contribute

Quick-start

There are two ways to deploy the Octopus, for quick-start, you can use the manifest YAML file to bring up the Octopus. The installer YAML file is under the deploy/e2e directory on Github.

# install octopus
$ kubectl apply -f https://raw.githubusercontent.com/cnrancher/octopus/master/deploy/e2e/all_in_one.yaml

# install ui
$ kubectl apply -f https://raw.githubusercontent.com/cnrancher/octopus-api-server/master/deploy/e2e/all_in_one.yaml

# install adaptors
$ kubectl apply -f https://raw.githubusercontent.com/cnrancher/octopus/master/adaptors/modbus/deploy/e2e/all_in_one.yaml
$ kubectl apply -f https://raw.githubusercontent.com/cnrancher/octopus/master/adaptors/opcua/deploy/e2e/all_in_one.yaml
$ kubectl apply -f https://raw.githubusercontent.com/cnrancher/octopus/master/adaptors/mqtt/deploy/e2e/all_in_one.yaml
$ kubectl apply -f https://raw.githubusercontent.com/cnrancher/octopus/master/adaptors/ble/deploy/e2e/all_in_one.yaml
$ kubectl apply -f https://raw.githubusercontent.com/cnrancher/octopus/master/adaptors/dummy/deploy/e2e/all_in_one.yaml

Optionally, you can use this repository hosts official Helm charts for Octopus. These charts are used to deploy Octopus to the Kubernetes/k3s Cluster.

# add octopus helm repo
$ helm repo add octopus http://charts.cnrancher.com/octopus
$ helm repo update

# create octopus-system namespace
$ kubectl create ns octopus-system

# install octopus, ui and adaptors
$ helm install --namespace octopus-system octopus octopus/octopus

Source code

Octopus is 100% open source software. Project source code is spread across a number of repos:

Name	Repo Address
Octopus UI	https://github.com/cnrancher/octopus-ui
Octopus API Server	https://github.com/cnrancher/octopus-api-server
Octopus Chart	https://github.com/cnrancher/octopus-chart
Octopus Simulator	https://github.com/cnrancher/octopus-simulator
Octopus Docs	https://github.com/cnrancher/docs-octopus

License

Copyright (c) 2020 Rancher Labs, Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at LICENSE file for details.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

octopus's Issues

update default repo to use cnrancher

Issue:
current drone CI is configured to push images to the cnrancher repo but the deployment YAML is set to use rancher

Solution:
set the default image repo to use cnrancher for all generated YAMLs

front-end web framework

Vue is a good choice, but we respect the wishes of the developer.
At the same time, we need to write some packaging scripts to ensure that CI can work.

add device annotations to adaptors and remove linux only nodeSelector

Improvement:

1. add pre-defined device annotations to adaptors for better UI display.
2. remove linux nodeSelector so that both adaptors and limb can be deployed to different os

*Adaptor image should support multi-arch design

adaptor's docker image should support multi-arch design since our k3s devices will support amd64/arm64 and armhf

[] bluetooth adaptor
[] modbus adaptor

Modus device link could lead to octopus limb excessive CPU load

Steps to reproduce:

1. Set up a local k3d cluster with one master and three nodes
2. Deploy octopus
3. Deploy Modbus adaptor: k apply -f all_in_one.yaml
4. Deploy a Modbus device link: k apply -f dl_modbus_rtu.yaml

Result:
CPU's temperature got high and to 90 degrees celsius. Even though deleting the corresponding device link, the related limb pod CPU usage kept high all the time

failed to re-register adaptor's socket on container restart

adaptor's websocket fails to re-register on container restart.

Error: failed to listen on: /var/lib/octopus/adaptors/ble.socket: listen unix /var/lib/octopus/adaptors/ble.socket: bind: address already in use

No permission to create events of DeviceLink

Environment information:

• Octopus version : master
• Installer : yaml

Cluster information:

• Cluster vendor : k3s
• Machine type : vm
• Kubernetes version (use kubectl version): v1.17.2
• Docker version (use docker version): 19.03.8

Machine specification:

Role	Number	Specification (CPU/RAM)
master	1	4U8G
agent	2	4U8G

Steps to reproduce (least amount of steps as possible):

1. follow to README.md to deploy Octopus and try dummy adaptor.

Result:

View the log of limb container, and find some errors as below:

events is forbidden: User system:serviceaccount:octopus-system:default cannot create resource events in API group  in the namespace default (will not retry!)

Other details that may be helpful:

It seems that there is not event creation permission in default ServiceAccount.

Except:

Ensure that the event can be created and can be viewed through kubectl describe devicelink ...

Dummy adaptor gets crashed

Steps to reproduce:

1. Deploy octopus
2. Deploy a dummy device
3. Edit DeviceLink correct, change the value of 'on' and save,

 model:
    apiVersion: devices.edge.cattle.io/v1alpha1
    kind: DummyDevice
  template:
    metadata:
      labels:
        device: correct
    spec:
      gear: middle
      "on": true

Result:
Dummy adaptor gets crashed.

2020-05-20T08:40:52.962Z	INFO	Mocking started
2020-05-20T08:42:13.106Z	INFO	Mocking finished
panic: close of closed channel
	panic: close of closed channel

goroutine 54 [running]:
github.com/rancher/octopus/adaptors/dummy/pkg/physical.(*device).off(0xc000428230)
	/go/src/github.com/rancher/octopus/adaptors/dummy/pkg/physical/device.go:86 +0x10f
github.com/rancher/octopus/adaptors/dummy/pkg/physical.(*device).Shutdown(0xc000428230)
	/go/src/github.com/rancher/octopus/adaptors/dummy/pkg/physical/device.go:49 +0x2b
github.com/rancher/octopus/adaptors/dummy/pkg/adaptor.(*Service).Connect.func1(0xc00041bb10)
	/go/src/github.com/rancher/octopus/adaptors/dummy/pkg/adaptor/service.go:51 +0x42
panic(0x13c36e0, 0x171b140)
	/usr/local/go/src/runtime/panic.go:679 +0x1b2
github.com/rancher/octopus/adaptors/dummy/pkg/physical.(*device).on(0xc000428230, 0xc0003a70d0, 0x6)
	/go/src/github.com/rancher/octopus/adaptors/dummy/pkg/physical/device.go:62 +0x231
github.com/rancher/octopus/adaptors/dummy/pkg/physical.(*device).Configure(0xc000428230, 0xc00042d101, 0xc0003a70d0, 0x6)
	/go/src/github.com/rancher/octopus/adaptors/dummy/pkg/physical/device.go:42 +0x48
github.com/rancher/octopus/adaptors/dummy/pkg/adaptor.(*Service).Connect(0xc00000e090, 0x177d880, 0xc0003dc100, 0x0, 0x0)
	/go/src/github.com/rancher/octopus/adaptors/dummy/pkg/adaptor/service.go:108 +0xc6
github.com/rancher/octopus/pkg/adaptor/api/v1alpha1._Connection_Connect_Handler(0x13c8ee0, 0xc00000e090, 0x17757e0, 0xc00042a000, 0x22f5598, 0xc000424000)
	/go/src/github.com/rancher/octopus/pkg/adaptor/api/v1alpha1/api.pb.go:417 +0xad
github.com/rancher/octopus/vendor/google.golang.org/grpc.(*Server).processStreamingRPC(0xc0003002c0, 0x1780820, 0xc0003bc300, 0xc000424000, 0xc000049470, 0x22ba1a0, 0x0, 0x0, 0x0)
	/go/src/github.com/rancher/octopus/vendor/google.golang.org/grpc/server.go:1199 +0xb2e
github.com/rancher/octopus/vendor/google.golang.org/grpc.(*Server).handleStream(0xc0003002c0, 0x1780820, 0xc0003bc300, 0xc000424000, 0x0)
	/go/src/github.com/rancher/octopus/vendor/google.golang.org/grpc/server.go:1279 +0xd30
github.com/rancher/octopus/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc0003a6030, 0xc0003002c0, 0x1780820, 0xc0003bc300, 0xc000424000)
	/go/src/github.com/rancher/octopus/vendor/google.golang.org/grpc/server.go:710 +0xbb
created by github.com/rancher/octopus/vendor/google.golang.org/grpc.(*Server).serveStreams.func1
	/go/src/github.com/rancher/octopus/vendor/google.golang.org/grpc/server.go:708 +0xa1

websoket is auto disconnect of some deivce CRDs in the UI

Environment information:

• Octopus version : master
• Installer : *

Cluster information:

• Cluster vendor : k3s
• Machine type : vm
• Kubernetes version (use kubectl version):
• Docker version (use docker version):

Machine specification:

Role	Number	Specification (CPU/RAM)
master	1	4U8G
agent	2	4U8G

Steps to reproduce (least amount of steps as possible):

1. install the octopus and the dummyDevice adaptor through the all-in-one.yaml
2. ks apply -f dl_specialdevice.yaml to install the specialdevice
3. launch the latest edge-api-server and the UI and check the WebSocket connection in the console, for CRDs like modbusdevice, opcuadevice, specialDummyDevice, etc will be auto-disconnect with the following message.

error: "failed to find schema modbusdevice"

Result:
UI is not able to receive the WebSocket message when the status of those CRD devices is updated.

Other details that may be helpful:

Except:
UI should be able to subscribe to those CRD resources and receive its updates via WS.

Related Issues:

high cpu load of octopu limb

Environment information:

• Octopus version : master
• Installer : *

Cluster information:

• Cluster vendor : k3s
• Machine type : vm
• Kubernetes version (use kubectl version):
• Docker version (use docker version):

Machine specification:

Role	Number	Specification (CPU/RAM)
master	1	1U2G
agent	2	1U2G

Steps to reproduce (least amount of steps as possible):

1. octopus limb didn't close connection upon failed adaptors.

2020-05-28T04:59:56.202Z	INFO	limb.setup	Registering metrics
2020-05-28T04:59:56.202Z	INFO	limb.setup	Registering APIs scheme
2020-05-28T04:59:56.202Z	INFO	limb.setup	Parsing arguments
2020-05-28T04:59:56.202Z	INFO	limb.setup	Landing on	{"node": "ip-172-31-39-223"}
2020-05-28T04:59:56.202Z	INFO	limb.setup	Creating controller manager
2020-05-28T04:59:56.757Z	INFO	controller-runtime.metrics	metrics server is starting to listen	{"addr": ":8080"}
2020-05-28T04:59:56.757Z	INFO	limb.setup	Creating suction cup manager
2020-05-28T04:59:56.757Z	INFO	limb.setup	Creating controllers
2020-05-28T04:59:56.757Z	INFO	limb.setup	Starting
2020-05-28T04:59:56.758Z	INFO	suctioncup.manager	Starting registration server
2020-05-28T04:59:56.758Z	INFO	controller-runtime.manager	starting metrics server	{"path": "/metrics"}
2020-05-28T04:59:56.758Z	INFO	suctioncup.manager	Starting reconciling queue	{"index": 0}
2020-05-28T04:59:56.758Z	INFO	suctioncup.manager	Starting reconciling queue	{"index": 1}
2020-05-28T04:59:56.858Z	INFO	controller-runtime.controller	Starting EventSource	{"controller": "limb_dl", "source": "kind source: /, Kind="}
2020-05-28T04:59:56.858Z	INFO	controller-runtime.controller	Starting Controller	{"controller": "limb_dl"}
2020-05-28T04:59:56.858Z	INFO	controller-runtime.controller	Starting workers	{"controller": "limb_dl", "worker count": 1}
2020-05-28T05:00:09.387Z	ERROR	suctioncup.registration	Unable to connect adaptor	{"adaptor": "adaptors.edge.cattle.io/ble", "error": "failed to dial adaptor: /var/lib/octopus/adaptors/ble.socket: context deadline exceeded", "errorVerbose": "context deadline exceeded\nfailed to dial adaptor: /var/lib/octopus/adaptors/ble.socket\ngithub.com/rancher/octopus/pkg/suctioncup/adaptor.NewAdaptor\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/adaptor/adaptor.go:56\ngithub.com/rancher/octopus/pkg/suctioncup/registration.(*server).Register\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/registration/server.go:106\ngithub.com/rancher/octopus/pkg/adaptor/api/v1alpha1._Registration_Register_Handler\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/adaptor/api/v1alpha1/api.pb.go:343\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:995\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:1275\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:710\nruntime.goexit\n\t/usr/local/Cellar/[email protected]/1.13.8/libexec/src/runtime/asm_amd64.s:1357"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/guangbochen/Sites/golang/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
github.com/rancher/octopus/pkg/suctioncup/registration.(*server).Register
	/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/registration/server.go:108
github.com/rancher/octopus/pkg/adaptor/api/v1alpha1._Registration_Register_Handler
	/Users/guangbochen/src/rancheredge/octopus/pkg/adaptor/api/v1alpha1/api.pb.go:343
google.golang.org/grpc.(*Server).processUnaryRPC
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:995
google.golang.org/grpc.(*Server).handleStream
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:1275
google.golang.org/grpc.(*Server).serveStreams.func1.1
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:710
2020-05-28T05:00:09.440Z	ERROR	suctioncup.registration	Unable to connect adaptor	{"adaptor": "adaptors.edge.cattle.io/modbus", "error": "failed to dial adaptor: /var/lib/octopus/adaptors/modbus.socket: context deadline exceeded", "errorVerbose": "context deadline exceeded\nfailed to dial adaptor: /var/lib/octopus/adaptors/modbus.socket\ngithub.com/rancher/octopus/pkg/suctioncup/adaptor.NewAdaptor\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/adaptor/adaptor.go:56\ngithub.com/rancher/octopus/pkg/suctioncup/registration.(*server).Register\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/registration/server.go:106\ngithub.com/rancher/octopus/pkg/adaptor/api/v1alpha1._Registration_Register_Handler\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/adaptor/api/v1alpha1/api.pb.go:343\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:995\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:1275\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:710\nruntime.goexit\n\t/usr/local/Cellar/[email protected]/1.13.8/libexec/src/runtime/asm_amd64.s:1357"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/guangbochen/Sites/golang/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
github.com/rancher/octopus/pkg/suctioncup/registration.(*server).Register
	/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/registration/server.go:108
github.com/rancher/octopus/pkg/adaptor/api/v1alpha1._Registration_Register_Handler
	/Users/guangbochen/src/rancheredge/octopus/pkg/adaptor/api/v1alpha1/api.pb.go:343
google.golang.org/grpc.(*Server).processUnaryRPC
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:995
google.golang.org/grpc.(*Server).handleStream
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:1275
google.golang.org/grpc.(*Server).serveStreams.func1.1
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:710
2020-05-28T05:00:26.169Z	INFO	suctioncup.registration.watcher	Watching path	{"path": "/var/lib/octopus/adaptors/modbus.socket"}
2020-05-28T05:00:26.179Z	INFO	controller.deviceLink.adaptorRegistration	Adaptor is registered	{"adaptor": "adaptors.edge.cattle.io/modbus"}
2020-05-28T05:00:26.192Z	INFO	suctioncup.registration.watcher	Watching path	{"path": "/var/lib/octopus/adaptors/ble.socket"}
2020-05-28T05:00:26.197Z	INFO	controller.deviceLink.adaptorRegistration	Adaptor is registered	{"adaptor": "adaptors.edge.cattle.io/ble"}
2020-05-28T06:15:51.145Z	INFO	suctioncup.registration.watcher	Watching path	{"path": "/var/lib/octopus/adaptors/agent-device.socket"}
2020-05-28T06:15:51.150Z	INFO	controller.deviceLink.adaptorRegistration	Adaptor is registered	{"adaptor": "adaptors.edge.cattle.io/agent-device"}
2020-05-28T06:23:35.817Z	INFO	suctioncup.registration.watcher	Unwatching path	{"path": "/var/lib/octopus/adaptors/agent-device.socket"}
2020-05-28T06:24:06.422Z	INFO	suctioncup.registration.watcher	Watching path	{"path": "/var/lib/octopus/adaptors/agent-device.socket"}
2020-05-28T06:24:58.930Z	INFO	suctioncup.registration.watcher	Unwatching path	{"path": "/var/lib/octopus/adaptors/agent-device.socket"}
2020-05-28T06:25:23.127Z	INFO	suctioncup.registration.watcher	Watching path	{"path": "/var/lib/octopus/adaptors/agent-device.socket"}
2020-05-28T06:28:09.354Z	INFO	suctioncup.registration.watcher	Unwatching path	{"path": "/var/lib/octopus/adaptors/agent-device.socket"}
2020-05-28T13:04:53.900Z	INFO	suctioncup.registration.watcher	Watching path	{"path": "/var/lib/octopus/adaptors/agent-device.socket"}
2020-05-28T19:50:31.209Z	ERROR	suctioncup.registration	Unable to connect adaptor	{"adaptor": "adaptors.edge.cattle.io/agent-device", "error": "failed to dial adaptor: /var/lib/octopus/adaptors/agent-device.socket: context deadline exceeded", "errorVerbose": "context deadline exceeded\nfailed to dial adaptor: /var/lib/octopus/adaptors/agent-device.socket\ngithub.com/rancher/octopus/pkg/suctioncup/adaptor.NewAdaptor\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/adaptor/adaptor.go:56\ngithub.com/rancher/octopus/pkg/suctioncup/registration.(*server).Register\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/registration/server.go:106\ngithub.com/rancher/octopus/pkg/adaptor/api/v1alpha1._Registration_Register_Handler\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/adaptor/api/v1alpha1/api.pb.go:343\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:995\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:1275\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:710\nruntime.goexit\n\t/usr/local/Cellar/[email protected]/1.13.8/libexec/src/runtime/asm_amd64.s:1357"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/guangbochen/Sites/golang/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
github.com/rancher/octopus/pkg/suctioncup/registration.(*server).Register
	/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/registration/server.go:108
github.com/rancher/octopus/pkg/adaptor/api/v1alpha1._Registration_Register_Handler
	/Users/guangbochen/src/rancheredge/octopus/pkg/adaptor/api/v1alpha1/api.pb.go:343
google.golang.org/grpc.(*Server).processUnaryRPC
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:995
google.golang.org/grpc.(*Server).handleStream
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:1275
google.golang.org/grpc.(*Server).serveStreams.func1.1
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:710
2020-05-30T16:12:50.223Z	ERROR	suctioncup.registration	Unable to connect adaptor	{"adaptor": "adaptors.edge.cattle.io/agent-device", "error": "failed to dial adaptor: /var/lib/octopus/adaptors/agent-device.socket: context deadline exceeded", "errorVerbose": "context deadline exceeded\nfailed to dial adaptor: /var/lib/octopus/adaptors/agent-device.socket\ngithub.com/rancher/octopus/pkg/suctioncup/adaptor.NewAdaptor\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/adaptor/adaptor.go:56\ngithub.com/rancher/octopus/pkg/suctioncup/registration.(*server).Register\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/registration/server.go:106\ngithub.com/rancher/octopus/pkg/adaptor/api/v1alpha1._Registration_Register_Handler\n\t/Users/guangbochen/src/rancheredge/octopus/pkg/adaptor/api/v1alpha1/api.pb.go:343\ngoogle.golang.org/grpc.(*Server).processUnaryRPC\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:995\ngoogle.golang.org/grpc.(*Server).handleStream\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:1275\ngoogle.golang.org/grpc.(*Server).serveStreams.func1.1\n\t/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:710\nruntime.goexit\n\t/usr/local/Cellar/[email protected]/1.13.8/libexec/src/runtime/asm_amd64.s:1357"}
github.com/go-logr/zapr.(*zapLogger).Error
	/Users/guangbochen/Sites/golang/pkg/mod/github.com/go-logr/[email protected]/zapr.go:128
github.com/rancher/octopus/pkg/suctioncup/registration.(*server).Register
	/Users/guangbochen/src/rancheredge/octopus/pkg/suctioncup/registration/server.go:108
github.com/rancher/octopus/pkg/adaptor/api/v1alpha1._Registration_Register_Handler
	/Users/guangbochen/src/rancheredge/octopus/pkg/adaptor/api/v1alpha1/api.pb.go:343
google.golang.org/grpc.(*Server).processUnaryRPC
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:995
google.golang.org/grpc.(*Server).handleStream
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:1275
google.golang.org/grpc.(*Server).serveStreams.func1.1
	/Users/guangbochen/Sites/golang/pkg/mod/google.golang.org/[email protected]/server.go:710

Result:

Other details that may be helpful:

Except:
should close the adaptor grpc connection in pkg/suctioncup/adaptor/adaptor.go

Related Issues:

device status isn't updated

Environment information:

• Octopus version : master
• Installer : without-webhook

Cluster information:

• Cluster vendor : k3s
• Machine type : vm
• Kubernetes version (use kubectl version): v1.18.2
• Docker version (use docker version): 19.03.8

Machine specification:

Role	Number	Specification (CPU/RAM)
master	1	4U8G
agent	2	4U8G

Steps to reproduce (least amount of steps as possible):

1. Deploy octopus and dummy adaptor
2. Apply dummy/deploy/e2d/dl_specialdevice.yaml
3. Check device yaml

Result:
Device status is not updated.

apiVersion: devices.edge.cattle.io/v1alpha1
kind: DummySpecialDevice
metadata:
  annotations:
    edge.cattle.io/adaptor-name: adaptors.edge.cattle.io/dummy
    edge.cattle.io/adaptor-node: edge-worker
  creationTimestamp: "2020-05-28T09:40:58Z"
  generation: 1
  labels:
    device: living-room-fan
  name: living-room-fan
  namespace: default
  ownerReferences:
  - apiVersion: edge.cattle.io/v1alpha1
    blockOwnerDeletion: true
    controller: true
    kind: DeviceLink
    name: living-room-fan
    uid: 056bf15c-8e59-4e22-875d-ee79f7c825da
  resourceVersion: "1802"
  selfLink: /apis/devices.edge.cattle.io/v1alpha1/namespaces/default/dummyspecialdevices/living-room-fan
  uid: d34d9b97-9018-4dc9-83d3-353b9205f680
spec:
  gear: slow
  "on": true
  protocol:
    location: living_room

Other details that may be helpful:
Octopus master and v0.0.2 image the same issue happens, but not in v0.0.1

octopus integration with rancher 2.4 using catalog app

Feature Request:
we need to support Rancher users to deploy the octopus and edge UI directly from the catalog apps (currently only for k3s clusters)

Todos:

• support to deploy the octopus and edge-ui from the rancher catalog apps (currently only need to support k3s cluster)
  • add questions.yaml to the octopus chart

Support multiple device model in any adaptor.

Improvement:

From the design of adaptor, we should support that one adaptor can recognize many device models, but there is not any attribute can recognize this.

References:

Pass the model attribute during the limb connects to the adaptor.

Related Issues:

Todos:

• Add a new attribute to the adaptor protoc interface.
• Give a case to implement this. It's better to modify the dummy device.

add test to adaptors

Description

There is no integration test and server simulator in existing adaptors

Improvement:

• add integration test to existing adaptors
• add server simulator to test with mock devices

Improve the accuracy of the adaptor's log

Improvement:

The current adaptor's log can not print the device namespaced name.

References:

Adjust logr implementation to support printing logs included device namespaced name.

Related Issues:

Todos:

• Use Fullfill instead of NewDelegatingLogger.

k3s steve api not be able to create new CRD object

k3s steve api not be able to create new CRD object

curl -u "${USERNAME}:${PASSWORD}" \
-X POST \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{"apiVersion":"edge.cattle.io/v1alpha1","kind":"DeviceLink","metadata":{"name":"correct"},"spec":{"adaptor":{"node":"edge-worker","name":"adaptors.edge.cattle.io/dummy","parameters":{"ip":"127.0.0.1"}},"model":{"apiVersion":"devices.edge.cattle.io/v1alpha1","kind":"DummyDevice"},"template":{"metadata":{"labels":{"device":"correct"}},"spec":{"gear":"slow","on":true}}}}' \
'https://192.168.64.5:32636/v1/edge.cattle.io.devicelinks'

Add bluetooth adaptor

Description:
Add Bluetooth adaptor that enables the user to connect to the BLE(Bluetooth low energy) devices.

apiVersion: edge.cattle.io/v1alpha1
kind: DeviceLink
metadata:
  name: xiaomi-temp-rs2200
spec:
  adaptor:
    node: ubuntu
    name: adaptors.edge.cattle.io/ble
    parameters:
      syncInterval: 30
      timeout: 60
  model:
    apiVersion: "devices.edge.cattle.io/v1alpha1"
    kind: "BluetoothDevice"
  template:
    metadata:
      labels:
        device: xiaomi-temp-rs2200
    spec:
      name: "MJ_HT_V1"
      # macAddress: ""
      properties:
      - name: data
        description: XiaoMi temp sensor with temperature and humidity data
        accessMode: NotifyOnly
        visitor:
          characteristicUUID: 226c000064764566756266734470666d
      - name: humidity
        description: Humidity in percent
        accessMode: ReadOnly
        visitor:
          characteristicUUID: f000aa0304514000b000000000000000
          # dataConverter is optional
          dataConverter:
            startIndex: 1
            endIndex: 0
            shiftRight: 2
            orderOfOperations:
            - operationType: Multiply # options are Add/Subtract/Multiply/Divide
              operationValue: "0.03125"
      - name: power-enabled
        description: turn the device power on or off
        accessMode: ReadWrite
        visitor:
          # characteristicUUID, dataWrite and defaultValue is required when accessMode is ReadWrite
          characteristicUUID: f000aa0104514000b000000000000001
          defaultValue: OFF # set the defaultValue by chosen one of option in the dataWrite
          dataWrite:
            ON: [1]
            OFF: [0]
          # dataConverter is optional
          dataConverter:
            startIndex: 1
            endIndex: 0
            shiftRight: 3
            orderOfOperations:
            - operationType: Multiply
              operationValue: "0.03125"

login page

1. Anyone with a static password for the admin account can log in through this page.
2. There is no need to log in again when accessing Edge UI through Rancher proxy

Device link cannot be deleted if node/model does not exist

Steps to reproduce:

1. Set up a local k3d cluster
2. Deploy octopus: k apply -f all_in_one_without_webhook.yaml
3. Deploy dummy adaptor: k apply -f all_in_one.yaml
4. Deploy a dummy device link: k apply -f dl_specialdevice.yaml
5. Delete adaptor by running: k delete -f all_in_one.yaml
6. Delete the device link by running: k delete -f dl_specialdevice.yaml

Result:
The device link cannot be deleted due to NodeExisted/ModelExisted is NotFound

Add MQTT support for *adaptors

Description:
Add MQTT support for existing adaptors(i.e, BLE, Modbus and OPC-UA)

Requirements:

1. one library for all adaptor with encapsulated methods to make send and receive MQTT topics
2. allows user to config MQTT endpoint and other advanced options like TLS, QoS..
3. support 1 or many MQTT topics config and bind each topic to the device property
4. [optional] allows users to config trigger conditions and do write operation to the device property upon received MQTT topic value.

Add annotation label to indicate the Device Model

Support multiple actions in build process management

Improvement:

It's better to support multiple actions in build process management because the combination action is very helpful for testing or rapid development.

References:

Recognize the comma-list action during the building, for example, make octopus build,package,deploy is to run build, package, deploy actions orderly.

Related Issues:

Todos:

• Adjust the build process management scripts.

overview page

First ensure that the static page is implemented, and try to get the data dynamically if time is available.

Device's spec fail to update with dl.spec.template.spec

Steps to reproduce (least amount of steps as possible):

1. Create devicelink, wait for dummydevice created
2. Edit dl.spec.template.spec
3. Check the yaml of dummydevice, finding spec not changed

Result:
Spec of the device stays the same all the time.

Other details that may be helpful:

modbus device protocol config do not change

Environment information:

• Octopus version : master
• Installer : *

Cluster information:

• Cluster vendor : k3s
• Machine type : vm
• Kubernetes version (use kubectl version):
• Docker version (use docker version):

Machine specification:

Role	Number	Specification (CPU/RAM)
master	1	4U8G
agent	2	4U8G

Steps to reproduce (least amount of steps as possible):

1. create a modbus tcp devicelink
2. wait for the device connected
3. change spec.protocol.tcp.ip from devicelink

Result:
The adaptor is still connecting to the old modbus device

Devicelink with DeviceConnected status "unhealthy" fail to update

Environment information:

• Octopus version : master
• Installer : *

Cluster information:

• Cluster vendor : k3s
• Machine type : vm
• Kubernetes version (use kubectl version):
• Docker version (use docker version):

Machine specification:

Role	Number	Specification (CPU/RAM)
master	1	4U8G
agent	2	4U8G

Steps to reproduce (least amount of steps as possible):

1. deploy octopus and adaptor(e.g. modbus)
2. apply devicelink with wrong protocol configuration and check the status to be DeviceConnected unhealthy
3. edit the dl.spec.template.spec

Result:
the dl's status isn't updated

Add Octupus helm3 Chart

Add Octopus helm3 Chart with:

1. all-in-one for all octopus components
2. options to enable/disable each add-on adaptors
3. options to enable/disable UI installation as sub-chart

reform device status property to a standarized object

Description:
the current data of status.properties of each device is different, we need to define a common object of the status property since they all used for displaying the device reported data.

Solution:
use the property object as below:
1.

status:
    properties:
    - dataType: float #string type
      name: temperature #string
      value: "24.1" # string
      updatedAt: "2020-04-03T05:08:46Z" # date

2. add an annotation to the device CRD to describe what values should be displayed for the status.property, e.g,

  devices.edge.cattle.io/status-property: '{"name":"string","dataType":"string","value":"string","updatedAt":"date"}'

add catalog controller & MQTT management

Description:
Background: users should be able to add either local or remote chart library to the k3s cluster and install them using helmchart config(check the example as below), users with the air-gapped environment should be able to configure and enable local helm chart library that provided by the edge.

apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: my-mqtt
  namespace: kube-system
spec:
  chart: emqx
  repo: https://chartmeseum.$namespace:80 # local hosted chart library
  targetNamespace: default

Add Catalog controller

1. allows user to enable helm library(i.e https://github.com/helm/chartmuseum) in the edge UI
   • user should be able to upload a helm chart and store it in the k3s local storage
   • user should be able to config to use other storage option like s3-compatible storage service
2. allows user to list/view catalog apps from the local library
3. allows users to deploy catalogs apps using k3s helmchart CRD.
4. allows user to manage/upgrade and delete catalog apps

Add MQTT chart
Add a curated collection of MQTT chart(i.e https://github.com/cnrancher/mqtt-charts) to the MQTT menu

namespace selector in the main page

1. For interactive methods, please refer to the rancher2.4 new dashboard.
2. When no NS is selected, it means all-namespaces.

make limb to support read k8s configmap and secret resource by name

Improvement:
Read OPC UA registration CA and private key files by providing file names.

References:
Requires changes from both limb and adaptor.
The file name could be a k8s secret name, and handle the read file content in the Octopus limb side and store it as a template file in adaptor for opc-ua registration.

Related Issues:
#32 (comment)

Change all socket file name to *.sock

Improvement:

Usually, the socket file names are suffixed with .sock.

References:

Refactor all socket file name from *.socket to *.sock.

Related Issues:

Todos:

• Refactor the socket file names

Support log parameterization in adaptors

Improvement:

It's better to parameterize the log of all adaptors.

References:

• Provide a logr log implementation.
• Parameterize all adaptors.

Related Issues:

Todos:

• Implement a logr util based on zap.
• Adjust the log parameters entrance of brain and limb.
• Add a log parameters entrance for all adaptors.

devicelink should add log and retry if no reference was found in the same namespace

Environment information:

• Octopus version : master
• Installer : all-in-one

Cluster information:

• Cluster vendor : k3s
• Machine type : vm
• Kubernetes version (use kubectl version):
• Docker version (use docker version):

Machine specification:

Role	Number	Specification (CPU/RAM)
master	1	4U8G
agent	2	4U8G

Steps to reproduce (least amount of steps as possible):

devicelink should add a log if no secret or configmap reference was found in the same namespace, also if adding the reference resource after maximum reconcile it will stop read the reference resource unless re-create the DL.

1. deploy octopus with mqtt support and apply the dummy all-in-one.yaml
2. run kubectl apply -f ./deploy/e2e/dl_specialdevice_with_mqtt.yaml to a new namespace and skip to create the secret certificate
3. the living-room-fan DL is showing as healthy but the MQTT connection is failed due to empty certificate and limb show error log as below

Result:

2020-06-12T08:15:14.031Z	ERROR	controller.deviceLink	Unable to fetch the reference parameters of DeviceLink	{"deviceLink": "testx/living-room-fan", "error": "Secret \"living-room-fan-mqtt-ca\" not found"}
github.com/rancher/octopus/pkg/util/log/zap.(*logger).Error
	/Users/guangbochen/src/rancheredge/octopus/pkg/util/log/zap/logr.go:33
github.com/rancher/octopus/pkg/limb/controller.(*DeviceLinkReconciler).Reconcile
	/Users/guangbochen/src/rancheredge/octopus/pkg/limb/controller/devicelink.go:299
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/Users/guangbochen/Sites/golang/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:256
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/Users/guangbochen/Sites/golang/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:232
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
	/Users/guangbochen/Sites/golang/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:211
k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1
	/Users/guangbochen/Sites/golang/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:152
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/Users/guangbochen/Sites/golang/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:153
k8s.io/apimachinery/pkg/util/wait.Until
	/Users/guangbochen/Sites/golang/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:88

devicelink should add a log and retry if no reference was found in the same namespace

Other details that may be helpful:

Except:

1. add a log to the DL if no reference resource was found.
2. add a log to the DL if maximum reconcile is reached for getting the reference resource(this is because we don't list watch those resources).

Related Issues:

octopus all-in-one image need to support multi-arch

octopus all-in-one image need to support multi-arch design since our k3s devices include arm64/amd64 and armhf arch.

refer: https://www.docker.com/blog/multi-arch-images/

IoT mngt page

1. 前后端开发联调确认表单项和具体逻辑
2. UI设计介入优化页面设计

Reduce reconciling and usless connections

Improvement:

As the description of #74, I found that imperfect design leads to continuous reconcile, which makes the CPU abnormally high. At the same time,
the condition moves from DeviceConnected to others that could leak a connection.

References:

Try to refactor the whole reconciling logic and disconnect the connection when it is no longer available.

Related Issues:

#74

Todos:

• Refactor reconciling logic
• Disconnect useless connection

Add k3d as an option for CI/CD automation scripts

Add k3d as an option for CI/CD automation scripts

websocket watch a not allowed resource

Environment information:

• Octopus version : master
• Installer : *

Cluster information:

• Cluster vendor : k3s
• Machine type : vm
• Kubernetes version (use kubectl version):
• Docker version (use docker version):

Machine specification:

Role	Number	Specification (CPU/RAM)
master	1	4U8G
agent	2	4U8G

Steps to reproduce (least amount of steps as possible):

1. install the octopus and the dummyDevice adaptor through the all-in-one.yaml
2. ks apply -f dl_specialdevice.yaml to install the specialdevice
3. launch the latest edge-api-server and the UI and check the WebSocket connection in the console, for resource like metrics.k8s.io.nodemetrics,metrics.k8s.io.podmetrics,componentstatus ,etc will be auto-disconnect with the following message.

"error":"watch not allowed"

Result:
Ui can't update data in pod list by wath method.

Other details that may be helpful:
These resources are the ones that do not allow watch methods in k8s, and it is recommended to update the data by using regular rounds of training

Except:

Related Issues:

left menus of the main page

• 全局概览
• 基础设施
  • 节点管理
  • 系统服务
  • 命名空间
  • 存储类别
• 工作负载
  • 工作负载
  • 密文管理
  • 配置映射
• IoT管理
• 边缘AI
• MQTT管理（待定）
• 系统管理
  • 系统设置
  • 数据备份

全局概览/IoT管理/系统设置可点击，其他项由于后端尚未实现，暂时不可点击。

Add octopus metrics

Description:
Add metrics server using prometheus exporter that allows user to view octopus component metrics

add device template revisions

Description:
Add template revision to the device template and allow the user to config and update device templates with multiple versions.

Requirements:

1. one device template can have one or many device template revisions.
2. while user creating the deviceLink using a specific device template revision, it should add metadata info to the deviceLink annotations(i.e device template name, revision name and revision generation)

Perceived adaptor unregister event

Add logic to perceive the adaptor unregistering.

Add MQTT adaptor

Description:
Add MQTT adaptor that allows user to config MQTT setting and add custom MQTT device upon the topics.

Libraries:
https://github.com/search?l=Go&q=mqtt&type=Repositories

Add deviceProperty templates to the edge api

Description:
Add device property templates feature to the edge-api-controller, those templates will be stored as deviceTemplate CRD in the k8s.

Requirements:

1. the template controller should validate against those device property templates upon the device custom resource definition.
2. allows filtering templates by the created user and device type annotations

Add OPC-UA Adaptor

add OPC-UA Adaptor
refer https://github.com/gopcua/opcua

add more related apps to the edge catalog

Add more apps to the edge catalogs (1st stage to ensure the installation is succeeded)
* [ ] Fluent-bit https://github.com/helm/charts/tree/master/stable/fluent-bit
* [ ] Flink-operator https://github.com/GoogleCloudPlatform/flink-on-k8s-operator

Add drone CI/CD integration for the octopus

Description:
Add drone CI/CD integration for the octopus project

Requirements:

• multi-arch build support for arm, amd64 and armhf
• drone scripts to run the CI lint and integration test against PR submission automatically
• run CI test and CD release to the docker hub when adding a new release tag

Add agent-device adaptor

Description:
add agent-device adaptor to the octopus that allows users to manage a group of devices (1…x) which are running all the same containerized applications (dockerfile, docker-compose, …. defined) on devices with the same architecture.
those agent-device are Linux devices (different architectures) and are compatible with k3s agent node.

Add Modbus adaptor

Support operating devices using ModbusTCP/ModbusRTU protocol for communication with the limb by Modbus adaptor.

The user would be able to deploy CRD for Modbus devicelink, which specifies

• parameters (IP address and port for TCP, serial port for RTU), through which connecting to Modbus devices
• properties, with the corresponding register type, starting address and register quantity for reading and writing data from/to different Modbus device registers.

The adaptor configures the devices according to the CRD and reads the status of the device to report back to the limb.