plusserver/kubernetes-icinga

Kubernetes / Icinga2 integration.

kubernetes-icinga's Issues

How do you cleanly teardown the monitoring?

Whats the procedure for cleanly tearing down the monitoring? Thus removing the hosts from the icinga server?

Error when building: cannot convert klog.V(9) (type klog.Verbose) to type bool

When building using the Dockerfile I get the errors below.
I'm trying to build this from source because I need to add the authentication plugins

 docker build -t foo .
Sending build context to Docker daemon  415.2kB
Step 1/9 : FROM golang:latest AS build
 ---> a1072a078890
Step 2/9 : WORKDIR /go/src/github.com/Nexinto/kubernetes-icinga
 ---> Using cache
 ---> 963059b61f67
Step 3/9 : COPY . .
 ---> Using cache
 ---> 357a1ffe1522
Step 4/9 : RUN go get k8s.io/client-go/...
 ---> Running in 1519be178146
# k8s.io/client-go/transport
/go/src/k8s.io/client-go/transport/round_trippers.go:70:11: cannot convert klog.V(9) (type klog.Verbose) to type bool
/go/src/k8s.io/client-go/transport/round_trippers.go:72:11: cannot convert klog.V(8) (type klog.Verbose) to type bool
/go/src/k8s.io/client-go/transport/round_trippers.go:74:11: cannot convert klog.V(7) (type klog.Verbose) to type bool
/go/src/k8s.io/client-go/transport/round_trippers.go:76:11: cannot convert klog.V(6) (type klog.Verbose) to type bool
# k8s.io/utils/trace
/go/src/k8s.io/utils/trace/trace.go:100:57: invalid operation: stepThreshold == 0 || stepDuration > stepThreshold || klog.V(4) (mismatched types bool and klog.Verbose)
/go/src/k8s.io/utils/trace/trace.go:112:56: invalid operation: stepThreshold == 0 || stepDuration > stepThreshold || klog.V(4) (mismatched types bool and klog.Verbose)
The command '/bin/sh -c go get k8s.io/client-go/...' returned a non-zero code: 2

I'm totally new to Go, any ideas?!

hostgroup cannot be created

My API user is unable to create a hostgroup even though permissions = ["*"]

[2019-03-10 00:27:23 -0600] information/HttpServerConnection: Request: GET /v1/objects/hosts/kubernetes.traefik.deploy-traefik (from [x.x.x.x]:44128, user: icinga)
[2019-03-10 00:27:23 -0600] information/HttpServerConnection: Request: GET /v1/objects/hostgroups/kubernetes.traefik (from [x.x.x.x]:44130, user: icinga)
[2019-03-10 00:27:23 -0600] information/HttpServerConnection: Request: PUT /v1/objects/hosts/kubernetes.traefik.deploy-traefik (from [x.x.x.x]:44128, user: icinga)
[2019-03-10 00:27:23 -0600] information/ConfigItem: Committing config item(s).
[2019-03-10 00:27:23 -0600] critical/config: Error: Validation failed for object 'kubernetes.traefik.deploy-traefik' of type 'Host'; Attribute 'groups': Object 'kubernetes.traefik' of type 'HostGroup' does not exist.
Location: in /var/lib/icinga2/api/packages/_api/x.x.x.x1448388007-1/conf.d/hosts/kubernetes.traefik.deploy-traefik.conf: 6:2-6:34
/var/lib/icinga2/api/packages/_api/x.x.x.x-1448388007-1/conf.d/hosts/kubernetes.traefik.deploy-traefik.conf(4):  check_command = "check_kubernetes"
/var/lib/icinga2/api/packages/_api/x.x.x.x-1448388007-1/conf.d/hosts/kubernetes.traefik.deploy-traefik.conf(5):  display_name = "kubernetes.traefik.deploy-traefik"
/var/lib/icinga2/api/packages/_api/x.x.x.x-1448388007-1/conf.d/hosts/kubernetes.traefik.deploy-traefik.conf(6):  groups = [ "kubernetes.traefik" ]
                                                                                                                                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2019-03-10 00:27:23 -0600] critical/config: 1 error
[2019-03-10 00:27:23 -0600] information/HttpServerConnection: Request: PUT /v1/objects/hostgroups/kubernetes.traefik (from [x.x.x.x]:44130, user: icinga)
[2019-03-10 00:27:23 -0600] critical/HttpServerConnection: Unhandled exception while processing Http request: Error: Missing permission: objects/create/hostgroup

I tried added explicit permissions to no avail:

object ApiUser "icinga" {
  password = "xxxxx"
  permissions = [ "*", "objects/create/hostgroup", "objects/create/host" ]
}

Am I incorrect in my assumption that the icinga api user should be able to create a hostgroup without there first being one in groups.conf?

v2.6.2 (Yes, quite out of date)

Updating controller-gen.yaml and zz_generated_controller.go

Hi,
i'm playing around with the code and trying to extend it's functionality but i'm stuck when it comes to updating controller-gen.yaml and zz_generated_controller.go files.

It seems that some tool/script is used to generate zz_generated_controller.go file from controller-gen.yaml. I cannot find it in this repo nor in any comments in source code.

Could you point me to the tool/script used for generating this controller file?

Build Kubernetes-icinga.go

i have this error when i try to build kubernetes-icinga.go :

command-line-arguments

./kubernetes-icinga.go:82:8: undefined: Controller
./kubernetes-icinga.go:92:16: undefined: HostGroupMapping
./kubernetes-icinga.go:94:16: undefined: HostMapping
./kubernetes-icinga.go:96:16: undefined: HostGroupMapping

Do you know why ? he don't know this values ?
thanks

Set `zone` attribute on the created objects

Our Icinga Masters do not have access to the Kubernetes API. Would it be possible to set the zone attribute, so the hosts are created in another zone than master?

[Feature] Restructure per Namespace

Hello,

i stumpled across this project and tried it. I really like how it is working but i have an idea which you may consider and i can help develop with if you want.

For an better overview i would like to see 1 Host per Namespace which then includes the deployments/replica and so on. Right now the mass number of "hosts" might be confusing.

Regards

how to send kubeconfig to check_kubernetes ?

Hi,

I'm currently to monitor multiple kubernetes cluster, with an external icinga2 server.
On my icinga2 server2, I have the differents kubeconfig files for each cluster.

icinga2 service:

object CheckCommand "check_kubernetes" {

   command = [
     PluginDir + "/check_kubernetes"
   ]

   arguments = {
     "-kubeconfig" = "$kubernetes_config$"
     "-type" = "$kubernetes_type$"
     "-name" = "$kubernetes_name$"
     "-namespace" = "$kubernetes_namespace$"
   }

}

Where can I define the kubeconfig path in the deployment on kubernetes-icinga in the differents clusters ?

Not recreating a monitored deployment

Hi there,

Really interesting project but just hit this issue:

When you remove a k8s deployment that is being monitored, the deployment (host) gets removed from icinga. However, recreating the deployment with the same name doesn't seem to recreate the deployment (host) in icinga.

Creating new deployments (hosts) with a different name works fine.

Digging through all the aggregated log files of all the containers I've come up with a few interesting entries (RBAC DENY, http 403, 404) but not sure if these are correlated:

September 11th 2018, 23:23:33.000 | time="2018-09-11T22:23:33Z" level=info msg="creating icinga host 'kubernetes.mynamespace.myapp'" <------ NOTE THIS DOESN'T CREATE A HOST

September 11th 2018, 23:23:33.000 | RBAC DENY: user \"system:serviceaccount:kube-system:kubernetes-icinga\" groups [\"system:serviceaccounts\" \"system:serviceaccounts:kube-system\" \"system:authenticated\"]

September 11th 2018, 23:23:33.000 | POST /api/v1/namespaces/mynamespace/events: (716.937µs) 403 [[kubernetes-icinga/v0.0.0 (linux/amd64) kubernetes/$Format] 10.152.122.18:46050]


September 11th 2018, 23:23:42.000 | DELETE /apis/icinga.nexinto.com/v1/namespaces/kube-system/hosts/po-kubernetes-icinga-7c866db4dc-mjbq7: (8.043371ms) 404 [[kubernetes-icinga/v0.0.0 (linux/amd64) kubernetes/$Format] 10.152.122.18:46050]

September 11th 2018, 23:23:42.000 | GET /api/v1/namespaces/kube-system/configmaps/kube-dns: (6.028904ms) 404 [[kubelet/v1.9.2+coreos.0 (linux/amd64) kubernetes/b427929] 127.0.0.1:33342]

September 11th 2018, 23:23:42.000 | DELETE /apis/icinga.nexinto.com/v1/namespaces/kube-system/hosts/rs-kubernetes-icinga-7c866db4dc: (3.967604ms) 404 [[kubernetes-icinga/v0.0.0 (linux/amd64) kubernetes/$Format] 10.152.122.18:46050]

September 11th 2018, 23:23:42.000 | DELETE /apis/icinga.nexinto.com/v1/namespaces/kube-system/hosts/po-kubernetes-icinga-7c866db4dc-mjbq7: (8.043371ms) 404 [[kubernetes-icinga/v0.0.0 (linux/amd64) kubernetes/$Format] 10.152.122.18:46050]

September 11th 2018, 23:23:42.000 | GET /api/v1/namespaces/kube-system/configmaps/kube-dns: (6.028904ms) 404 [[kubelet/v1.9.2+coreos.0 (linux/amd64) kubernetes/b427929] 127.0.0.1:33342]

September 11th 2018, 23:23:42.000 | DELETE /apis/icinga.nexinto.com/v1/namespaces/kube-system/hosts/rs-kubernetes-icinga-7c866db4dc: (3.967604ms) 404 [[kubernetes-icinga/v0.0.0 (linux/amd64) kubernetes/$Format] 10.152.122.18:46050]

Many thanks,
Wayne

Why the need for CRDs?

Salve, I was happy to find your project because I was looking for a way to dynamically create objects in Icinga to attach checks to. Your project seems to be exactly about that.

What I do not understand yet, is why the objects are also created in Kubernetes as CRDs. The objects to be checked are objects in Kubernetes and they should be objects in Icinga, but why do we need a third kind of object?

The controller needs to synchronize the state between Kubernetes API and Icinga API continuously anyway, so how do these CRDs help with that?

Set display_name to something shorter

Looking at the resulting objects in Icinga, I think that the displayed names are maybe too noisy and setting display_name to the actual object name without prefixes would be less overwhelming.