jakubkulhan/ingress-merge

Merge Ingress Controller for Kubernetes

Merge Ingress Controller

Merge Ingress Controller combines multiple ingress resources into a new one.

Motivation

Different ingress controllers behave differently when creating services/load balancers satisfying the ingress resources of the managed class. Some create single service/LB for all ingress resources, some merge resources according to hosts or TLS certificates, other create separate service/LB per ingress resource.

E.g. AWS ALB Ingress Controller creates a new load balancer for each ingress resource. This can become quite costly. There is an issue to support reusing ALBs across ingress resources, however, it won't be implemented anytime soon.

Merge Ingress Controller allows you to create ingress resources that will be combined together to create a new ingress resource that will be managed by different controller.

Setup

Install via Helm:

helm install --namespace kube-system --name ingress-merge ./helm

Example

Create multiple ingresses & one config map that will provide parameters for the result ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: foo-ingress
  annotations:
    kubernetes.io/ingress.class: merge
    merge.ingress.kubernetes.io/config: merged-ingress
spec:
  rules:
  - host: foo.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: foo-svc
          servicePort: 80

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: bar-ingress
  annotations:
    kubernetes.io/ingress.class: merge
    merge.ingress.kubernetes.io/config: merged-ingress
spec:
  rules:
  - host: bar.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: bar-svc
          servicePort: 80

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: merged-ingress
data:
  annotations: |
    kubernetes.io/ingress.class: other

Merge Ingress Controller will create new ingress resource named by the config map with rules combined together:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: merged-ingress
  annotations:
    kubernetes.io/ingress.class: other
spec:
  rules:
  - host: bar.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: bar-svc
          servicePort: 80

  - host: foo.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: foo-svc
          servicePort: 80

Annotations

Annotation	Default Value	Description	Example
kubernetes.io/ingress.class		Use merge for this controller to take over.	kubernetes.io/ingress.class: merge
merge.ingress.kubernetes.io/config		Name of the ConfigMap resource that will be used to merge this ingress with others. Because ingresses do not support to reference services across namespaces, neither does this reference. All ingresses to be merged, the config map & the result ingress use the same namespace.	merge.ingress.kubernetes.io/config: merged-ingress
merge.ingress.kubernetes.io/priority	0	Rules from ingresses with higher priority come in the result ingress rules first.	merge.ingress.kubernetes.io/priority: 10
merge.ingress.kubernetes.io/result		Marks ingress created by the controller. If all source ingress resources are deleted, this ingress is deleted as well.	merge.ingress.kubernetes.io/result: "true"

Configuration keys

Key	Default Value	Description	Example
name	name of the ConfigMap	Name of the result ingress resource.	name: my-merged-ingress
labels		YAML/JSON-serialized labels to be applied to the result ingress.	labels: '{"app": "loadbalancer", "env": "prod"}'
annotations	{"merge.ingress.kubernetes.io/result":"true"}	YAML/JSON-serialized labels to be applied to the result ingress. merge.ingress.kubernetes.io/result with value true will be always added to the annotations.	annotations: '{"kubernetes.io/ingress.class": "alb"}
backend		Default backend for the result ingress (spec.backend). Source ingresses must not specify default backend (such ingresses won't be merged).	backend: '{"serviceName": "default-backend-svc", "servicePort": 80}

License

Licensed under MIT license. See LICENSE file.

ingress-merge's Issues

Question: Multiple configmap?

I have multiple charts, and they have their own configmaps , and resources defined inside the templates of each helm chart.
just wanted to know, wherever we define, ingress.yaml in that adding the same configmap.yaml will be supported or how you suggest using?
let me know in case not able to demonstrate my query.

doesn't owner health-check endpoint

If we annotate a particular ingress with custom health check endpoint using
alb.ingress.kubernetes.io/healthcheck-path: /cust

The same is not automatically reflected in the required target group. Also, it would be great if the target group name can have the actual ingress name rather than ingress-merge.

Can't add default backend with Ingress resource

Currently the only way to add default backend configuration is through the ingress-merge ConfigMap. It will be useful to be able to define the default backend in Ingress resource. This will enable the following scenario:

1. Create ingress-merge ConfigMap without default backend.
2. Deploy service A with Ingress resource.
3. Test service A.
4. Deploy service B which uses service A and which adds the default backend through Ingress resource.
5. Test service B.

Currently service B should be deployed before the ingress-merge ConfigMap is created and before service A is deployed. If service B can add the default backend in its Ingress resource, the ingress-merge controller will support the scenario described above.

The feature proposal is:

1. Keep the current workflow.
2. Respect default backend configuration from Ingress resource.
3. Fail if there is default backend in the ingress-merge ConfigMap and in some Ingress resource which is configured to use the ingress-merge ConfigMap.
4. Fail if two or more Ingress resources configured to use one ingress-merge ConfigMap have default backend configuration.

Merging annotations ?

Is this possible to merge annotations from several ingresses into the generated one ?

I mean i have some redirection rules in the original annotations that are not merged into the generated annotation, is there a way to get them merged somehow ?

ALB multiple certificates

Hi
I want to use multiple certificates for ALB Ingress Controller.
https://kubernetes-sigs.github.io/aws-alb-ingress-controller/guide/ingress/annotation/#ssl

I have to join alb.ingress.kubernetes.io/certificate-arn annotation with a comma.
Can ingress-merge do that?

Re-discovery and recovery?

So I'm just getting started but what happens in this scenario?

1. I create config map for merge.
2. I create some ingress objects that in turn create 1 "master" ingress object.
3. the ingress-merge controller pod gets restarted.
4. I want to update some of my "child" ingress objects

From what I can tell, the new ingress-merge controller does not know that it created the "master" ingress object and can't create a new one because of the name clash?

E0410 20:23:09.917300       1 controller.go:422] Could not create ingress [mynamespace/my-service]: ingresses.extensions "my-service" already exists

Am I using it wrong?

Question: Grouping ingress

How can we create multiple load balancer? Since each alb has limitations on rules and target groups
Rules per load balancer (not counting default rules): 100
Targets per load balancer: 1000

Would be helpful if we can have another annotation which help in grouping the resources to a load balancer.

Does not work with latest Ingress apiVersion

I updated my Ingress manifest to this apiVersion and this doesn't work anymore, i.e., an ALB is NOT created.

apiVersion: networking.k8s.io/v1
kind: Ingress
...

Setting Priority Doesn’t Work

When I use the set priority annotation in the docs it causes ingress rules to fail to be added to the ALB. Has anyone gotten this to work? It’s a perfect idea, but I don’t think it works.