Comments (13)
Implementing Calico compatibility is possible (and used to be in the repo) however it was recently taken out because Calico’s default IPAM method was changed so that it no longer uses the podCIDR allocated by the K8s node controller.
To support this, Kilo will need to add a Calico client and read Calico IPPool CRs to determine the subnets allocated to each node.
If you’d like to give re-implementing it a shot, please let me know; I would be happy to review a PR. Otherwise, it’s high on my list for the project.
from kilo.
any further progress on this?
from kilo.
Thank you for replying. I'd be happy to give it a shot and see what I can do about that. Any suggestions to start?
from kilo.
My understanding that this effort will block using Kilo on GKE, is that correct?
from kilo.
We're also interested in in seeing this implemented! Our use case is to leverage the VPN feature to build site-to-site VPN for connecting legacy service with a Kubernetes cluster.
from kilo.
My understanding that this effort will block using Kilo on GKE, is that correct?
Yes it will, unfortunately kilo can't be deployed on GKE at the moment.
from kilo.
No progress on this? :(
from kilo.
yeah can we get calico suport, alot of us have non-GKE environments
from kilo.
Was there any progress on this?
from kilo.
Running Kilo on top of a GKE cluster is still not possible?
from kilo.
Can't we just re-add calico compatibility as it is in case the podCIDDR is the only issue? Calico IPools are great to have subnets per site and use different site specific or top of rack BGP peers. The IP block assigned to a node is stored in a BlockAffinity CR. We are using a script to update the podCIDR from the generated blockaffinity after a new node joined so in our case this would always match.
It is somewhat hacky but this is what we do to fix this issue:
nodeList=$(kubectl get nodes -o jsonpath='{.items[*].metadata.name}')
for node in $nodeList; do
echo -n "* Node $node..."
calicoName=$(kubectl get blockaffinities -o jsonpath='{.items[*].metadata.name}' | tr " " "\n" | awk "/$node/ {print $1}")
echo -n " subnet=${calicoName}"
calicoCIDR=$(kubectl get blockaffinities $calicoName -o jsonpath='{.spec.cidr}')
echo -n ", ${calicoCIDR}"
podCIDR=$(kubectl get node $node -o jsonpath='{.spec.podCIDR}')
echo -n " == ${podCIDR}"
if [ "${podCIDR}" == "${calicoCIDR}" ]; then
echo " OK"
continue
fi
if [ -z "${calicoCIDR}" ]; then
echo " ERROR"
echo " Could not determine calico CIDR. Did you switch the nodes site? In this case make"
echo " sure there is only one blockaffinity object for this node."
continue
fi
echo " ERROR"
echo
echo "!!!WARNING: Continuing might disrupt workloads running on the node!!!"
echo
echo -n "Shall we continue to fix it now (node should be drained before)? [y/n]? "
read a
if [ "$a" != "y" ]; then
continue
fi
echo "Saving node yaml to ~/${node}.yaml..."
kubectl get node $node -o yaml >~/${node}.yaml
echo "Replacing CIDR (s/${podCIDR}/${calicoCIDR}/)..."
sed -i "s/${podCIDR/\//\\/}/${calicoCIDR/\//\\/}/" ~/${node}.yaml
echo "Deleting node..."
kubectl delete node $node
kubectl create -f ~/${node}.yaml
echo "FIXED!"
done
from kilo.
This is a blocker for usage with LKE (linode) too.
from kilo.
I made subnet being able to use podCIDR querying calico blockaffinity resource. That made calico start, and this non-ready node error is not showing anymore. Now I'm trying to understand the calico compatibility layer you wrote to see if I can make it work.
from kilo.
Related Issues (20)
- Installation help HOT 3
- NAT Node not ready, cannot ping wireguard HOT 1
- The pod kilo-* in node was evicted when the memory is out HOT 2
- Connection to K8S Service - SourceIP is not preserved (Source NAT) HOT 7
- istio support HOT 1
- Add Kilo in Cilium USER.md HOT 4
- 在k3s上运行失败 HOT 2
- Peering clusters behind nat HOT 3
- Calico or Althea support HOT 1
- [Question]How Kilo works?
- nodes with same subnet in cluster
- spamming error "exit status 1: iptables v1.8.4 (nf_tables): table `filter' is incompatible, use 'nft' tool." HOT 8
- Request: Add feature to specify source ip address for all egress HOT 7
- Cluster with control-plane running in GKE and edge nodes behind NAT HOT 1
- does kilo support aws eks with aws-vpc-cni? HOT 2
- Use private network where possible in fully meshed network HOT 4
- Use Pod/Service IP as Egress Point / Egress Gateway Implementation
- Kilo Incorrectly Chooses an eth0 IP Over Node's Configured Internal IP HOT 6
- kgctl connect improve availability
- Has anyone tried to hack `k0s` support together?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kilo.