Comments (13)
I just filed an issue into CAPI and then we can track that up to Neutron if that's the path to go, but a revert would be nice for now.
from magnum-cluster-api.
I just did some more troubleshooting. setting this Security Group
openstack security group rule create 76192aca-20eb-43e6-bd1c-331979e57157 --remote-group ba33108a-ceb1-4ef0-98cd-455fa916166f --protocol ipip --ingress
creates this iptables rule:0 0 RETURN 94 -- * * 0.0.0.0/0 0.0.0.0/0 match-set NIPv476192aca-20eb-43e6-bd1c- src
So that does get translated into iptables 'correctly' as ipip = proto 94 according to https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml
So it seems that the protocol that needs to be allowed on iptables would be '4': 'ipencap' instead of '94': 'ipip'...? https://github.com/openstack/neutron-lib/blob/75ee705ed5f4144483686d7e237ba78afe6d1dff/neutron_lib/constants.py#L459This is some good research, I wonder if inside
iptables
, it used to use94
at some point and so the translation is failing depending on the OS.yeap,
94
is KA9Q NOS compatible IP over IP tunneling and here is a bit more explanation https://puck.nether.net/lists/juniper-nsp/1269.html
useful to know!
anyways, we've got the fix merged and backported into 0.7, the CAPO team will make a release on Monday for 0.7.1 and by using that these issues will be solved :)
kubernetes-sigs/cluster-api-provider-openstack#1493
from magnum-cluster-api.
looks like this is a default from here and we could override it if we wanted
https://github.com/kubernetes-sigs/cluster-api-provider-openstack/blob/5a17c37f0a0d4d44211f2339a6893752b43e11da/pkg/cloud/services/networking/securitygroups_rules.go#L79-L115
from magnum-cluster-api.
We should probably bubble this up to the Cluster API project, which might bubble up to Neutron, since this feels like a Neutron bug.
from magnum-cluster-api.
@mnaser I think this is the right approach fixing this upstream. However how can we mitigate this in fast way in the meantime?
from magnum-cluster-api.
I just did some more troubleshooting.
setting this Security Group
openstack security group rule create 76192aca-20eb-43e6-bd1c-331979e57157 --remote-group ba33108a-ceb1-4ef0-98cd-455fa916166f --protocol ipip --ingress
creates this iptables rule:
0 0 RETURN 94 -- * * 0.0.0.0/0 0.0.0.0/0 match-set NIPv476192aca-20eb-43e6-bd1c- src
So that does get translated into iptables 'correctly' as ipip = proto 94 according to https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml
So it seems that the protocol that needs to be allowed on iptables would be '4': 'ipencap' instead of '94': 'ipip'...?
https://github.com/openstack/neutron-lib/blob/75ee705ed5f4144483686d7e237ba78afe6d1dff/neutron_lib/constants.py#L459
from magnum-cluster-api.
According to IANA and the Calico Documentaiton Calico uses IP in IP which is IP protocol number 4 not 94. https://docs.tigera.io/calico/3.25/networking/configuring/vxlan-ipip
Cluster API seems to set this wrong from my point of view.
EDIT: @mnaser
This is causing the problem. It is only merged in main but not 0.7 release. What concern me that is passed CI flawless and makes the cluster unusable.
replace "4" with "ipip"
kubernetes-sigs/cluster-api-provider-openstack#1290
but still it is a neutron bug
https://docs.openstack.org/api-ref/network/v2/index.html?expanded=create-security-group-rule-detail#security-group-rules-security-group-rules
from magnum-cluster-api.
I just did some more troubleshooting. setting this Security Group
openstack security group rule create 76192aca-20eb-43e6-bd1c-331979e57157 --remote-group ba33108a-ceb1-4ef0-98cd-455fa916166f --protocol ipip --ingress
creates this iptables rule:0 0 RETURN 94 -- * * 0.0.0.0/0 0.0.0.0/0 match-set NIPv476192aca-20eb-43e6-bd1c- src
So that does get translated into iptables 'correctly' as ipip = proto 94 according to https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtmlSo it seems that the protocol that needs to be allowed on iptables would be '4': 'ipencap' instead of '94': 'ipip'...? https://github.com/openstack/neutron-lib/blob/75ee705ed5f4144483686d7e237ba78afe6d1dff/neutron_lib/constants.py#L459
This is some good research, I wonder if inside iptables
, it used to use 94
at some point and so the translation is failing depending on the OS.
from magnum-cluster-api.
According to IANA and the Calico Documentaiton Calico uses IP in IP which is IP protocol number 4 not 94. https://docs.tigera.io/calico/3.25/networking/configuring/vxlan-ipip
Cluster API seems to set this wrong from my point of view.
EDIT: @mnaser
This is causing the problem. It is only merged in main but not 0.7 release. What concern me that is passed CI flawless and makes the cluster unusable.
replace "4" with "ipip" kubernetes-sigs/cluster-api-provider-openstack#1290
but still it is a neutron bug https://docs.openstack.org/api-ref/network/v2/index.html?expanded=create-security-group-rule-detail#security-group-rules-security-group-rules
For the record, we're not running into that issue.. however, that mapping is not the source of truth, see this: https://github.com/openstack/neutron/blob/master/neutron/agent/linux/iptables_firewall.py#L745-L773
I'm curious to see if your environments somehow are seeing a mismatch... could you double check that?
from magnum-cluster-api.
ok, have a fix up:
kubernetes-sigs/cluster-api-provider-openstack#1489
now this is technically more of a CAPI issue than the driver, but we'll close this once we fix this upstream and also fix it inside Atmosphere
from magnum-cluster-api.
two things:
- fix for
hack/stack.sh
is done, so this won't happen anymore - fix for atmosphere is done: vexxhost/atmosphere#334
will keep this open to track CAPO :)
from magnum-cluster-api.
ugh, not so quick
this was included in 0.7.0, going to keep working with upstream on a fix soon, if not, we'll do our own local patch for it.
from magnum-cluster-api.
I just did some more troubleshooting. setting this Security Group
openstack security group rule create 76192aca-20eb-43e6-bd1c-331979e57157 --remote-group ba33108a-ceb1-4ef0-98cd-455fa916166f --protocol ipip --ingress
creates this iptables rule:0 0 RETURN 94 -- * * 0.0.0.0/0 0.0.0.0/0 match-set NIPv476192aca-20eb-43e6-bd1c- src
So that does get translated into iptables 'correctly' as ipip = proto 94 according to https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml
So it seems that the protocol that needs to be allowed on iptables would be '4': 'ipencap' instead of '94': 'ipip'...? https://github.com/openstack/neutron-lib/blob/75ee705ed5f4144483686d7e237ba78afe6d1dff/neutron_lib/constants.py#L459This is some good research, I wonder if inside
iptables
, it used to use94
at some point and so the translation is failing depending on the OS.
yeap, 94
is KA9Q NOS compatible IP over IP tunneling and here is a bit more explanation https://puck.nether.net/lists/juniper-nsp/1269.html
from magnum-cluster-api.
Related Issues (20)
- Catch up the latest changes in upstream ovn-kubernetes
- bug: nodegroup delete is failing when there are more than 2 nodegroups
- bug: failed to update cluster template
- bug:ci pr body is mandatory now HOT 2
- OVN as backend for octavia HOT 6
- add exception for unsupported cni
- Cluster creation failed with vexxhost images HOT 8
- OVN octavia backend unsuported algorithm HOT 5
- Publish conformance tests for all versions
- fr: support CA cert rotation
- Stop installing Calico from docker.io
- Enable encryption between kubelet and kube-apiserver on Magnum clusters
- feature: add option to allow cinder CSI availability zone override HOT 4
- `KeyError: 'annotations'` during cluster updates
- `KeyError: 'status'` for node group updates
- Neutron floating IP delete policy cause cluster to stuck in delete progress
- Failed rolling upgrade leaves wrong template ID
- Failed rolling upgrade leaves cluster stuck in UPDATE_FAILED state
- Reduce lock time for operations
- Make use of servergroups for cluster nodes HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from magnum-cluster-api.