[Deprecated] Kubernetes Exec Plugin
License: Apache License 2.0
Calico networking for Kubernetes can now be found in the Calico CNI repository.
We recommend migrating to use the Calico CNI plugin for Kubernetes deployments. Documentation and guides for using the CNI plugin with Kubernetes can be found in the the Calico docs site.
Hi,
I have a similar issue to #44. Running calico-kubernetes with kubernetes-1.1.0-alpha1, tried calico-node 0.7.0 and 0.6.0, docker version 1.8.0. Manual calico-docker setups work, but running kubelet with calico network plugin does not setup any profiles (networking works if profile is created manually and is attached to the endpoint).
# calicoctl endpoint show --detailed
+---------------------------+-----------------+------------------------------------------------------------------+----------------------------------+-----------------+-------------------+----------+--------+
| Hostname | Orchestrator ID | Workload ID | Endpoint ID | Addresses | MAC | Profiles | State |
+---------------------------+-----------------+------------------------------------------------------------------+----------------------------------+-----------------+-------------------+----------+--------+
| kubernetes-minion-simas-1 | docker | 4f0d7178638c23359c231d51141e46f35cf0916081ddc309553b2dcc5fbdd44e | c84cd2486c2711e584790e23dbe0df45 | 192.168.1.3/32 | 02:33:72:8e:04:e1 | | active |
| kubernetes-minion-simas-1 | docker | f965cc1a77642cc0738fe83c2a52fe6f5c2e001edec2a29a63008068c621c1fd | c7af6d6e6c2711e59f020e23dbe0df45 | 192.168.1.2/32 | 3e:39:5c:3a:81:60 | | active |
| kubernetes-minion-simas-2 | docker | 143c5e13c39299abfe8643369a7ec864855623074f390cde202ab16a32dd165f | c9ee62066c2711e5971c0e46fb8fddff | 192.168.2.2/32 | 12:ee:9e:cd:d7:7c | | active |
| kubernetes-minion-simas-2 | docker | 7ab5e1ec14c365e3fb0b539218118677387d3ad17bacecfd3d36462dfe2cc7db | c9ba880a6c2711e59f0a0e46fb8fddff | 192.168.2.3/32 | 62:4a:0e:e2:33:2c | | active |
| kubernetes-minion-simas-2 | docker | fe049aea7d841614a62ebec3eb6287d9cdba8c2ce4d5d0208e2b44fc89877520 | 0cf527846c2c11e5b79d0e46fb8fddff | 192.168.2.99/32 | e6:4b:6f:2c:31:0e | | active |
+---------------------------+-----------------+------------------------------------------------------------------+----------------------------------+-----------------+-------------------+----------+--------+
# calicoctl profile show --detailed
+------+------+-----------------+-------------+-------------+-------+
| Name | Host | Orchestrator ID | Workload ID | Endpoint ID | State |
+------+------+-----------------+-------------+-------------+-------+
+------+------+-----------------+-------------+-------------+-------+
kubelet.log can be found at https://gist.github.com/geriBatai/1ac724b4da266d4d513d
when I create a service in k8s,it need a IP by creating docker0
Name: shen
Namespace: default
Image(s): jayway:nginx
Node: 10.10.0.7/10.10.0.7
Labels: <none>
Status: Running
Reason:
Message:
IP: 172.17.0.15 ###that's it ####
Replication Controllers: <none>
Containers:
nginx:
Image: jayway:nginx
State: Running
Started: Tue, 01 Dec 2015 03:11:18 -0500
Ready: True
Restart Count: 0
Conditions:
Type Status
Ready True
But , by creating pods with calico,in the container，it only has a IP just from calico .
under the scene，how to work with between the service of k8s and calico
As of v0.6.0, the plugin supports configuration via an on-disk .ini file.
We'd like this to be the mainline configuration option (until we get etcd config in place) so we should update our guides to show this.
We'd like to emit some (INFO and above) logs to the kubelet, while emitting all DEBUG and above logs to /var/log/calico/kubernetes/<pluginlog>.log'.
When looking at the kubelet logs, you'll get some context for why the plugin was being called.
When looking at the pluginlog logfile, you'll get detailed debugging information.
FYI. Why we have to run calico/node container on kubernetes master?
Moving configuration into etcd will simplify plugin deployment, and prevent misconfigurations where one instance of the plugin is configured separately from another instance of the plugin.
We can still use environment variables for override on configuration options where it makes sense.
Hello,
I was wondering what are the drawbacks of using static routes instead of calico.
I mean i've discussed with my collegue of the network team :
Does it make sense ?
What are the advantages/drawbacks of using calico ?
Regards,
Smana
I'm trying to get autoscaling working with calico as my network layer and its not working in AWS.
horizontal autoscaling needs heapster running in the kube-system namespace. Heapster needs to hit the kublet port (10255) on every k8 node.
The issue I'm having is when I run heapster in kubernetes I'm able to connect to the port just fine but the kubernetes controller needs to connect to heapster via the api proxy and that is running on the master and it times out because the master node cannot reach the calico /16 vlan.
If i add the following route rules on master/nodes
ip rule add from 192.168.0.0/16 table docker
ip route add 10.122.41.0/24 table docker dev tunl0
ip route add 10.122.42.0/24 table docker dev tunl0
ip route add 10.122.43.0/24 table docker dev tunl0
That is able to work but the heapster->kublet port breaks.
I'm in sort of a catch 22 because I can run heapster in docker on the master but connections aren't getting returned.. I see the ping but then its lost.
[root@kube-node_e5aa5956 ~] [dev] # tcpdump -v icmp -i any
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 65535 bytes
14:10:05.285965 IP (tos 0x0, ttl 63, id 24290, offset 0, flags [DF], proto ICMP (1), length 84)
ip-10-122-0-20.ec2.internal > ip-10-122-43-138.ec2.internal: ICMP echo request, id 7680, seq 14, length 64
14:10:06.286078 IP (tos 0x0, ttl 63, id 24291, offset 0, flags [DF], proto ICMP (1), length 84)
ip-10-122-0-20.ec2.internal > ip-10-122-43-138.ec2.internal: ICMP echo request, id 7680, seq 15, length 64
14:10:07.288024 IP (tos 0x0, ttl 63, id 24292, offset 0, flags [DF], proto ICMP (1), length 84)
ip-10-122-0-20.ec2.internal > ip-10-122-43-138.ec2.internal: ICMP echo request, id 7680, seq 16, length 64
14:10:08.288130 IP (tos 0x0, ttl 63, id 24293, offset 0, flags [DF], proto ICMP (1), length 84)
ip-10-122-0-20.ec2.internal > ip-10-122-43-138.ec2.internal: ICMP echo request, id 7680, seq 17, length 64
14:10:09.291316 IP (tos 0x0, ttl 63, id 24294, offset 0, flags [DF], proto ICMP (1), length 84)
ip-10-122-0-20.ec2.internal > ip-10-122-43-138.ec2.internal: ICMP echo request, id 7680, seq 18, length 64
14:10:10.292318 IP (tos 0x0, ttl 63, id 24295, offset 0, flags [DF], proto ICMP (1), length 84)
ip-10-122-0-20.ec2.internal > ip-10-122-43-138.ec2.internal: ICMP echo request, id 7680, seq 19, length 64
^C
I never see the return.
10.122.0.20 is my master
My status is as follows
[root@kube-master ~] [dev] # calicoctl status
calico-node container is running. Status: Up 21 hours
Running felix version 1.3.0rc5
IPv4 BGP status
+---------------+-------------------+-------+------------+-------------+
| Peer address | Peer type | State | Since | Info |
+---------------+-------------------+-------+------------+-------------+
| 10.122.42.163 | node-to-node mesh | up | 2015-12-23 | Established |
| 10.122.43.138 | node-to-node mesh | up | 2015-12-23 | Established |
+---------------+-------------------+-------+------------+-------------+
IPv6 BGP status
+--------------+-----------+-------+-------+------+
| Peer address | Peer type | State | Since | Info |
+--------------+-----------+-------+-------+------+
+--------------+-----------+-------+-------+------+
Those are my two k8 nodes.. I'm able to ping from container->container just fine if the container is running on a k8 node. If a container is running on the master it doesn't work
EDIT
route table for a node
[root@kube-node_123d65a4 ~] [dev] # ip route
default via 10.122.42.1 dev eth0
10.122.42.0/24 dev eth0 proto kernel scope link src 10.122.42.163
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.42.1
192.168.0.0 dev cali9cc2065aa99 scope link
blackhole 192.168.0.0/26 proto bird
192.168.0.1 dev cali9cf4e624a99 scope link
192.168.0.2 dev cali0f5e619aa99 scope link
192.168.0.3 dev cali47ded9a4a9b scope link
192.168.0.64/26 via 10.122.43.138 dev tunl0 proto bird onlink
192.168.0.128/26 via 10.122.0.20 dev tunl0 proto bird onlink
Route table for master
[root@kube-master ~] [dev] # ip route
default via 10.122.0.1 dev eth0
10.122.0.0/24 dev eth0 proto kernel scope link src 10.122.0.20
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.42.1
192.168.0.0/26 via 10.122.42.163 dev tunl0 proto bird onlink
192.168.0.64/26 via 10.122.43.138 dev tunl0 proto bird onlink
blackhole 192.168.0.128/26 proto bird
The plugin makes an attempt to clean up IP addresses and endpoints if it encounters an error, but it doesn't handle all cases. The plugin should be more robust to cases like:
Any time the plugin returns a non-zero exit code during a create, it needs to delete the endpoint and un-assign any IP addresses.
User babusatasiya on Calicousers Slack configured his calico_kubernetes.ini like this
[config]
ETCD_AUTHORITY=http://10.0.0.101:2379
KUBE_API_ROOT=http://private.domain.com:8080/api/v1/
DEFAULT_POLICY=allow
CALICO_IPAM=true
KUBE_AUTH_TOKEN=
And he got this error:
ValueError: invalid literal for int() with base 10: '//10.0.0.101:2379'
2015-12-17 04:55:54,234 2654 [9d831b8476b2] ERROR Unhandled Exception killed plugin
Traceback (most recent call last):
File "<string>", line 962, in run_protected
File "<string>", line 990, in run
File "<string>", line 99, in __init__
File "/code/build/calico/out00-PYZ.pyz/pycalico.datastore", line 176, in __init__
ValueError: invalid literal for int() with base 10: '//10.0.0.101:2379'
2015-12-17 04:56:28,564 2842 [9d831b8476b2] INFO Executing Calico pod-deletion hook
2015-12-17 04:56:28,564 2842 [9d831b8476b2] ERROR Unhandled Exception killed plugin
Traceback (most recent call last):
File "<string>", line 962, in run_protected
File "<string>", line 987, in run
File "<string>", line 99, in __init__
File "/code/build/calico/out00-PYZ.pyz/pycalico.datastore", line 176, in __init__
ValueError: invalid literal for int() with base 10: '//10.0.0.101:2379'
The problem is that there is a http:// in the ETCD_AUTHORITY. We need to either just deal with this case, or drop a better error message.
From #33
Furthermore, i've just done the test of rebooting my host and all the network interfaces disapeared :(
Calico still seems to be configured as previouslycalicoctl profile show
+------------------------------------+
| Name |
+------------------------------------+
| busybox |
| elasticsearch_logging_v1_1jzvs |
| elasticsearch_logging_v1_az6c8 |
| fluentd_elasticsearch_10.115.77.87 |
| fluentd_elasticsearch_10.115.77.88 |
| fluentd_elasticsearch_10.115.77.89 |
| kibana_logging_v1_ifztn |
| kube_dns_v8_l6rtn |
| kube_dns_v8_tq8xf |
| kube_ui_v1_ym5ni |
| monitoring_heapster_v8_9cy06 |
| monitoring_heapster_v8_pt4ey |
| monitoring_influx_grafana_v1_obu8h |
+------------------------------------+
But there'is no calixxxx interface on the host anymore.
The pods on this host are unreachableThe docker daemon is still configured on cbr0
/usr/bin/docker -d -H fd:// --bridge=cbr0 --iptables=false --ip-masq=false
Maybe i forgot something ?
Hi all,
Great stuff happing here. Unfortunately I can't get the combination of calico-kubernetes and kubernetes-1.1.0-alpha1 to work on AWS
Yes, AWS, so I run my configuration with this address pool:
etcdctl get /calico/v1/ipam/v4/pool/192.168.0.0-16
{"masquerade": true, "cidr": "192.168.0.0/16", "ipip": "tunl0"}
I started out with small experiment using plain docker containers on a AWS cluster: got that to work! So then I tried to get the following setup to work:
I launched a replica controller with 2 replicas. Pods get successfully deployed to both nodes and the ip-addresses assigned are in the range associated with the docker bridge on both nodes.
However I can't ping one container from the other. So something is going in the communication.
What does my setup look like?
kubernetes master
- name: calico-node.service
runtime: true
command: start
content: |
[Unit]
Description=Start Calico on this node
Requires=docker.service
Requires=network-online.target
After=docker.service
After=network-online.target
[Service]
User=root
PermissionsStartOnly=true
ExecStartPre=-/usr/bin/wget -N -P "/home/core" "https://github.com/projectcalico/calico-docker/releases/download/v0.6.0/calicoctl"
ExecStartPre=-/usr/bin/chmod +x /home/core/calicoctl
ExecStart=/home/core/calico-node.sh $private_ipv4 127.0.0.1:2379
RemainAfterExit=yes
Type=oneshot
...
write_files:
- path: /home/core/calico-node.sh
owner: root
permissions: 0755
content: |
#! /usr/bin/bash
# Usage: calico-node.sh <node ip> <etcd authority>
# Install necessary kernel modules
sudo modprobe xt_set
sudo modprobe ip6_tables
sudo /home/core/calicoctl checksystem --fix
sudo ETCD_AUTHORITY=$2 /home/core/calicoctl node --ip=$1
nodes/minions:
- name: cbr0.netdev
command: start
content: |
[NetDev]
Name=cbr0
Kind=bridge
- name: cbr0.network
command: start
content: |
[Match]
Name=cbr0
[Network]
Address=192.168.1.1
- name: docker.service
drop-ins:
- name: 50-calico-kubernetes.conf
content: |
[Service]
Environment=DOCKER_OPTS='--bridge=cbr0 --iptables=false --ip-masq=false'
- name: calico-node.service
runtime: true
command: start
content: |
[Unit]
Description=Start Calico on this node
Requires=docker.service
Requires=network-online.target
After=docker.service
After=network-online.target
[Service]
User=root
PermissionsStartOnly=true
ExecStartPre=-/usr/bin/wget -N -P "/home/core" "https://github.com/projectcalico/calico-docker/releases/download/v0.6.0/calicoctl"
ExecStartPre=-/usr/bin/chmod +x /home/core/calicoctl
ExecStart=/home/core/calico-node.sh $private_ipv4 127.0.0.1:2379
RemainAfterExit=yes
Type=oneshot
- name: kubelet.service
enable: true
content: |
[Unit]
After=download-kubernetes.service
ConditionFileIsExecutable=/opt/kubernetes/server/bin/kubelet
Description=Kubernetes Kubelet
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
Wants=download-kubernetes.service
[Service]
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests/
ExecStart=/opt/kubernetes/server/bin/kubelet \
--address=0.0.0.0 \
--port=10250 \
--hostname_override=%H \
--api_servers=http://<k8-master-ip-address>:8080 \
--logtostderr=true \
--config=/etc/kubernetes/manifests/ \
--network_plugin=calico \
--network_plugin_dir=/etc/kubelet-plugins
Restart=always
RestartSec=10
[Install]
WantedBy=kubernetes-minion.target
- name: proxy.service
enable: true
content: |
[Unit]
After=download-kubernetes.service
ConditionFileIsExecutable=/opt/kubernetes/server/bin/kube-proxy
Description=Kubernetes Proxy
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
Wants=download-kubernetes.service
[Service]
ExecStart=/opt/kubernetes/server/bin/kube-proxy \
--master=http://10.150.16.32:8080 \
--legacy-userspace-proxy=false \
--logtostderr=true
Restart=always
RestartSec=10
[Install]
WantedBy=kubernetes-minion.target
...
write_files:
- path: /home/core/calico-node.sh
owner: root
permissions: 0755
content: |
#! /usr/bin/bash
# Usage: calico-node.sh <node ip> <etcd authority>
# Install necessary kernel modules
sudo modprobe xt_set
sudo modprobe ip6_tables
sudo /home/core/calicoctl checksystem --fix
sudo ETCD_AUTHORITY=$2 /home/core/calicoctl node --ip=$1 --kubernetes
tunnel0 devices are created on the host machines. If I try to ping one container from the other it fails.
Looking at the data in etcd it appears as if there are no security policies created at all, which is not what I would expect.
Just to make sure that the calico network does work basically, I created a simple setup with 2 containers on both nodes, launched without kubernetes but by hand. Exactly like the Docker with Calico networking tutorial: https://github.com/projectcalico/calico-docker/blob/master/docs/getting-started/default-networking/Demonstration.md
These 2 plain containers can ping each other.
It wouldn't surprise if I made a configuration error but it is hard to debug which :-)
The plugin ignores host networked containers (e.g. kube-proxy). However, due to a bug in our code, it doesn't check if the container is host-networked until AFTER it attempts to do some stuff.
It should check the NetworkMode first thing, and exit successfully without doing anything if it encounters a NetworkMode=host container.
Hello,
I have a private cluster so i use the "NodePort" service type.
With flannel i would have used an external load balancer to spread the load over my nodes but maybe there'a better way with calico ? (ecmp ...)
Regards,
Smana
I've been playing with kubernetes in AWS and when I create a new service with a nodeport this version of kube-proxy
https://github.com/projectcalico/calico-kubernetes/releases/download/v0.1.1/kube-proxy
It never opens a port an sets the iptables rules on the host to send traffic to the proxy.
All I see in the logs is
I0925 15:54:22.329156 31557 endpoints_tracker.go:115] EndpointsTracker: Setting endpoints for default/nginx: to [192.168.1.7:80 192.168.1.8:80 192.168.2.3:80 192.168.2.4:80]
In the proxy that comes with 1.0.3 I see something like
I0925 17:24:57.241261 14656 iptables.go:303] running iptables -A [KUBE-PORTALS-HOST -t nat -m comment --comment kube-system/kube-dns:dns -p udp -m udp --dport 53 -d 10.254.0.10/32 -j DNAT --to-destination 10.122.2.6:58129]
I0925 17:24:57.242735 14656 proxier.go:466] Opened iptables from-host portal for service "kube-system/kube-dns:dns" on UDP 10.254.0.10:53
I0925 17:24:57.242770 14656 proxier.go:326] Adding new service "kube-system/kube-dns:dns-tcp" at 10.254.0.10:53/TCP
I0925 17:24:57.242822 14656 proxier.go:268] Proxying for service "kube-system/kube-dns:dns-tcp" on TCP port 44191
From what I've been told this will be fixed with the 1.1 release but just making it known that I haven't found a way to use the --legacy-userspace-proxy=false to enable security in the proxy with 1.0.3-1.0.6
Hello,
I think my kubelet service is properly configured now.
When i run a pod, the container gets its address from the pool
ip a show cbr0
5: cbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
inet 10.233.1.1/24 scope global cbr0
valid_lft forever preferred_lft forever
inet6 fe80::507b:20ff:fe97:c0cf/64 scope link
valid_lft forever preferred_lft forever
ps aux | grep /usr/bin/docker
/usr/bin/docker -d -H fd:// --bridge=cbr0 --iptables=false --ip-masq=false
docker exec k8s_worker.41ff1f2e_redis-slave-nruvk_default_08421053-4d68-11e5-af10-525400c14d0a_77d68d0f ip a show eth0
18: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:1b:00:d7:e0:27 brd ff:ff:ff:ff:ff:ff
inet 10.233.1.5/32 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::501b:ff:fed7:e027/64 scope link
valid_lft forever preferred_lft forever
However if i run a container with docker i can't ping the address of the container spawned by kubernetes.
docker run --rm -ti base/archlinux /bin/bash
[root@87d9b164ffe9 /]# ip a show eth0
24: eth0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
link/ether 02:42:0a:e9:01:08 brd ff:ff:ff:ff:ff:ff
inet 10.233.1.8/24 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::42:aff:fee9:108/64 scope link
valid_lft forever preferred_lft forever
[root@87d9b164ffe9 /]# ping 10.233.1.5
PING 10.233.1.5 (10.233.1.5) 56(84) bytes of data.
From 10.233.1.8 icmp_seq=1 Destination Host Unreachable
Note : This container (run with docker) can ping another container launched the same way on another node.
The difference i can see comes from how the interfaces are created on the host, the kubernetes interfaces are not attached to the bridge cbr0.
21: cali09089e184d6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether c6:28:14:7c:36:cd brd ff:ff:ff:ff:ff:ff
inet 10.115.77.87/32 scope global cali09089e184d6
valid_lft forever preferred_lft forever
inet6 fe80::c428:14ff:fe7c:36cd/64 scope link
valid_lft forever preferred_lft forever
25: veth4582dc5: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master cbr0 state UP group default
link/ether 16:ca:4c:7e:e0:f0 brd ff:ff:ff:ff:ff:ff
inet6 fe80::14ca:4cff:fe7e:e0f0/64 scope link
valid_lft forever preferred_lft forever
When i look to the kubelet logs i get the following error log :
Aug 28 09:34:54 kubenode-1 kubelet[29139]: I0828 09:34:54.047335 29139 exec.go:130] SetUpPod 'exec' network plugin output: Warning: '-format' is deprecated, it will be replaced by '--format' soon. See usage.
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Warning: '-format' is deprecated, it will be replaced by '--format' soon. See usage.
Aug 28 09:34:54 kubenode-1 kubelet[29139]: /tmp/_MEIc3W7lo/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
Aug 28 09:34:54 kubenode-1 kubelet[29139]: /tmp/_MEIc3W7lo/requests/packages/urllib3/connectionpool.py:768: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Using ETCD_AUTHORITY=10.115.77.86:4001
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Using CALICOCTL_PATH=/usr/local/bin/calicoctl
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Using KUBE_API_ROOT=https://10.115.77.86:443/api/v1/
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Args: ['/usr/libexec/kubernetes/kubelet-plugins/net/exec/calico/calico', 'setup', 'default', 'redis-slave-nruvk', 'cc864fdf31cb3c14f44a14e13c144c41c7a2f06c053e4ab5942320cbc7a008e2']
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Executing Calico pod-creation hook
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Configuring docker container cc864fdf31cb3c14f44a14e13c144c41c7a2f06c053e4ab5942320cbc7a008e2
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Docker-assigned IP was 10.233.1.5
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Deleting eth0
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Configuring Calico networking.
Aug 28 09:34:54 kubenode-1 kubelet[29139]: IP 10.233.1.5 added to cc864fdf31cb3c14f44a14e13c144c41c7a2f06c053e4ab5942320cbc7a008e2
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Adding IP 10.115.77.87 to interface cali09001bb24d6
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Finished configuring network interface
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Failed to open auth_file ([Errno 2] No such file or directory: '/var/lib/kubelet/kubernetes_auth'), assuming insecure mode
Aug 28 09:34:54 kubenode-1 kubelet[29139]: Traceback (most recent call last):
Aug 28 09:34:54 kubenode-1 kubelet[29139]: File "<string>", line 341, in <module>
Aug 28 09:34:54 kubenode-1 kubelet[29139]: File "<string>", line 46, in create
Aug 28 09:34:54 kubenode-1 kubelet[29139]: File "<string>", line 158, in _configure_profile
Aug 28 09:34:54 kubenode-1 kubelet[29139]: File "<string>", line 189, in _get_pod_config
Aug 28 09:34:54 kubenode-1 kubelet[29139]: File "<string>", line 220, in _get_api_path
Aug 28 09:34:54 kubenode-1 kubelet[29139]: File "/code/build/calico_kubernetes/out00-PYZ.pyz/json", line 338, in loads
Aug 28 09:34:54 kubenode-1 kubelet[29139]: File "/code/build/calico_kubernetes/out00-PYZ.pyz/json.decoder", line 366, in decode
Aug 28 09:34:54 kubenode-1 kubelet[29139]: File "/code/build/calico_kubernetes/out00-PYZ.pyz/json.decoder", line 384, in raw_decode
Aug 28 09:34:54 kubenode-1 kubelet[29139]: ValueError: No JSON object could be decoded
Aug 28 09:34:54 kubenode-1 kubelet[29139]: , exit status 255
Thanks for your help :)
Smana
Kubelet runs pod updates concurrently, so we see logs from multiple plugin invocations interleaved. It's impossible to tell which log comes from which invocation.
We should prefix each log with a correlator, probably the 12-char concatenated docker ID.
We should be able to bake this into the log command and formatter so it doesn't need to be explicitly passed to every log statement.
We currently create a single policy profile per-pod. This will likely pose scale-issues as Kubernetes deployments begin to scale up.
One proposed solution is to generate profiles based on the hash of the policy statement and tags, limiting the number of duplicate profiles.
Overall, this needs some more thought. Some (but definitely not all) things to consider:
I see the plugin crashing (due to a configuration error) but it is not emitted in the logfile.
The log is being repeated 10 times per second...
Currently, Calico policy will not be updated if the policy annotation or labels are updated on a given pod. Essentially, Calico-policy in Kubernetes is immutable.
Since Kubernetes network plugins are only informed of creation / deletion events, not updates, we either need to augment the network plugin API to give us this information, or watch the API server for pod updates.
The plugin generates log output to /var/log/calico/kubernetes/calico.log but this log doesn't have any log wrapping which can lead to giant logfiles.
We should follow the example of the other Calico component logs, and put plugin logs in their own directory.
For example,
/var/log/calico/kubernetes/exec-plugin/current.log
As seen here: http://stackoverflow.com/questions/31265993/docker-networking-namespace-not-visible-in-ip-netns-list
The PID no longer exists after we delete the Docker created veth, and we attempt to use ip netns exec, which fails since it cannot find the symlink in /var/run/netns.
At a minimum, 100% line coverage (since it's Python). Ensuring our test cases are rigorous is a separate issue.
The current KUBE_API_ROOT variable allows the user to specify the exact API version for the plugin to use. e.g. `http://api-url:8080/api/vX"
We don't actually want to allow the user to configure the API version, since the plugin is written to support a specific version (currently v1).
The information that the user does need to provide:
Image gcr.io/google_containers/hyperkube is now at v1.1.2
Another follow-up question: I don't understand why I can't connect to a remote container using the following setup:
|'ve got a calico-kubernetes setup working using calicoctl 0.7.0 & kubernetes 1.1.0-alpha1!
I came to the conclusion that the setup is working because a created a replica controller with 2 replica's that both launched on different host machines and I can ping the remote container from the other container. Great succes! :-)
the kubelet configuration data:
ETCD_AUTHORITY=127.0.0.1:2379
CALICOCTL_PATH=/home/core/calicoctl
KUBE_API_ROOT=http://<k8-master>:8080/api/v1/
CALICO_IPAM=true
So I also use Calico IPAM, couldn't get this working on AWS with Docker IPAM.
AWS is relevant here: I don't have access to a full blown L2/L3 network so I configurered Calico to use an IPIP tunnel:
sudo ETCD_AUTHORITY=127.0.0.1:2379 ./calicoctl pool add 192.168.0.0/16 --ipip --nat-outgoing
calicoctl endpoint show --detailed
+--------------------------------------------+-----------------+------------------------------------------------------------------+----------------------------------+----------------+-------------------+---------------------------------------------+--------+
| Hostname | Orchestrator ID | Workload ID | Endpoint ID | Addresses | MAC | Profiles | State |
+--------------------------------------------+-----------------+------------------------------------------------------------------+----------------------------------+----------------+-------------------+---------------------------------------------+--------+
| ip-10-150-32-32.eu-west-1.compute.internal | docker | 886094dd3e98126e52a5e2edac611aaec31eac08418b23f99723a6ebdaa4439b | 24b0137065e211e5a3ed06b84976d5a7 | 192.168.1.0/32 | fa:db:c6:ed:23:13 | default_todo-release-153-5w1wv_886094dd3e98 | active |
| ip-10-150-48-32.eu-west-1.compute.internal | docker | ca99437e7a49d22aa78735e2234676162672df11aad3318d37f26e6a9be3126b | 24a7a72665e211e591930abd28c5cad9 | 192.168.0.0/32 | 32:10:6f:99:74:d1 | default_todo-release-153-3qha2_ca99437e7a49 | active |
+--------------------------------------------+-----------------+------------------------------------------------------------------+----------------------------------+----------------+-------------------+---------------------------------------------+--------+
calicoctl profile show --detailed
+---------------------------------------------+--------------------------------------------+-----------------+------------------------------------------------------------------+----------------------------------+--------+
| Name | Host | Orchestrator ID | Workload ID | Endpoint ID | State |
+---------------------------------------------+--------------------------------------------+-----------------+------------------------------------------------------------------+----------------------------------+--------+
| default_todo-release-153-3qha2_ca99437e7a49 | ip-10-150-48-32.eu-west-1.compute.internal | docker | ca99437e7a49d22aa78735e2234676162672df11aad3318d37f26e6a9be3126b | 24a7a72665e211e591930abd28c5cad9 | active |
| default_todo-release-153-5w1wv_886094dd3e98 | ip-10-150-32-32.eu-west-1.compute.internal | docker | 886094dd3e98126e52a5e2edac611aaec31eac08418b23f99723a6ebdaa4439b | 24b0137065e211e5a3ed06b84976d5a7 | active |
+---------------------------------------------+--------------------------------------------+-----------------+------------------------------------------------------------------+----------------------------------+--------+
calicoctl profile default_todo-release-153-3qha2_ca99437e7a49 rule show
Inbound rules:
1 allow
Outbound rules:
1 allow
calicoctl profile default_todo-release-153-5w1wv_886094dd3e98 rule show
Inbound rules:
1 allow
Outbound rules:
1 allow
so who can connect to who?
Looking at the routing configuration on host1 I don't understand why I can't reach the remote container on host2:
default via 10.150.32.1 dev eth0 proto dhcp src 10.150.32.32 metric 1024
10.150.32.0/20 dev eth0 proto kernel scope link src 10.150.32.32
10.150.32.1 dev eth0 proto dhcp scope link src 10.150.32.32 metric 1024
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.42.1
192.168.0.0 via 10.150.48.32 dev tunl0 proto bird onlink
192.168.1.0 dev cali24b0137065e scope link
We currently allow configuration of which calicoctl to use via the CALICOCTL_PATH environment variable.
This is unnecessary - we should be able to do this by using whichever calicoctl binary exists in the system's PATH.
This will improve plugin portability between OSes (where the path to calicoctl might be different) and remove an extra bit of configuration, which is nice too.
Currently, semantic rules can only be added by calicoctl one at a time. Ideally we want to support profile updating as opposed to appending with this format. We also do not want this plugin to call calicoctl at all, but some work needs to be done to allow the pycalico datastore to accept our semantic format.
A symptom of this is that we need to flush out the default rules before adding in new policy. This is currently done naively with calicoctl calls that target rules that are assumed to exist. If these defaults change, new policy will be disrupted. Default rule replacement should be more dynamic (and preferably without calicoctl calls)
Hello,
Following this issue : #34
I'm trying to upgrade my cluster.
As suggested by Casey i've configured my node as follows :
calicoctl pool show
+----------------+---------+
| IPv4 CIDR | Options |
+----------------+---------+
| 10.233.64.0/18 | |
+----------------+---------+
ip a sho dev cbr0
4: cbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
inet 172.16.42.2/24 scope global cbr0
valid_lft forever preferred_lft forever
inet6 fe80::8464:dbff:fec6:ac32/64 scope link
valid_lft forever preferred_lft forever
ps aux | grep docker
root 8056 2.6 1.8 1348252 38420 ? Ssl 11:14 1:05 /usr/bin/docker daemon -H fd:// --bridge=cbr0 --iptables=false --ip-masq=false
Nov 12 12:01:58 kbnode1 kubelet[17740]: I1112 12:01:58.715419 17740 exec.go:149] Status 'exec' network plugin output: 2015-11-12 12:01:58,687 18467 INFO Begin Calico network plugin execution
Nov 12 12:01:58 kbnode1 kubelet[17740]: 2015-11-12 12:01:58,687 18467 INFO Plugin Args: ['/usr/libexec/kubernetes/kubelet-plugins/net/exec/calico/calico', 'status', 'kube-system', 'kubedns-louck', '4babfcc54b8f17787007a7cce3f00b9e764824b802b8f33e91e232c0
Nov 12 12:01:58 kbnode1 kubelet[17740]: 2015-11-12 12:01:58,688 18467 INFO Executing Calico pod-status hook
Nov 12 12:01:58 kbnode1 kubelet[17740]: 2015-11-12 12:01:58,694 18467 INFO Retrieved IP Address: 10.233.65.0
Nov 12 12:01:58 kbnode1 kubelet[17740]: {"ip": "10.233.65.0", "kind": "PodNetworkStatus", "apiVersion": "v1beta1"}
Nov 12 12:01:58 kbnode1 kubelet[17740]: 2015-11-12 12:01:58,694 18467 INFO Calico network plugin execution complete
Nov 12 12:01:58 kbnode1 kubelet[17740]: , <nil>
Nov 12 12:01:58 kbnode1 kubelet[17740]: E1112 12:01:58.715512 17740 manager.go:377] NetworkPlugin calico failed on the status hook for pod 'kubedns-louck' - invalid character '-' after top-level value
calicoctl endpoint show --d | grep kubedns-louc
| kbnode1 | docker | 4babfcc54b8f17787007a7cce3f00b9e764824b802b8f33e91e232c0d8a75322 | 17ed20da893511e5aafe5254008dc07e | 10.233.65.0/32 | 46:12:50:0d:be:ec | kube-system_kubedns-louck_4babfcc54b8f | active |
kubectl describe po kubedns-louck --namespace=kube-system | grep ^IP
IP: 172.16.42.6
6m 25s 9 {kubelet 10.115.99.11} spec.containers{skydns} Unhealthy Liveness probe failed: Get http://172.16.42.6:8080/healthz: dial tcp 172.16.42.6:8080: no route to host
Is there some configuration parameter i've missed ?
kubelet is running with the following command
/usr/local/bin/kubelet --logtostderr=true --v=5 --api_servers=https://10.115.99.10:443 --address=0.0.0.0 --hostname_override=10.115.99.11 --allow_privileged=true --cluster_dns=10.233.0.10 --cluster_domain=cluster.local --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --config=/etc/kubernetes/manifests --network_plugin=calico
kubectl version
Client Version: version.Info{Major:"1", Minor:"1", GitVersion:"v1.1.1", GitCommit:"92635e23dfafb2ddc828c8ac6c03c7a7205a84d8", GitTreeState:"clean"}
Server Version: version.Info{Major:"1", Minor:"1", GitVersion:"v1.1.1", GitCommit:"92635e23dfafb2ddc828c8ac6c03c7a7205a84d8", GitTreeState:"clean"}
calicoctl version
0.10.0
Thank you
Hello,
I'm currently testing calico with kubernetes but i'm facing an issue.
I followed the documentation : https://github.com/kubernetes/kubernetes/blob/master/docs/getting-started-guides/ubuntu-calico.md
And calico seems to be happy :
calicoctl status
calico-node container is running. Status: Up 36 minutes
IPv4 BGP status
+--------------+-------------------+-------+----------+-------------+
| Peer address | Peer type | State | Since | Info |
+--------------+-------------------+-------+----------+-------------+
| 10.115.77.88 | node-to-node mesh | up | 12:57:29 | Established |
| 10.115.77.89 | node-to-node mesh | up | 12:58:48 | Established |
| 10.115.77.87 | node-to-node mesh | up | 12:58:40 | Established |
+--------------+-------------------+-------+----------+-------------+
IPv6 BGP status
+--------------+-----------+-------+-------+------+
| Peer address | Peer type | State | Since | Info |
+--------------+-----------+-------+-------+------+
+--------------+-----------+-------+-------+------+
The containers can talk each others from node to node.
Unfortunately when i start kubelet on nodes i get the following message :
/usr/local/bin/kubelet --logtostderr=true --v=0 --api_servers=https://10.115.77.86:443 --address=0.0.0.0 --hostname_override=10.115.77.87 --allow_privileged=true --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --network_plugin=calico
I0827 13:37:12.027317 935 manager.go:127] cAdvisor running in container: "/"
I0827 13:37:12.028076 935 fs.go:93] Filesystem partitions: map[/dev/vda2:{mountpoint:/ major:254 minor:2}]
I0827 13:37:12.031702 935 manager.go:156] Machine: {NumCores:2 CpuFrequency:2260998 MemoryCapacity:3167059968 MachineID:c39561d3284963021ae3cb2b995631a5 SystemUUID:7869686E-14AA-25EA-A478-2454DC0FA17A BootID:af96ffa8-08d3-45cd-b799-c1e7f8441f18 Filesystems:[{Device:/dev/vda2 Capacity:31571550208}] DiskMap:map[254:0:{Name:vda Major:254 Minor:0 Size:32212254720 Scheduler:none}] NetworkDevices:[{Name:cbr0 MacAddress:4e:1f:03:44:18:a4 Speed:0 Mtu:1500} {Name:eth0 MacAddress:52:54:00:d7:32:b7 Speed:0 Mtu:1500} {Name:tunl0 MacAddress:00:00:00:00 Speed:0 Mtu:1480}] Topology:[{Id:0 Memory:3167059968 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]} {Id:1 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:4194304 Type:Unified Level:2}]}] Caches:[]}]}
I0827 13:37:12.033000 935 manager.go:163] Version: {KernelVersion:3.16.0-4-amd64 ContainerOsVersion:Debian GNU/Linux 8 (jessie) DockerVersion:1.6.2 CadvisorVersion:0.15.1}
I0827 13:37:12.033851 935 plugins.go:69] No cloud provider specified.
I0827 13:37:13.065601 935 docker.go:295] Connecting to docker on unix:///var/run/docker.sock
I0827 13:37:13.066002 935 server.go:673] Watching apiserver
failed to create kubelet: Network plugin "calico" failed init: exit status 255
Could you please help me to find out how to get it work ?
Some information :
docker version : 1.6.2 (i know it is not the latest, tell me if that could be the cause)
calicoctl version : 0.5.1
kubernetes version : 1.0.3
OS : debian jessie
Thanks,
Smana
Hello,
I'm doing some bgp tests:
The idea is to have nodes on different datacenters and get the containers talking each others no matter where they're located.
I configured the first node as follows :
calicoctl bgp default-node-as 6xx80
calicoctl bgp peer add 10.17.252.2 as 6xx06
calicoctl bgp peer add 10.17.252.3 as 6xx06
calicoctl status
calico-node container is running. Status: Up 5 minutes
IPv4 BGP status
+--------------+-------------------+-------+----------+-------------+
| Peer address | Peer type | State | Since | Info |
+--------------+-------------------+-------+----------+-------------+
| 10.17.252.11 | node-to-node mesh | up | 08:16:59 | Established |
| 10.17.252.27 | node-to-node mesh | up | 08:16:59 | Established |
| 10.17.252.2 | global | up | 08:16:59 | Established |
| 10.17.252.3 | global | up | 08:17:38 | Established |
+--------------+-------------------+-------+----------+-------------+
So far it is working as expected but as soon as i configure the second node i have an inconsistent state :
calicoctl bgp default-node-as 6xx81
calicoctl bgp peer add 10.17.252.18 as 6xx07
calicoctl bgp peer add 10.17.252.19 as 6xx07
calicoctl status
calico-node container is running. Status: Up 29 minutes
IPv4 BGP status
+--------------+-------------------+-------+----------+------------------------------------------------+
| Peer address | Peer type | State | Since | Info |
+--------------+-------------------+-------+----------+------------------------------------------------+
| 10.17.252.11 | node-to-node mesh | up | 08:41:12 | Established |
| 10.17.252.12 | node-to-node mesh | up | 08:41:12 | Established |
| 10.17.252.2 | global | start | 08:41:07 | Active Socket: Connection reset by peer |
| 10.17.252.3 | global | start | 08:41:07 | Active Socket: Connection reset by peer |
| 10.17.252.18 | global | start | 08:41:16 | Idle Received: Bad peer AS |
| 10.17.252.19 | global | start | 08:41:19 | Idle Received: Bad peer AS |
+--------------+-------------------+-------+----------+------------------------------------------------+
I thought the default-node-as was specific to the node where i run the command but when i look the etcd data i noticed that it is replace and the value is share by all the nodes.
65381
Could you please help me to finalize this POC ?
Regards,
Smana
See #31
Currently, Calico policy is always enabled in the plugin (implementing namespace isolation by default). This behavior breaks services when using the old userspace kube-proxy.
This behavior should be configurable so that a user can choose to disable the default namespace isolation policy.
Labels, tags, and profile names are passed to Felix which use them as rule-names (for human readability). However, Kubernetes accepts the / character (as well as a few others), which Felix will error on. Therefore, we need to properly escape these characters.
We've temporarily worked around this issue by replacing incompatible characters with underscores. However, there is the potential for clashes between escaped strings, so more robust escaping is warranted.
Marking as low priority, since these clashes (though possible) are pretty unlikely in real-world scenarios.
With calico-kubernetes v0.6.0, I'm seeing a logging error:
Nov 14 00:30:34 kubernetes-minion-1 kubelet[22990]: 2015-11-14 00:30:33,099 28521 [00bf223be692] INFO Configuring Calico network interface
Nov 14 00:30:34 kubernetes-minion-1 kubelet[22990]: 2015-11-14 00:30:33,148 28521 [00bf223be692] INFO Using Calico IPAM
Nov 14 00:30:34 kubernetes-minion-1 kubelet[22990]: Traceback (most recent call last):
Nov 14 00:30:34 kubernetes-minion-1 kubelet[22990]: File "/code/build/calico_kubernetes/out00-PYZ.pyz/logging", line 850, in emit
Nov 14 00:30:34 kubernetes-minion-1 kubelet[22990]: File "/code/build/calico_kubernetes/out00-PYZ.pyz/logging", line 723, in format
Nov 14 00:30:34 kubernetes-minion-1 kubelet[22990]: File "/code/build/calico_kubernetes/out00-PYZ.pyz/logging", line 467, in format
Nov 14 00:30:34 kubernetes-minion-1 kubelet[22990]: KeyError: 'identity'
Nov 14 00:30:34 kubernetes-minion-1 kubelet[22990]: Logged from file pycalico.ipam, line 403
Not sure if it's benign yet. Can you do a code read to see if we're setting the Formatter on the wrong log handler Alex?
Dec 18 20:07:37 calico-03 kubelet[1353]: E1218 20:07:37.136491 1353 manager.go:377] NetworkPlugin calico failed on the status hook for pod 'pinger-5u1i8' - invalid character '-' after top-level value
Dec 18 20:07:37 calico-03 kubelet[1353]: W1218 20:07:37.794202 1353 manager.go:1875] Hairpin setup failed for pod "pinger-itw5b_default": open /sys/devices/virtual/net/calif72e8e06a5c/brport/hairpin_mode: no s
I was running a slightly customized plugin so no need to panic yet 😄
Logging is currently being streamed to stdout. This causes the Kubernetes status hook to fail since Kubernetes looks to stdout for status hook results. I.e.
Oct 20 00:30:43 kube-node1 kubelet[942]: E1020 00:30:43.193752 942 manager.go:343] NetworkPlugin calico failed on the status hook for pod 'no-access' - invalid character '-' after top-level value
Therefore, Kubernetes fails to learn about the IP address that Calico assigns to a pod.
It would be convenient from a debugging standpoint to have a way to determine which plugin binary is being used (either via logging, or a command like "calico_kubernetes version" which displays the version.
We'd have to be careful to always pin the version in release binaries.
The Kubernetes Plugin API supports plugin IPAM as of this PR #5069
Calico has some IPAM that would add value to Kubernetes. Let's join the two.
Annotations are typically namespaced in Kubernetes.
So,
annotations:
policy: "..."
Should become something like
annotations:
projectcalico.org/policy: "..."
Seen in #28
When the plugin is configured to use secure comms with the API, but no BearerToken exists at /var/lib/kubelet/kubernetes_auth, the error is unclear. The plugin should emit a clear log indicating the problem.
Hello,
I'm currently doing tests with 2 nodes located on 2 different datacenters.
The problem is that my nodes can't access to the Internet.
I tried to run calicoctl but it didn't work with the following command :
ETCD_AUTHORITY=10.17.252.11:4001 /usr/local/bin/calicoctl node --ip=10.17.252.27 --kubernetes
Override the host:port of the ETCD server by setting the environment variable
ETCD_AUTHORITY [default: 127.0.0.1:4001]
Usage: calicoctl <command> [<args>...]
status Print current status information
node Configure the main calico/node container and establish Calico networking
container Configure containers and their addresses
profile Configure endpoint profiles
endpoint Configure the endpoints assigned to existing containers
pool Configure ip-pools
bgp Configure global bgp
checksystem Check for incompatabilities on the host system
diags Save diagnostic information
See 'calicoctl <command> --help' to read about a specific subcommand.
I don't understand what's happening and there is no debug mode to figure it out.
Note:
ls -ltr /usr/libexec/kubernetes/kubelet-plugins/net/exec/calico/calico
-rwxr-xr-x 1 root root 0 2015-09-08 08:06 /usr/libexec/kubernetes/kubelet-plugins/net/exec/calico/calico
docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
calico/node v0.5.1 ef4aab54c2ce 7 weeks ago 261.5 MB
Could you please help me to understand ?
Regards,
Smana
Hello,
Sorry to bother you again but i'd like to have more information on how is configured a pod's networking.
I already have a kubernetes cluster working with calico and deployed with Ansible.
ip netns list
bridge name bridge id STP enabled interfaces
cbr0 8000.000000000000 no
5: cbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff
inet 10.233.1.1/24 scope global cbr0
valid_lft forever preferred_lft forever
inet6 fe80::10eb:6ff:fedf:9583/64 scope link
valid_lft forever preferred_lft forever
9: calic5d012a84ff: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 26:fc:40:73:c9:9c brd ff:ff:ff:ff:ff:ff
inet 10.115.77.87/32 scope global calic5d012a84ff
valid_lft forever preferred_lft forever
inet6 fe80::24fc:40ff:fe73:c99c/64 scope link
valid_lft forever preferred_lft forever
15: califb85ed004ff: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 7a:bd:cc:6d:d3:f3 brd ff:ff:ff:ff:ff:ff
inet 10.115.77.87/32 scope global califb85ed004ff
valid_lft forever preferred_lft forever
inet6 fe80::78bd:ccff:fe6d:d3f3/64 scope link
It is the gateway for the other nodes to reach the pods.
kubenode-2 ~ 0 # ip r
default via 10.115.255.253 dev eth0
10.0.0.0/8 via 10.115.255.254 dev eth0
10.115.0.0/16 dev eth0 proto kernel scope link src 10.115.77.88
10.233.0.0/24 via 10.115.77.86 dev eth0 proto bird
10.233.1.0/24 via 10.115.77.87 dev eth0 proto bird
10.233.1.2 via 10.115.77.87 dev eth0 proto bird
10.233.1.3 via 10.115.77.87 dev eth0 proto bird
10.233.1.4 via 10.115.77.87 dev eth0 proto bird
10.233.1.5 via 10.115.77.87 dev eth0 proto bird
So i guess that when a packet arrive it is passed to the right interface based on mac address ? right ?
* How are the packets forwarded to the pod ?
Thank you,
Smana
Seen when running on default GCE provider.
I0818 17:25:00.291314 21474 exec.go:106] Init 'exec' network plugin output: /usr/libexec/kubernetes/kubelet-plugins/net/exec/calico/calico: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.14' not found (required by /tmp/_MEIOk8dJX/libz.so.1)
, exit status 1
failed to create kubelet: Network plugin "calico" failed init: exit status 1
Now that we have sufficient field testing, we should be enabling CALICO_IPAM by default.
This is a minor change to the code and an update to https://github.com/projectcalico/calico-docker/blob/master/docs/kubernetes/KubernetesIntegration.md
The Kubelet might call multiple instances of the plugin at the same time. All logs are written to the same file, which makes correlating logs to plugin instances confusing. The logs should at least have the process ID so we can follow a single instance of the plugin.
Hello,
It seems I hit an issue with a pods defined as a manifests. The very first time (at least) a pod is started as a manifest, networking setup seems to stop before applying a profile. The container is not reachable from another host.
Some detail on my setup :
Calico-kubernetes plugin version : 0.2.0
calicoctl version : 0.7.0
Pool configured : 172.17.0.0/16 (ipip with workaround from calico-docker issue #426)
Node IP is 192.168.200.6
Master IP is 192.168.200.2
After a "fresh install" here is logs from the plugin : https://gist.github.com/slaws/995ae34856b6f8d8ddf0
Then a reboot : https://gist.github.com/slaws/bbb67679ba978c6000e3
And logs after a docker kill on the pod : https://gist.github.com/slaws/7bef22b5ae712c5fa756
After the docker kill, I can reach containers from another host.
I'll join logs from the kubelet (with --v=5) ASAP.
After talking with
Support the various types of authentication for plugins as described here.
https://github.com/kubernetes/kubernetes/blob/master/docs/admin/authentication.md
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.