containernetworking / cni Goto Github PK

View Code? Open in Web Editor NEW

5.4K 220.0 1.1K 4.39 MB

Container Network Interface - networking for Linux containers

Home Page: https://cni.dev

License: Apache License 2.0

Shell 0.97% Go 98.96% Makefile 0.07%

docker networking containers kubernetes

cni's Introduction

CNI - the Container Network Interface

What is CNI?

CNI (Container Network Interface), a Cloud Native Computing Foundation project, consists of a specification and libraries for writing plugins to configure network interfaces in Linux containers, along with a number of supported plugins. CNI concerns itself only with network connectivity of containers and removing allocated resources when the container is deleted. Because of this focus, CNI has a wide range of support and the specification is simple to implement.

As well as the specification, this repository contains the Go source code of a library for integrating CNI into applications and an example command-line tool for executing CNI plugins. A separate repository contains reference plugins and a template for making new plugins.

The template code makes it straight-forward to create a CNI plugin for an existing container networking project. CNI also makes a good framework for creating a new container networking project from scratch.

Here are the recordings of two sessions that the CNI maintainers hosted at KubeCon/CloudNativeCon 2019:

Contributing to CNI

We welcome contributions, including bug reports, and code and documentation improvements. If you intend to contribute to code or documentation, please read CONTRIBUTING.md. Also see the contact section in this README.

The CNI project has a weekly meeting. It takes place Mondays at 11:00 US/Eastern. All are welcome to join.

Why develop CNI?

Application containers on Linux are a rapidly evolving area, and within this area networking is not well addressed as it is highly environment-specific. We believe that many container runtimes and orchestrators will seek to solve the same problem of making the network layer pluggable.

To avoid duplication, we think it is prudent to define a common interface between the network plugins and container execution: hence we put forward this specification, along with libraries for Go and a set of plugins.

Who is using CNI?

Container runtimes

3rd party plugins

The CNI team also maintains some core plugins in a separate repository.

How do I use CNI?

Requirements

The CNI spec is language agnostic. To use the Go language libraries in this repository, you'll need a recent version of Go. You can find the Go versions covered by our automated tests in .travis.yaml.

Reference Plugins

The CNI project maintains a set of reference plugins that implement the CNI specification. NOTE: the reference plugins used to live in this repository but have been split out into a separate repository as of May 2017.

Running the plugins

After building and installing the reference plugins, you can use the priv-net-run.sh and docker-run.sh scripts in the scripts/ directory to exercise the plugins.

note - priv-net-run.sh depends on jq

Start out by creating a netconf file to describe a network:

$ mkdir -p /etc/cni/net.d
$ cat >/etc/cni/net.d/10-mynet.conf <<EOF
{
	"cniVersion": "0.2.0",
	"name": "mynet",
	"type": "bridge",
	"bridge": "cni0",
	"isGateway": true,
	"ipMasq": true,
	"ipam": {
		"type": "host-local",
		"subnet": "10.22.0.0/16",
		"routes": [
			{ "dst": "0.0.0.0/0" }
		]
	}
}
EOF
$ cat >/etc/cni/net.d/99-loopback.conf <<EOF
{
	"cniVersion": "0.2.0",
	"name": "lo",
	"type": "loopback"
}
EOF

The directory /etc/cni/net.d is the default location in which the scripts will look for net configurations.

Next, build the plugins:

$ cd $GOPATH/src/github.com/containernetworking/plugins
$ ./build_linux.sh # or build_windows.sh

Finally, execute a command (ifconfig in this example) in a private network namespace that has joined the mynet network:

$ CNI_PATH=$GOPATH/src/github.com/containernetworking/plugins/bin
$ cd $GOPATH/src/github.com/containernetworking/cni/scripts
$ sudo CNI_PATH=$CNI_PATH ./priv-net-run.sh ifconfig
eth0      Link encap:Ethernet  HWaddr f2:c2:6f:54:b8:2b  
          inet addr:10.22.0.2  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::f0c2:6fff:fe54:b82b/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:1 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:1 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:90 (90.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

The environment variable CNI_PATH tells the scripts and library where to look for plugin executables.

Running a Docker container with network namespace set up by CNI plugins

Use the instructions in the previous section to define a netconf and build the plugins. Next, docker-run.sh script wraps docker run, to execute the plugins prior to entering the container:

$ CNI_PATH=$GOPATH/src/github.com/containernetworking/plugins/bin
$ cd $GOPATH/src/github.com/containernetworking/cni/scripts
$ sudo CNI_PATH=$CNI_PATH ./docker-run.sh --rm busybox:latest ifconfig
eth0      Link encap:Ethernet  HWaddr fa:60:70:aa:07:d1  
          inet addr:10.22.0.2  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::f860:70ff:feaa:7d1/64 Scope:Link
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:1 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:1 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:90 (90.0 B)  TX bytes:0 (0.0 B)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

What might CNI do in the future?

CNI currently covers a wide range of needs for network configuration due to its simple model and API. However, in the future CNI might want to branch out into other directions:

Dynamic updates to existing network configuration
Dynamic policies for network bandwidth and firewall rules

If these topics are of interest, please contact the team via the mailing list or IRC and find some like-minded people in the community to put a proposal together.

Where are the binaries?

The plugins moved to a separate repo: https://github.com/containernetworking/plugins, and the releases there include binaries and checksums.

Prior to release 0.7.0 the cni release also included a cnitool binary; as this is a developer tool we suggest you build it yourself.

Contact

For any questions about CNI, please reach out via:

Email: cni-dev
IRC: #containernetworking channel on freenode.net
Slack: #cni on the CNCF slack. NOTE: the previous CNI Slack (containernetworking.slack.com) has been sunsetted.

Security

If you have a security issue to report, please do so privately to the email addresses listed in the MAINTAINERS file.

cni's People

Contributors

Stargazers

Watchers

Forkers

eyakubovich jonboulle x57575 hustcts squaremo symmetric dcbw wizard-cxy thockin bprashanth cgonyeo tomdee huhoo kinvolk-archives fawadkhaliq aanm neujie gyliu513 achanda nstott cloudxtreme thunderye weizhang555 chengtiesheng jellonek johnbelamaric dbellotti philips metahertz freehan brecode gvsurenderreddy jainvipin dmueller2001 tmrts fnordahl anguslees logingood jieyu ramitsurana wangshuanpopcs luxas leecalcote feiskyer getmakered michaelhenkel jaysonqy ibuildthecloud hustcat rancher bswang thed00de nimayd rosenhouse srikalyan lukasredynk linearregression shmurthy62 aps-container-service zreigz gooops nhlfr niravthakkar wonderfry autocybernetics antoni dshulyak genti-t junfeiyang jjlakis intelsdi-x jaydunk moypray duanedsouza rmohr tgraf l00196012 40a lstoll rushmorem vchander izogain bowei yuewko squeed digideskio skysec punalpatel yaozengzeng rkamudhan xbinzh jmuhbagir cf-container-networking everesio markstgodard pangbingqiang jiam tacy bboreham hassanshabbirahmed

cni's Issues

plugin's should return interface information in the result

As long as port forwarding is not dealt within CNI there's a lot of work left for the runtime alone.
CNI could make this easier by returning the names/indexes of the created interfaces.

It may be useful to return more/other information than interface names, so an implementation could just add a generic dictionary to the result and let the content depend on the plugins.

spec: add ability for ADD to return arbitrary k/v data to be passed back to DEL

Current design requires the same set of args/data to be passed to both the ADD and DEL commands. This is slightly tricky b/c the net conf (JSON) is often stored on disk. In that case, the container runtime needs to save a copy of it in case it gets changed by the user prior to DEL command execution.

Since the runtime must be saving data to disk, we can clean the API a bit by having the ADD command return a dictionary with arbitrary key/values for the runtime to persist. It would then pipe it back to DEL command. It is more general b/c it allows the plugin to not have to serialize state to disk itself (at least for a subset of cases).

Example ADD:

$plugin_type < /etc/cni/mynet.conf
{
    "ip": "1.2.3.4/24",
    "data": {
        "masqChain": "xyz123",
        "bridgeName": "abc"
    }
}

Followed by DEL:

$plugin_type <<EOF
{
    "masqChain": "xyz123",
    "bridgeName": "abc"
}
EOF

bridge plugin should allow bridging a host iface

As a convenience feature, bridge plugin should have an ability to connect host interface to the bridge (when the bridge is created).

(moved from rkt/rkt#477)

Error message for missing IPAM plugins confusing

The error message from the IPAM plugin search is confusing particularly if the IPAM entry is missing from the config but required.

CNI should have something to say about limiting cross-talk

CNI should have something to say about how to limit cross-talk between networks.
This is tricky to specify as a particular plugin might not know what kind of isolation even exists in the infrastructure. For example, if "bridge" is configured in non-gateway mode (bridged to host's interface), it might not know if it is isolated at L2 from other networks.

In a lot of cases you can trivially isolate via routes in a container's network namespace. However this requires the container to be ran with CAP_NET_ADMIN revoked to ensure the container is not able to change the routes. A cleaner solution provides network isolation outside of the container's namespace.

broken functionality with newer revisions of github.com/vishvananda/netlink

After bumping the netlink library, the following configuration doesn't work as expected:

{
        "name": "default",
        "type": "ptp",
        "ipMasq": true,
        "ipam": {
                "type": "host-local-ptp",
                "subnet": "172.16.28.0/24",
                "routes": [
                        { "dst": "0.0.0.0/0" }
                ]
        }
}

The host can't be reached from within the new network namespace anymore. Diagnostics:

# ip -4 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if19: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state LOWERLAYERDOWN group default  link-netnsid 0
    inet 172.16.28.5/31 scope global eth0
       valid_lft forever preferred_lft forever
# ip -4 r
default via 172.16.28.4 dev eth0 
172.16.28.4/31 dev eth0  proto kernel  scope link  src 172.16.28.5 
# ping -c1 172.16.28.4
PING 172.16.28.4 (172.16.28.4) 56(84) bytes of data.
From 172.16.28.5: icmp_seq=1 Destination Host Unreachable

Question: Configuration of IPAM

Hey All,

Great work going on here, much appreciated (not said enough):

Wanted to ask - why an inline struct was defined here and not a type?

https://github.com/appc/cni/blob/master/pkg/types/types.go#L61

We're integration this into a project and want to be able to configure the IPAM (using your type) upfront. Would you mind if I PR'ed this into being a concrete type or was there a specific design reason for it being as shown above?

Cheers!

Evaluate port mapping

Some containers need to have port mapping between them and the host. The current stance of CNI is that it is outside the scope (and rkt handles port mapping itself). The reasoning behind this is that port mapping is not associated with any particular network but rather is "global" to the container. At the same time port mapping is implemented as a DNAT rule that maps to a particular IP in the container and thus into a network. Therefore it may also make sense to make that a part of the plugin's responsibilities.

DHCP plugin fails with "network is down" error

Seems to be happening with both macvlan and ipvlan. Might be that the interface is DOWN when IPAM is invoked.

Change the plugin model to be job agnostic

I am creating this issue to summarize @thockin's comments on #2. In particular, Tim's proposed to make the spec less opinionated. Current spec defines a network as a group of containers that are able to talk to each other. The user defines which networks a given container needs to join. The container runtime executes a plugin for each network and the job of said plugin is to join the container to the network. As such, there's an assumption that a plugin will create an interface, assign it an IP, etc.

Tim proposes to have the container runtime call out to a plugin (or maybe multiple) and the plugin can do whatever it wants. For example, you can have a plugin to setup IP Masquerade or to block some traffic or change an IP of an existing interface.

Tim's proposal is quite flexible but also a fundamental shift from the current spec. I'd like to discuss this further and also schedule a hangout for Fri or next week: http://doodle.com/arqu4y9rgt9ybfp3

Multiple interfaces in a POD

Hello guys, I was wondering if there are any thoughts/implementations on how to add multiple interfaces inside a POD and connect them to different, for example bridges.

Example:

{
    "interface": [{
        "name": "mynet",
        "type": "bridge",
        "bridge": "cni0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.22.0.0/16",
            "routes": [{
                "dst": "0.0.0.0/0"
            }]
        }
    }, {
        "name": "mynet2",
        "type": "bridge",
        "bridge": "cni1",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.23.0.0/16",
            "routes": [{
                "dst": "0.0.0.0/0"
            }]
        }
    }]
}

If not would it be easy for someone to point at the right files to start hacking ? I am pretty new to open source communities but I would love to contribute here. I also understand that a POD is designed to have one IP address, but the the use case I am experimenting with adds this sort of complexity.

Thanks again

golint warnings

Hi guys, there are some lint warnings in CNI code, I want to make a PR to fix them, is it OK:-)

$ golint ./...
cnitool/cni.go:27:2: exported const EnvCNIPath should have comment (or a comment on this block) or be unexported
libcni/api.go:24:6: exported type RuntimeConf should have comment or be unexported
libcni/api.go:31:6: exported type NetworkConfig should have comment or be unexported
libcni/api.go:36:6: exported type CNI should have comment or be unexported
libcni/api.go:41:6: exported type CNIConfig should have comment or be unexported
libcni/api.go:45:1: exported method CNIConfig.AddNetwork should have comment or be unexported
libcni/api.go:50:1: exported method CNIConfig.DelNetwork should have comment or be unexported
libcni/conf.go:26:1: exported function ConfFromBytes should have comment or be unexported
libcni/conf.go:34:1: exported function ConfFromFile should have comment or be unexported
libcni/conf.go:42:1: exported function ConfFiles should have comment or be unexported
libcni/conf.go:65:1: exported function LoadConf should have comment or be unexported
pkg/invoke/args.go:22:6: exported type CNIArgs should have comment or be unexported
pkg/invoke/args.go:32:1: receiver name should not be an underscore
pkg/invoke/args.go:36:1: exported function ArgsFromEnv should have comment or be unexported
pkg/invoke/args.go:40:6: exported type Args should have comment or be unexported
pkg/invoke/args.go:50:1: exported method Args.AsEnv should have comment or be unexported
pkg/invoke/exec.go:44:1: exported function ExecPluginWithResult should have comment or be unexported
pkg/invoke/exec.go:55:1: exported function ExecPluginWithoutResult should have comment or be unexported
pkg/invoke/find.go:23:1: exported function FindInPath should have comment or be unexported
pkg/ip/ipforward.go:21:1: exported function EnableIP4Forward should have comment or be unexported
pkg/ip/ipforward.go:25:1: exported function EnableIP6Forward should have comment or be unexported
pkg/ipam/ipam.go:28:1: exported function ExecAdd should have comment or be unexported
pkg/ipam/ipam.go:35:1: exported function ExecDel should have comment or be unexported
pkg/types/args.go:24:1: exported function LoadArgs should have comment or be unexported
pkg/types/types.go:23:1: comment on exported type IPNet should be of the form "IPNet ..." (with optional leading article)
pkg/types/types.go:38:1: exported method IPNet.MarshalJSON should have comment or be unexported
pkg/types/types.go:42:1: exported method IPNet.UnmarshalJSON should have comment or be unexported
pkg/types/types.go:72:1: exported method Result.Print should have comment or be unexported
pkg/types/types.go:83:6: exported type Route should have comment or be unexported
pkg/types/types.go:88:6: exported type Error should have comment or be unexported
pkg/types/types.go:98:1: exported method Error.Print should have comment or be unexported
pkg/types/types.go:117:1: exported method IPConfig.MarshalJSON should have comment or be unexported
pkg/types/types.go:127:1: exported method IPConfig.UnmarshalJSON should have comment or be unexported
pkg/types/types.go:139:1: exported method Route.UnmarshalJSON should have comment or be unexported
pkg/types/types.go:150:1: exported method Route.MarshalJSON should have comment or be unexported
plugins/ipam/dhcp/daemon.go:40:6: exported type DHCP should have comment or be unexported
plugins/ipam/dhcp/lease.go:51:6: exported type DHCPLease should have comment or be unexported
plugins/ipam/dhcp/lease.go:279:1: exported method DHCPLease.IPNet should have comment or be unexported
plugins/ipam/dhcp/lease.go:291:1: exported method DHCPLease.Gateway should have comment or be unexported
plugins/ipam/dhcp/lease.go:295:1: exported method DHCPLease.Routes should have comment or be unexported
plugins/ipam/dhcp/lease.go:306:16: should omit type time.Duration from declaration of var baseDelay; it will be inferred from the right-hand side
plugins/ipam/host-local/allocator.go:26:6: exported type IPAllocator should have comment or be unexported
plugins/ipam/host-local/allocator.go:33:1: exported function NewIPAllocator should have comment or be unexported
plugins/ipam/host-local/allocator.go:71:1: comment on exported method IPAllocator.Get should be of the form "Get ..."
plugins/ipam/host-local/allocator.go:136:1: comment on exported method IPAllocator.Release should be of the form "Release ..."
plugins/ipam/host-local/config.go:37:6: exported type IPAMArgs should have comment or be unexported
plugins/ipam/host-local/config.go:41:6: exported type Net should have comment or be unexported
plugins/ipam/host-local/config.go:46:1: comment on exported function LoadIPAMConfig should be of the form "LoadIPAMConfig ..."
plugins/ipam/host-local/backend/store.go:19:6: exported type Store should have comment or be unexported
plugins/ipam/host-local/backend/disk/backend.go:26:6: exported type Store should have comment or be unexported
plugins/ipam/host-local/backend/disk/backend.go:31:1: exported function New should have comment or be unexported
plugins/ipam/host-local/backend/disk/backend.go:44:1: exported method Store.Reserve should have comment or be unexported
plugins/ipam/host-local/backend/disk/backend.go:65:1: exported method Store.Release should have comment or be unexported
plugins/ipam/host-local/backend/disk/backend.go:69:1: comment on exported method Store.ReleaseByID should be of the form "ReleaseByID ..."
plugins/main/bridge/bridge.go:36:6: exported type NetConf should have comment or be unexported
plugins/main/ipvlan/ipvlan.go:32:6: exported type NetConf should have comment or be unexported
plugins/main/macvlan/macvlan.go:32:6: exported type NetConf should have comment or be unexported
plugins/main/ptp/ptp.go:42:6: exported type NetConf should have comment or be unexported
plugins/meta/flannel/flannel.go:42:6: exported type NetConf should have comment or be unexported

ptp: host veth interface has no IP address assigned

Happening on my laptop since a recent system upgrade. It's unclear what's causing this, and no error is produced when e.g. using the scripts to enter a new namespace created by CNI.

The journal shows the following error messages during the CNI plugin setup:

Nov 11 18:26:32 steveej-laptop NetworkManager[955]: <warn>  (veth9a2ffe01): failed to find device 63 'veth9a2ffe01' with udev

My current NetworkManager version 1.0.6

Invalidate arp cache when reusing ips

I create a netcat server that looks up hostname, and a netcat client. I kept the client up, and deleted/recreated the server. It kept coming up with the same ip. However, on the host:

root@e2e-test-beeps-minion-jzy6:/home/beeps# arp -a
? (10.245.1.5) at 02:e6:f9:ed:a0:8b [ether] on cbr0
? (10.245.1.3) at 1a:20:f0:ed:ec:cd [ether] on cbr0
? (10.245.1.4) at de:30:22:0a:5c:24 [ether] on cbr0
? (10.240.0.1) at 42:01:0a:f0:00:01 [ether] on eth0

---[duration T1]---

? (10.245.1.5) at <incomplete> on cbr0
? (10.245.1.3) at 1a:20:f0:ed:ec:cd [ether] on cbr0
? (10.245.1.4) at de:30:22:0a:5c:24 [ether] on cbr0
? (10.240.0.1) at 42:01:0a:f0:00:01 [ether] on eth0

if during duration T1 I send some udp packets to the server through nc -u, tcpdump on the eth0 of the server (inside the container), shows:

Client:

# nc -u 10.245.1.5 8081
hostName
hostName
hostName

--- 3 failures

hostName
netserver-4

Server:

15:16:17-beeps~/goproj/src/k8s.io/kubernetes] (kubelet_plugins)$ ./repro.sh 
pod "netserver-4" deleted
pod "netserver-4" created
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
23:16:34.978072 IP (tos 0x0, ttl 62, id 42388, offset 0, flags [DF], proto UDP (17), length 37)
    10.245.2.6.35824 > 10.245.1.5.8081: [udp sum ok] UDP, length 9
23:16:41.778665 IP (tos 0x0, ttl 62, id 42414, offset 0, flags [DF], proto UDP (17), length 37)
    10.245.2.6.35824 > 10.245.1.5.8081: [udp sum ok] UDP, length 9
23:16:46.928938 IP (tos 0x0, ttl 62, id 43415, offset 0, flags [DF], proto UDP (17), length 37)
    10.245.2.6.35824 > 10.245.1.5.8081: [udp sum ok] UDP, length 9


-----3 failures




23:18:03.738968 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.245.1.5 tell 10.245.1.1, length 28
23:18:04.738960 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.245.1.5 tell 10.245.1.1, length 28
23:18:05.738956 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.245.1.5 tell 10.245.1.1, length 28
23:18:07.873561 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.245.1.5 tell 10.245.1.1, length 28
23:18:07.873586 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.245.1.5 is-at 02:1a:0e:6b:ba:36, length 28

------now another one 

23:18:07.873600 IP (tos 0x0, ttl 62, id 53565, offset 0, flags [DF], proto UDP (17), length 37)
    10.245.2.6.35824 > 10.245.1.5.8081: [udp sum ok] UDP, length 9
23:18:07.873811 IP (tos 0x0, ttl 64, id 6569, offset 0, flags [DF], proto UDP (17), length 39)
    10.245.1.5.8081 > 10.245.2.6.35824: [bad udp cksum 0x1919 -> 0xd673!] UDP, length 11
23:18:12.874950 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.245.1.1 tell 10.245.1.5, length 28
23:18:12.874980 ARP, Ethernet (len 6), IPv4 (len 4), Reply 10.245.1.1 is-at 52:17:41:2a:8b:a4, length 28

This is with v0.1.0, @eyakubovich thoughts?

Edit: net.conf

{
  "bridge":"cbr0",
  "ipam":{
    "routes":[{"dst":"0.0.0.0/0"}],
    "subnet":"10.245.1.0/24",
    "type":"host-local"
  },
  "isGateway":true,
  "name":"kubenet",
  "type":"bridge"
}

dhcp: add support for DNS

Option 6 is used to communicate the DNS server. The dhcp plugin should output this in "dns" field of the result JSON.

Add DNS field to result (#99)
Populate DNS field with DHCP information

*: DNS option only allows to configure nameserver entries

http://linux.die.net/man/5/resolv.conf has some more options than nameserver.
I suggest to add search and domain. I'm not sure if we want/need the others.

/cc @alban

plugin/bridge: host interface has no IP assigned

Config

{
    "name": "bridge1_ipv4",
    "type": "bridge",
    "master": "enp0s20u3u1u3",
    "ipMasq": true,
    "isGateway": true,
    "ipam": {
            "type": "host-local",
            "subnet": "10.0.0.0/24",
            "rangeStart": "10.0.0.230",
            "rangeEnd": "10.0.0.250",
            "routes": [
                {"dst": "0.0.0.0/0"}
            ]
    }
}

Situation

veth host end is DOWN
cni0 bridge on host is DOWN (not a problem)

# bridge link show
...
2: veth53d096e0 state DOWN @(null): <BROADCAST,MULTICAST> mtu 1500 master cni0 state disabled priority 32 cost 2

Add Open vSwitch support

I think it would be nice if appc supports OVS. Is there any plan to support OVS?

Add IPv6 support

The spec does allow for IPv6 but the plugins only support IPv4 (with the exception of host-local).

Tracking:

Set up testing

Add test script and integrate with TravisCI.

When does CNI_COMMAND equal DEL?

I have a bash prototype plugin with two functions activate_network (for when the container starts) and deactivate_network for when then container exits. My environment is as follows:-

rkt version 0.8.0
appc version 0.6.1

and I have the following block at the end of the plugin:-

if [[ "${CNI_COMMAND}" == "ADD" ]]; then
        activate_network
elif [[ "${CNI_COMMAND}" == "DEL" ]]; then
        deactivate_network
fi

When the container starts, $CNI_COMMAND is equal to ADD so networking is setup fine. However on exit, deactivate_network is never run. Am I missing something?

Investigate using p-t-p device for "ptp" plugin

The old p-t-p dialup links were set up so that all interfaces had the same IP on the server. I think we can use the same idea here. This should allow us to use only one IP per container in the case of "ptp" plugin.

What happens if the container already has the interface inside the container?

Name of the interface inside the container. This is the name that should be assigned to the interface created inside the container (network namespace); consequently it must comply with the standard Linux restrictions on interface names.

The spec doesn't have anything related to that. So what is the procedure? Are the plugins allowed to remove, make the necessary modifications, and then recreate the interface with the same name?

Question: Does CNI support multi-hosts network

Hi,

Can you please let me know if CNI currently support multi-host (overlay) network so that containers in different hosts can communicate in the same network? Thanks!

ipvlan plugin did not work

ipvlan did not work for @thockin. Ask him for details.

meet error in ubuntu system when build cni

When I run build, it will meet the error as log in the screen catch as below.

my environment:
Linux zenlin-pc 3.13.0-53-generic #89-Ubuntu SMP Wed May 20 10:34:39 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

Can anyone help me to fix it, or tell me to how to fix it ,I just want to use the cni, and I am going to contribute to it.THX.

spec: UPDATE/CHANGE action

If details of the network change while the pod is alive there needs to be some way to run the CNI plugin to potentially update the pod's network setup for those changes. To be clear, this would be for an existing network that changes some metadata or internal implementation details that the CNI plugin could determine and act upon. For example, in our simple OpenShift multitenant plugin, the network's VNID can change based on administrator action and we need to run the pod network config plugin to updates some pod-specific rules on our OVS vswitch.

Since CNI plugins are executables/scripts that only run on ADD/DEL and quit after their work is done, they have no mechanism for watching for network/config changes. The container runtime (the thing calling ADD/DEL) is the most likely candidate for the UPDATE request, based on internal information it gets from somewhere (a plugin of its own, or other events).

I wouldn't consider this a full ADD/DEL because the network object itself isn't changing, just some arbitrary/internal attributes of the network object. Second, ADD/DEL would obviously interrupt connectivity and might change the IP address of the pod during its lifetime which is undesirable.

Thoughts?

bridge: does not set the gateway on the route

First raised in rkt/rkt#862

bridge does correctly sets the IP on on the bridge (if isGateway is true) but not on the route.

host-local should use /run and not /var for persistance

There's no reason for the IP assignments on the host to survive a reboot.

plugins: audit use of netns path, mandate containerid?

As discovered and discussed in rkt/rkt#851 - there are problems with the ways that some of the plugins utilise the CNI_NETNS parameter:

the veth plugin uses it as a source of entropy for generating interface names for the host. Unfortunately, rkt supplies a (consistent) relative path to this file (which is useful for other reasons and not strictly prohibited by the current wording of the spec). Consequently, multiple invocations of the veth plugin for different containers end up using conflicting "entropy".
macvlan has a similar issue
the host-local IPAM allocator uses the (raw) netns name as an identifier for its local store; in this case, multiple allocations happen to succeed but removals will conflict

Some of these problems would be addressed by mandating that the CNI_NETNS parameter must be an absolute path (which should by definition mean it is unique to that container), but this introduces other issues (for example, in rkt this path might change between when its networking is being set up and destroyed).

A better solution is probably going to be to make CNI_CONTAINERID required, rather than optional as it currently is in the latest draft of the spec, and have plugins use this as a unique container identifier (and safe source of entropy) instead.

spec: plugin shouldn't care if it has been wrapped

Currently the CNI works best if there are two levels of plugins: main and ipam.
Fewer levels (i.e. just main plugin) works fine but more levels are a bit problematic. Consider how the ipam plugin finds its configuration. The JSON that it receives is the same one that was fed to the main plugin. E.g.

{
    "name": "foo",
    "type": "bridge",
    "ipam": {
        "type": "host-local",
        "subnet": "10.1.2.0/24"
    }
}

The host-local IPAM plugin looks for its config under the ipam key. It will also look for the network name under the name key. But what happens if we wanted to have 3 layers? For example, what if we wanted to add another plugin that will install firewall rules to make sure the network can't cross talk to another network:

{
    "name": "foo",
    "type": "isolator",
    "deny": [ "10.0.0.0/8" ],
    "net": {
        "type": "bridge",
        "ipam": {
            "type": "host-local",
            "subnet": "10.1.2.0/24"
        }
    }
}

(If this is actually a config file that a user enters, the UX is horrible but we'll ignore it for now).

The problem with this approach is that both bridge and host-local plugins now need to look for their configs in a slightly different place (net and net.ipam, respectively). There's of course nothing to tell them that. The network name is still at name key.

How should we handle this? One approach is to have the invoking plugin feed only the JSON that is specific for the plugin. For example, an ipam plugin would just receive:

{
    "type": "host-local",
    "subnet": "10.1.2.0/24"
}

The network name would then be passed via an environment variable: CNI_NETNAME. I think this approach would work well but the nested JSON does not present the best UX, in the cases where these configs are entered by the user and stored on disk. Ideas for better handling UX are welcomed.

bridge plugins needs to teardown ipmasq

On Del command, bridge plugin does not remove the installed IP masquerade rules.

Version JSON docs

Config and result JSONs should carry version info. Will need to spell out a strategy on how to also version plugin specific portions of the configs.

missing functional tests in CNI

We should add functional for all combinations of plugins and their configuration.
rkt has partial functional tests for networking, some of which probably fit better in here.

This issue is for overview and tracking purposes.

Investigate ipMasq support from macvlan and ipvlan

I don't think it's really possible to get IP masquerade to work with macvlan as it operates at L2 and probably does not call into netfilter code. In that case, support for it should be removed. ipvlan maybe can work (maybe in one of the modes) but needs to be investigated.

Loopback devices - UP by default on host-bridge?

Hey All,

On the host-bridge plugin the loopback device is down by default. Is there any particular reason for that or could we set it to up as part of the plugin itself? Maybe we introduce a conf value to handle whether or not to up the loopback device in the namespace?

Add socket activation support to dhcp daemon

Under systemd, socket activation support would be highly valuable to save on resource and eliminate startup races.

passing dns resolvers out of a plugin

We need to define the where in the returned JSON metadata a plugin can tell the caller what the DNS configuration should be. This will essentially be a priority ordered list of IP addresses:

"dns-resolvers": ["8.8.8.8", "8.8.4.4"]

Need absolute netns path to write a generic CNI plugin

Implementation starts to get messy when different orchestrators use different paths to the CNI_NETNS. To configure the netns, we need the absolute path, so for rkt, we do something like this:

netns_path = '%s/%s/%s' % (RKT_NETNS_ROOT, env['CNI_CONTAINERID'], env['CNI_NETNS'])

This will be different for other orchestrators using CNI, so it's currently not clear how to write a generic CNI plugin if the prefix of CNI_NETNS isn't known.

docs: create page with an overview of supported CNI_ARGS

plugins: add ipvlan main plugin

Support for ipvlan devices would be nice to have. This should not be much different from macvlan.

macvlan: neighbours are not aware of ARP changes

I would expect for the kernel to take care of this, but somehow it doesn't seem to work if IPs are reused. This issue stems from rkt/rkt#1765.

[Proposal] - Add host-local IPAM support for IPv4 and IPv6 at same time

Currently the host-local IPAM doesn't allow to have a pool of IPv4 and IPv6 addresses.
Does it make sense to have that?

IPAM allocator and backend should be decoupled

We could reorganize the current implementation which could be done right now without losing compatibility with existing configurations.

ipam
- host-local.go (implements combination of disk and sequential)
- store
  - store.go (package, interface)
  - disk/...
- allocator
  - allocator.go (package, interface)
  - sequential/...

However, the next step could be to configure the store and allocator in the configuration file, so that any combination could be chosen. We can probably still allow predefined combinations for known types as host-local.

/cc @aanm

Draft of the spec

The spec is currently being drafted here. It'll be moved into an .md file in this repository shorly.

https://docs.google.com/document/d/1pGJiQhA_5lZsCeswsKDm5v_brkeqaT1Bmn4zANCynGg/edit?usp=sharing

Missing FLANNEL_NETWORK in flannel documentation.

In https://github.com/appc/cni/blob/master/Documentation/flannel.md example subnet.env does not have FLANNEL_NETWORK variable, without which https://github.com/appc/cni/blob/master/plugins/meta/flannel/flannel.go#L205 will panic.

Btw. there is no error checking if required variables are in subnet.env file.
If this variables could be missing, then in https://github.com/appc/cni/blob/master/plugins/meta/flannel/flannel.go#L72 pointer struct members should be initialized with empty values like:

se := &subnetEnv{
  nw: &net.IPNet{},
  sn: &net.IPNet{},
}

plugins: define runtime configurable attributes

Which plugins should support configurable attributes?
What should these be?

main
- bridge
  - MAC
- macvlan
  - MAC
ipam
- host-local(-ptp)
  - IP