Giter Site home page Giter Site logo

libnetwork-plugin's Introduction

Build Status

Repository status

The libnetwork plugin integration is no longer included as a core component in Calico releases, and is not being actively developed or supported by the core Calico team. The latest release of Calico which includes this plugin is the v2.6.x series of releases.

This repository relies on community contributions for enhancements and bug fixes, which will be reviewed and merged by the Calico team on a best-effort basis.

If you would like to become a maintainer of this project, please reach out to the Calico team in the #calico-dev slack channel.

Libnetwork plugin for Calico

This plugin for Docker networking (libnetwork) is intended for use with Project Calico. For Calico's Kubernetes integration, see the CNI plugin.

Supported options for confguration

Working with Networks

  • When creating a network, the --subnet option can be passed to docker network create. The subnet must match an existing Calico pool, and any containers created on that network will use an IP address from that Calico Pool.
  • Other than --driver and --ipam-driver, no other options are supported on the docker network create command.

Working with Containers

When creating containers, use the --net option to connect them to a network previously created with docker network create

  • The --ip option can be passed to docker run to assign a specific IP to a container.
  • The --mac and --link-local options are currently unsupported.

Working with the code

  • Clone the repo (clone it into your GOPATH and make sure you use projectcalico in the path, not your fork name).
  • Create the vendor directory (make vendor). This uses glide in a docker container to create the vendor directory.
  • Build it in a container using make dist/libnetwork-plugin. The plugin binary will appear in the dist directory.
  • Running tests can be done in a container using make test-containerized. Note: This works on linux, but can require additional steps on Mac.
  • Submit PRs through GitHub. Before merging, you'll be asked to squash your commits together, so 1 PR = 1 commit.
  • Before submitting your PR, please make sure tests pass and run make static-checks. Both these will be done by the CI system too though.

How to Run It During Development

make run-plugin

Running the plugin in a container requires a few specific options docker run --rm --net=host --privileged -e CALICO_ETCD_AUTHORITY=$(LOCAL_IP_ENV):2379 -v /run/docker/plugins:/run/docker/plugins -v /var/run/docker.sock:/var/run/docker.sock --name calico-node-libnetwork calico/node-libnetwork /calico

  • --net=host Host network is used since the network changes need to occur in the host namespace
  • privileged since the plugin creates network interfaces
  • -e CALICO_ETCD_AUTHORITY=a.b.c.d:2379 to allow the plugin to find a backend datastore for storing information
  • -v /run/docker/plugins:/run/docker/plugins allows the docker daemon to discover the plugin
  • -v /var/run/docker.sock:/var/run/docker.sock allows the plugin to query the docker daemon

How to Test It

On Linux

make test is all you need.

On OSX/Windows

On OSX/Windows you can't run Docker natively. To allow the Makefile to write the build libnetwork-plugin to your host's filesystem and to allow the test to access the Docker daemon via the unix socket, the user id and group id of the docker user are needed. For boot2docker the user id is 1000 and group id 100.

Run make test like this: LOCAL_USER_ID=1000 LOCAL_GROUP_ID=100 make test-containerized

IPv6 Usage

Note: IPv4 can't be disabled, IPv6 is enabled in addition to IPv4.

Docker IPv6 support must be enabled e.g.

dockerd --cluster-store=etcd://127.0.0.1:2379 --ipv6 --fixed-cidr-v6="2001:db8:1::/64"

Start the libnetwork-plugin

sudo dist/libnetwork-plugin

Add an IPv6 address to the host

sudo ip addr add fd80:24e2:f998:72d7::1/112 dev eth1

Start calico/node, without using the calico/node libnetwork-plugin, also pass in the host IPv6 address

sudo calicoctl node run --disable-docker-networking --ip6=fd80:24e2:f998:72d7::1

Create an IPv6 network

docker network create --ipv6 -d calico --ipam-driver calico-ipam my_net

Run containers on the IPv6 network

  docker run --net my_net --name workload-A -tid busybox
  docker run --net my_net --name workload-B -tid busybox

Check IPv6 network connectivity

  docker exec workload-A ping -6 -c 4 workload-B.my_net
  docker exec workload-B ping -6 -c 4 workload-A.my_net

Check IPv4 network connectivity

  docker exec workload-A ping -4 -c 4 workload-B.my_net
  docker exec workload-B ping -4 -c 4 workload-A.my_net

Known limitations

The following is a list of known limitations when using the Calico libnetwork driver:

  • It is not possible to add multiple networks to a single container. However, once a container endpoint is created, it is possible to manually add additional Calico profiles to that endpoint (effectively adding the container into another network).

Configuring

To change the prefix used for the interface in containers that Docker runs, set the CALICO_LIBNETWORK_IFPREFIX environment variable.

  • The default value is "cali"

To enable debug logging set the CALICO_DEBUG environment variable.

The plugin creates a Calico profile resource for the Docker network used (e.g. docker run --net <network> ...). This is enabled by default. It can be disabled by setting the environment: CALICO_LIBNETWORK_CREATE_PROFILES=false.

The plugin can copy Docker container labels to the corresponding Calico workloadendpoint. This feature is disabled by default. It can be enabled by setting the environment: CALICO_LIBNETWORK_LABEL_ENDPOINTS=true.

Workloadendpoint labelling

If you want to use Calico policies you need labels on the Calico workloadendpoint. The plugin can set labels by copying a subset of the Docker container labels.

To enable this feature you need to set the environment: CALICO_LIBNETWORK_LABEL_ENDPOINTS=true.

Only container labels starting with org.projectcalico.label. are used. This prefix is removed and the remaining key is used a label key in the workloadendpoint.

Example: docker run --label org.projectcalico.label.foo=bar --net <calico network> <image> ... will create a workloadendpoint with label foo=bar. Of course you can use multiple --label org.projectcalico.label.<key>=<value> options.

NOTE: the labels are added to the workloadendpoint using an update, because the container information is not available at the moment the workloadendpoint resource is created.

Troubleshooting

Logging

Logs are sent to STDOUT. If using Docker these can be viewed with the docker logs command.

Monitoring

Check the plugin health by executing API calls.

NetworkDriver:

# echo -e "GET /NetworkDriver.GetCapabilities HTTP/1.0\r\n\r\n" | nc -U /run/docker/plugins/calico.sock
HTTP/1.0 200 OK
Content-Type: application/vnd.docker.plugins.v1.1+json
Date: Thu, 08 Dec 2016 10:00:41 GMT
Content-Length: 19

{"Scope":"global"}

IpamDriver:

# echo -e "GET /IpamDriver.GetCapabilities HTTP/1.0\r\n\r\n" | nc -U /run/docker/plugins/calico-ipam.sock
HTTP/1.0 200 OK
Content-Type: application/vnd.docker.plugins.v1.1+json
Date: Thu, 08 Dec 2016 10:02:51 GMT
Content-Length: 29

{"RequiresMACAddress":false}

Analytics

libnetwork-plugin's People

Contributors

ahrkrak avatar alexaltair avatar alexwlchan avatar bcreane avatar bjwbell avatar caseydavenport avatar cmgs avatar deitch avatar djlwilder avatar djosborne avatar dyakutkin avatar fasaxc avatar gunjan5 avatar jsok avatar lukasa avatar luke-mino-altherr avatar matthewdupre avatar mgleung avatar ozdanborne avatar paultiplady avatar robbrockbank avatar symmetric avatar tomdee avatar trimbiggs avatar xychu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

libnetwork-plugin's Issues

lack of NetworkDriver.ProgramExternalConnectivity method causes failure at docker 1.12

I use calico-libnetwork v0.8.0, these are relevant logs:

  File "/usr/local/libnetwork-plugin/local/lib/python2.7/site-packages/flask/app.py", line 1476, in _find_error_handler
    .get(code))
  File "/usr/local/libnetwork-plugin/local/lib/python2.7/site-packages/flask/app.py", line 1465, in find_handler
    handler = handler_map.get(cls)
AttributeError: 'function' object has no attribute 'get'
 - - [12/Aug/2016:11:46:39 +0000] "POST /NetworkDriver.ProgramExternalConnectivity HTTP/1.1" 500 - "-" "-"

I checked repo and saw following two methods are not implemted yet:

/NetworkDriver.ProgramExternalConnectivity
/NetworkDriver.RevokeExternalConnectivity

Populate workloadendpoint with labels from plugin configuration

Imagine a libnetwork plugin configured with a (new) config key like:

CALICO_WORKLOADENDPOINT_LABELS=foo=bar,baz=quux

Resulting in all workloadendpoints created by that plugin to have 2 labels set at create time:

  • foo: bar
  • baz: quux

This allows us to add labels describing the Mesos agent: datacenter, rack, core count etc.

Probably not useful for policies and Felix, but it allows 'ecosystem' tools to gather information currently not available.

I will submit a PR, but I wanted to open an issue for discussion first.

How to restore container after node restarted

Docker 1.10.3
Calico: 0.18.0

After a Docker node restarted, i cannot start the containers deployed with calico:

Error response from daemon: service endpoint with name flex.worker.worker.v1-i1-d0 already exists
Error: failed to start containers: flex.worker.worker.v1-i1-d0

I tried docker network disconnect & connect, not ok.
Also tried rm this container directly, not ok.

So is there any way i can start this container?

Docker image not buildable at the moment?

It looks like there's some issue with the gliderlabs/alpine image, as I've been unable to get the libnetwork image to build using the current HEAD and the makefile.

Non-Calico IPAM driver error with subnet

Command

docker network create --driver calico --ipam-driver calico-ipam --subnet=192.169.0.0/16 --gateway 192.169.0.1 test2

StdErr

Error response from daemon: NetworkDriver.CreateNetwork: Non-Calico IPAM driver is used

Docker version 1.12.3, build 6b644ec

Plugin doesn't work with Docker 17.x

calico/node STs have started failing with new versions of docker on my machine, suggests the plugin is not working (or needs to be installed differently).

Expected Behavior

docker network create should work with calico plugin

Current Behavior

docker exec -it host1 sh -c 'docker network create --driver calico --ipam-driver calico-ipam  testnet1'
  # Return code: 1
  # Error response from daemon: legacy plugin: plugin not found
  # 

Possible Solution

Error message hints that it can't find the plugin or our socket. Maybe we just need update where we're installed?

Steps to Reproduce (for bugs)

  1. install docker 17.05.0-ce (also repros on 1.12)
  2. install plugin as the STs do
  3. run above command

Context

Your Environment

  • Docker version: 17.05.0-ce
  • Libnetwork-plugin version: v1.1.0 I think
  • How you deployed the libnetwork plugin: calico node STs
  • Operating System and version: Ubuntu 16.04
  • Link to your project (optional):

Include a basic orchestrator in the libnetwork-plugin to handle hard shutdowns

On an unclean shutdown, the plugin leaks IP addresses assigned to workloads. This tends to be problematic when interacting with external job schedulers which may then try to relaunch an IP address targeted workload onto another node, but be rejected due to the IP being seemingly already in use.

It seems like this could be resolved by allowing some optional leader election functionality for assigned IP addresses (i.e. let the plugin be more of an orchestrator out of the box).

Operation would be:

  • Before task launch, the plugin creates a new leader election request for an IP under it's own etcd key (something like (/calico-libnetwork)
  • If it loses the election, then the task is failed since another plugin is already holding the IP actively.
  • If it wins, then it keeps the leader watch alive until the container finishes.

On an unclean shutdown, the leadership challenge would timeout, allowing stale IPs to force released and reclaimed automatically.

Additionally it on election loss while a task is running, it would need to issue a deliberate task-kill to the existing workload on its own node that is running - this would resolve issues that could arise during network partitions.

I've done some basic testing on the idea and it scales fairly well on etcd up to 5000 or so leader locks, since the actual rate of contention on keys is quite low (and the keys are independent). By setting longer timeouts, load can be reduced further.

Calico plugin should enforce that a --gateway option can't be passed to `docker network create`

i.e. the following command should produce a sensible error
docker network create $RANDOM -d calico --ipam-driver calico-ipam --subnet=192.169.0.0/16 --gateway 1.2.3.4

The gateway option has no meaning on a calico network and can cause confusion if we let users specific it.

(In fact, the plugin makes use of the gateway to transport information between API calls, so using a user specified gateway breaks this mechanism)

rc3 goes dead with Too many open files

My reproduction with rc3 (68ca090):

  • restart libnetwork-plugin
  • have a Calico network created
  • exec while :; do docker run --rm --net calico busybox date;done - thus triggering calls to the plugin
  • run a watch on lsof -p $(pgrep -f libnetwork-plugin) | grep -c socket and see the number going up
  • keep running the container create loop and eventually (in our case once the counter is 1014) spot this in the libnetwork output/log:
level=error msg="Veth creation error, hostInterfaceName=caliba064851e4e, tempInterfaceName=tempba064851e4e: too many open files"
  • after this event the plugin logs these events and returns HTTP 500 to Docker:
level=error msg="Hostname fetching error: open /proc/sys/kernel/hostname: too many open files"
  • and it never recovers, only a restart helps and the leak starts all over

My assumption: the dockerCli created in https://github.com/projectcalico/libnetwork-plugin/blob/master/driver/network_driver.go#L141 is not closed anywhere. I will try a defer close.

Plugin doesn't work with the --mac-address option to docker run

docker run --mac-address 11:22:33:44:55:66 --net run1584365895 -tid --name run1584365895 busybox

Produces the following error

23d7d8ed58d29d7bfe061dd4a5ec365d3cc8cf63887c576f3a2b743498879d5a
docker: Error response from daemon: failed to create endpoint run1584365895 on network run1584365895: driver modified interface MAC address: endpoint interface MAC address present (11:22:33:44:55:66). Cannot be modified with ee:ee:ee:ee:ee:ee.; rolled back.

Use Docker container labels to populate the workloadEndpoint

v1.0.0-rc2 creates a workloadEndpoint resource without labels. To use the full power of policy selectors in policies we'd like to enrich the workloadEndpoints with labels passed via the docker run ... --label foo=bar <image> construct.

I've done some prototyping already.

What does not work:

  • Inside a CreateEndpoint method call its impossible to get the container's label, because the NetworkInspect response does not include the container's ID yet.
  • Same goes for the Join method.
  • The container ID is available in the NetworkInfo method call, but a ContainerInspect on the container ID blocks forever. My assumption: Docker locks on the container object before doing the NetworkInfo call.

What does work (I've got a working prototype):

  • Start a goroutine once the workloadEndpoint has been created in CreateEndpoint
  • Inside the goroutine: start polling NetworkInspect until its updated with the container ID
  • Then attempt a ContainerInspect on the ID to get the labels.
  • Upon success: filter the labels for the ones starting with "org.projectcalico.<something>." and add those (without prefix) to the workloadEndpoint and do an update.
  • If NetworkInspect or ContainerInspect fails, try again, until success.

I've created this issue to start a spec discussion, so we all feel comfortable with this solution.

Note: once moby/moby#27638 gets merged and released it may give us a cleaner method of populating a workloadEndpoint, but I can not wait.

/cc @lxpollitt @tomdee

libnetwork-plugin ippools affinity

We are looking for a way to define affinity between calico ippools and host(endpoints).

This request is a bit like: https://github.com/projectcalico/calico-cni/issues/152 but we are (obviously) using libnetwork-plugin.
Our use-case is that we have hard limits on ip-space that we can use per environment and want to be in control of who is using what. We also have 1 etcd cluster for all environments because we want to glue our "legacy" environments to these new environments, and apply policies through felix.

Because currently the ippools live in /calico/v1/ipam/v4/pool we cannot control the usage of them.

Is there currently any way to pin pools to certain hosts(endpoints)? If not we are happy to create this functionality, but then we need some guidelines.

lvs (linux virtual server) in libnetwork

Hi, I had test lvs (linux virtual server) for docker. When I use libnetwork driver, it can't work correctly.
I had created a demo project to re-procedure it. (https://github.com/allanhung/docker-load-balancer)
Please read the demo.txt for create the demo env and run the './run.sh VIP'.
When I execute 'curl VIP', it will be responsed for the docker with bridge driver. And it will not response for the libnetwork driver. I have no idea what's wrong. Please help. Thanks a lot.

Ipam error from docker containerizer.

Running mesos scale test. This run was with 20 nodes, creating 100 containers on each.

2 failed on different nodes. Diags are from one of these. The container that failed is towards the end of list (see mesos agent logs). I can't see any restarts in the logs.

docker.log:May 04 14:50:44 dlagent-011 docker[1828]: time="2016-05-04T14:50:44.376509850Z" level=warning msg="Unable to connect to plugin: /run/docker/plugins/calico.sock:/IpamDriver.RequestAddress, retrying in 1s"
docker.log:May 04 14:50:45 dlagent-011 docker[1828]: time="2016-05-04T14:50:45.376803106Z" level=warning msg="Unable to connect to plugin: /run/docker/plugins/calico.sock:/IpamDriver.RequestAddress, retrying in 2s"
docker.log:May 04 14:50:47 dlagent-011 docker[1828]: time="2016-05-04T14:50:47.378181479Z" level=warning msg="Unable to connect to plugin: /run/docker/plugins/calico.sock:/IpamDriver.RequestAddress, retrying in 4s"
docker.log:May 04 14:50:51 dlagent-011 docker[1828]: time="2016-05-04T14:50:51.378846368Z" level=warning msg="Unable to connect to plugin: /run/docker/plugins/calico.sock:/IpamDriver.RequestAddress, retrying in 8s"
docker.log:May 04 14:50:59 dlagent-011 docker[1828]: time="2016-05-04T14:50:59.440497346Z" level=error msg="Handler for POST /v1.23/containers/de73e797cc698a2dfd545e65a4124f4df848503a242285087d1ab2c58627624f/start returned error: Post http://%2Frun%2Fdocker%2Fplugins%2Fcalico.sock/IpamDriver.RequestAddress: http: ContentLength=56 with Body length 0"

Diags here https://transfer.sh/1Boiv/diags.tar.xz

Version output includes unwanted logs

INFO[0000] No config file specified, loading config from environment 
INFO[0000] Datastore type: etcdv2                       
v1.0.0-beta-rc1

The logs shouldn't be printed when just asking for the version output

make test-containerized fails on OSX

Wanting to add tests to #103 requires me to have a working make test-containerized on OSX. I am using latest docker-machine + docker from brew. It worked before, but changed since somewhere Dec 2016.

I have some workarounds, but eventually I get stuck.

I need to override LOCAL_USER_ID, because my local uid is not the same as the user in my docker-machine. See #127 for a fixing pull request.

Still this is not enough.

My logs on using master:

LOCAL_USER_ID=1000 LOCAL_IP_ENV=$DOCKER_HOSTIP make test-containerized
docker rm -f docker-binary-container 2>&1
Error response from daemon: No such container: docker-binary-container
Makefile:131: recipe for target 'test-containerized' failed
make: [test-containerized] Error 1 (ignored)
docker create --name docker-binary-container docker:rc-dind
a852f1347bbd3c01cc3db7dca64ffa7c39765280bd66ec2cd6740baf922fb75f
docker cp docker-binary-container:/usr/local/bin/docker .
docker rm -f docker-binary-container
docker-binary-container
docker run -ti --rm --net=host \
	-v /Users/rtoma/goprojects/src/github.com/projectcalico/libnetwork-plugin:/go/src/github.com/projectcalico/libnetwork-plugin \
	-v /var/run/docker.sock:/var/run/docker.sock \
	-v /Users/rtoma/goprojects/src/github.com/projectcalico/libnetwork-plugin/.go-pkg-cache:/go/pkg/:rw \
	-v /Users/rtoma/goprojects/src/github.com/projectcalico/libnetwork-plugin/docker:/usr/bin/docker	\
	-e PLUGIN_LOCATION=/Users/rtoma/goprojects/src/github.com/projectcalico/libnetwork-plugin/dist/libnetwork-plugin \
	-e EXTRA_GROUP_ID=`getent group docker | cut -d: -f3` \
	-e LOCAL_USER_ID=1000 \
	-e LOCAL_IP_ENV=192.168.99.100 \
	calico/go-build sh -c '\
		cd  /go/src/github.com/projectcalico/libnetwork-plugin && \
		make test'
/bin/sh: getent: command not found
Starting with UID : 1000
Makefile:145: warning: NUL character seen; rest of line ignored
CGO_ENABLED=0 ginkgo -v tests/*
Running Suite: Custom IF Prefix Libnetwork Suite
================================================
Random Seed: 1483971158
Will run 1 of 1 specs

Running plugin with custom ENV docker run
  creates a container on a network with correct IFPREFIX
  /go/src/github.com/projectcalico/libnetwork-plugin/tests/custom_if_prefix/libnetwork_env_var_test.go:64
make[1]: Entering directory '/go/src/github.com/projectcalico/libnetwork-plugin'
Makefile:66: recipe for target 'run-etcd' failed
docker run --detach \
--net=host \
--name calico-etcd quay.io/coreos/etcd \
etcd \
--advertise-client-urls "http://192.168.99.100:2379,http://127.0.0.1:2379" \
--listen-client-urls "http://0.0.0.0:2379"
Makefile:66: recipe for target 'run-etcd' failed
make[1]: Leaving directory '/go/src/github.com/projectcalico/libnetwork-plugin'

Makefile:145: warning: NUL character seen; rest of line ignored
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
make[1]: [run-etcd] Error 1 (ignored)
docker: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.25/containers/create?name=calico-etcd: dial unix /var/run/docker.sock: connect: permission denied.
See 'docker run --help'.
make[1]: *** [run-etcd] Error 126


•! Panic [4.991 seconds]
Running plugin with custom ENV
/go/src/github.com/projectcalico/libnetwork-plugin/tests/custom_if_prefix/libnetwork_env_var_test.go:66
  docker run
  /go/src/github.com/projectcalico/libnetwork-plugin/tests/custom_if_prefix/libnetwork_env_var_test.go:65
    creates a container on a network with correct IFPREFIX [It]
    /go/src/github.com/projectcalico/libnetwork-plugin/tests/custom_if_prefix/libnetwork_env_var_test.go:64

    Test Panicked
    exit status 2
    /usr/local/go/src/runtime/panic.go:458

    Full Stack Trace
    	/usr/local/go/src/runtime/panic.go:458 +0x243
    github.com/projectcalico/libnetwork-plugin/utils/test.RunPlugin(0xa861c1, 0x22)
    	/go/src/github.com/projectcalico/libnetwork-plugin/utils/test/test_utils.go:112 +0x404
    github.com/projectcalico/libnetwork-plugin/tests/custom_if_prefix.glob..func1.1.1()
    	/go/src/github.com/projectcalico/libnetwork-plugin/tests/custom_if_prefix/libnetwork_env_var_test.go:17 +0x4b
    github.com/projectcalico/libnetwork-plugin/vendor/github.com/onsi/ginkgo/internal/leafnodes.(*runner).runSync(0xc420236c60, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
    	/go/src/github.com/projectcalico/libnetwork-plugin/tests/custom_if_prefix/libnetwork_suite_test.go:14 +0x64
    testing.tRunner(0xc42006c540, 0xab68c0)
    	/usr/local/go/src/testing/testing.go:610 +0x81
    created by testing.(*T).Run
    	/usr/local/go/src/testing/testing.go:646 +0x2ec

------------------------------


Summarizing 1 Failure:

[Panic!] Running plugin with custom ENV docker run [It] creates a container on a network with correct IFPREFIX
/usr/local/go/src/runtime/panic.go:458

Ran 1 of 1 Specs in 4.992 seconds
FAIL! -- 0 Passed | 1 Failed | 0 Pending | 0 Skipped --- FAIL: TestLibnetwork (4.99s)
FAIL

Ginkgo ran 1 suite in 9.535531035s
Test Suite Failed
Makefile:126: recipe for target 'test' failed
make: *** [test] Error 1
Makefile:131: recipe for target 'test-containerized' failed
make: *** [test-containerized] Error 2

What I see in the logs: something needs access to /var/run/docker.sock, but I dont have a clue in what layer. Its like Inception with Docker-in-Docker....

Running containers fails - auto_assign_ips() got an unexpected keyword argument 'hostname'

$ docker run --net net3 --name workload-D -tid busybox
9807c95ff6f41e7c1851f909e065373ee1d5e516c549e90978cf424a3a21794f
Error response from daemon: Cannot start container 9807c95ff6f41e7c1851f909e065373ee1d5e516c549e90978cf424a3a21794f: Plugin Error: IpamDriver.RequestAddress, {
  "Err": "auto_assign_ips() got an unexpected keyword argument 'hostname'"
}
docker ps
CONTAINER ID        IMAGE                           COMMAND               CREATED              STATUS              PORTS               NAMES
315c5637c2b1        calico/node-libnetwork:v0.7.0   "./start.sh"          About a minute ago   Up About a minute                       calico-libnetwork
3b09d91732b1        calico/node:v0.14.0             "/sbin/start_runit"   About a minute ago   Up About a minute                       calico-node


Error response from daemon: NetworkDriver.CreateNetwork: Non-Calico IPAM driver is used

This error is hit when networks are created without the --subnet argument.

2016/11/05 17:31:29 RequestPool response JSON={"PoolID":"CalicoPoolIPv4","Pool":"0.0.0.0/0","Data":{"com.docker.network.gateway":"0.0.0.0/0"}}
2016-11-05 17:31:29.223445 I | Entering go-plugins-helpers createnetwork
2016/11/05 17:31:29 CreateNetwork JSON={"NetworkID":"d169372db9a4dc0e3937c95aa9adaa34acd001b395a7609527c7b93cac55a464","Options":{"com.docker.network.enable_ipv6":false,"com.docker.network.generic":{}},"IPv4Data":[{"AddressSpace":"","Pool":"0.0.0.0/0","Gateway":"0.0.0.0/0","AuxAddresses":null}],"IPv6Data":[]}
2016/11/05 17:31:29 Non-Calico IPAM driver is used

Add (back) IPv6 support

IPv6 support was potentially broken in previous Python versions and support was removed during the Go rewrite. We want to add it back.

I don't think we can until this Docker bug is fixed - moby/moby#28055

When we do, we'll need to make the following changes:
RequestPool - Return the correct pool, poolID and gateway.
For reference the IPv4 reponse is{"PoolID":"CalicoPoolIPv4","Pool":"0.0.0.0/0","Data":{"com.docker.network.gateway":"0.0.0.0/0"}}
RequestAddress - Check whether the request.PoolID is the V6 poolID. If so then take V6 actions.
CreateNetwork - Check the IP6Data for non-Calico IPAM drivers
CreateEndpoint - Ensure that IP address in AddressIPv6 makes it into the Calico endpoint object
Join - Fetch the IPv6 link local address from the host veth and add that as a route into the reponse.

Add ability to specify profiles as a docker network option

docker network create allows you to specify arbitrary options that are passed through to the network driver.

I'd like to add an option to specify which calico profiles the network corresponds to. Useful for a couple of reasons:

  • You get to name the profile associated with a network rather than the libnetwork plugin assigning an arbitrary (and long) UUID for the profile.
  • You could specify multiple profiles. When creating an Endpoint, all of the profiles would be added in the order they are listed.

This would simplify finding the profile so that you can apply advanced policy to a libnetwork'ed container. It would also allow you to easily specify multiple profiles per endpoint. This does not workaround the problem of tweaking which profiles are on an endpoint once the endpoint is created -- however IMHO since a container is lightweight you should just restart the container pointing at a network with the correct profiles.

To implement:

  • Read in the calico-profile=<list of profiles> option and store in our private Network data.
  • When creating the Calico Endpoint data, set the profiles as per the data in the Network data. By default, we should create any profiles that don't exist.
  • If the profiles list is not specified we revert to the current behavior
  • Maybe also add a calico-profile-create=true|false (default true) to specify whether we should attempt to create the profiles when they do not exist (or whether we should just assume that they exist).

We should then update the calico-container docs to reference this new way to access advanced policy in the libnetwork case.

Verify/Fix IPv6 support

Recently ran through the IPv6 demo with Docker 1.12 and v0.9.0 of the plugin. It looks as though the plugin is no longer recognizing IPv6 - either a problem with the demo or the plugin - not sure, but needs investigation.

How do I configure external routing

How do I configure external routing in the new Docker 1.9 libnetwork with calico?

The particular problem I run into is that I can get packets into the container, but not out again.

I'll demonstrate.

I create a network called 'netalphabeta', and add a calico rule for inbound ICMP:

[trinity10] Executing task 'create_networks'
[trinity10] run: docker network create --driver=calico --subnet 192.168.91.0/24 netalphabeta
[trinity10] out: d40eea57e99baac50e57c2dc7159eac0378f13d849de8e31c98db1cc12cdc1f1
[trinity10] out: 

[trinity10] run: docker network ls
[trinity10] out: NETWORK ID          NAME                DRIVER
[trinity10] out: 6a080db420ef        netsolr             calico              
[trinity10] out: d40eea57e99b        netalphabeta        calico              
[trinity10] out: 13fd5c86b05f        none                null                
[trinity10] out: 87b139f81899        host                host                
[trinity10] out: bfcd754ecf23        bridge              bridge              
[trinity10] out: 

[trinity10] Executing task 'configure_network_profiles'
[trinity10] run: docker network ls
[trinity10] out: NETWORK ID          NAME                DRIVER
[trinity10] out: 6a080db420ef        netsolr             calico              
[trinity10] out: d40eea57e99b        netalphabeta        calico              
[trinity10] out: 13fd5c86b05f        none                null                
[trinity10] out: 87b139f81899        host                host                
[trinity10] out: bfcd754ecf23        bridge              bridge              
[trinity10] out: 

[trinity10] run: ./calicoctl profile show
[trinity10] out: +------------------------------------------------------------------+
[trinity10] out: |                               Name                               |
[trinity10] out: +------------------------------------------------------------------+
[trinity10] out: | 6a080db420efe11b19c3812f394236549ac7eab1b8c62b42fdc4959c03ec6ba5 |
[trinity10] out: | d40eea57e99baac50e57c2dc7159eac0378f13d849de8e31c98db1cc12cdc1f1 |
[trinity10] out: +------------------------------------------------------------------+
[trinity10] out: 

[trinity10] run: ./calicoctl profile d40eea57e99baac50e57c2dc7159eac0378f13d849de8e31c98db1cc12cdc1f1 rule add inbound allow icmp
[trinity10] run: docker network ls
[trinity10] out: NETWORK ID          NAME                DRIVER
[trinity10] out: 6a080db420ef        netsolr             calico              
[trinity10] out: d40eea57e99b        netalphabeta        calico              
[trinity10] out: bfcd754ecf23        bridge              bridge              
[trinity10] out: 13fd5c86b05f        none                null                
[trinity10] out: 87b139f81899        host                host                
[trinity10] out: 

My physical LAN network layout is:

[macbook 192.168.88.9] -- [router1 192.168.88.1,192.168.0.1] -- [router2 192.168.0.2,192.168.77.1] -- [docker host trinity10 192.168.77.10].

My router1 has a static route for 192.168.91.0/24 to 192.168.0.2.
My router2 receives routes from Calico, so it has a dynamic route for my containers on trinity10.

Next I run a container on that network, with tcpdump listening for pings on cali0, and try to ping to it from my macbook.

mak@trinity10:~/tcpdump$ docker run --privileged --net netalphabeta -it makuk66/docker-tcpdump /bin/bash -c 'ip addr list; ip route list; sudo tcpdump -i cali0 -n icmp'
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
278: cali0: <NO-CARRIER,BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff
    inet 192.168.91.4/24 scope global cali0
       valid_lft forever preferred_lft forever
    inet6 fe80::ecee:eeff:feee:eeee/64 scope link tentative 
       valid_lft forever preferred_lft forever
280: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 
    link/ether 02:42:ac:12:00:05 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.5/16 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe12:5/64 scope link tentative 
       valid_lft forever preferred_lft forever
default via 172.18.0.1 dev eth1 
172.18.0.0/16 dev eth1  proto kernel  scope link  src 172.18.0.5 
192.168.91.0/24 dev cali0  proto kernel  scope link  src 192.168.91.4 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on cali0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:54:40.758298 IP 192.168.88.9 > 192.168.91.4: ICMP echo request, id 46974, seq 0, length 64
11:54:41.762171 IP 192.168.88.9 > 192.168.91.4: ICMP echo request, id 46974, seq 1, length 64
11:54:42.766340 IP 192.168.88.9 > 192.168.91.4: ICMP echo request, id 46974, seq 2, length 64
^C
mak@crab 513 ~ $ ping 192.168.91.4
PING 192.168.91.4 (192.168.91.4): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
^C

So the echo requests go into the container, but there are no replies.

Looking at the links and routes, we see a cali0 and eth1, with the default route going to eth1.
If I tcpdump eth1, I don't see any ICMP either.

If I add a specific route, I can make this work:

mak@trinity10:~/tcpdump$ docker run --privileged --net netalphabeta -it makuk66/docker-tcpdump /bin/bash -c 'sudo ip route add 192.168.88.0/24 dev cali0; ip addr list; ip route list; tcpdump -i cali0 -n icmp'
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
282: cali0: <NO-CARRIER,BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff
    inet 192.168.91.4/24 scope global cali0
       valid_lft forever preferred_lft forever
    inet6 fe80::ecee:eeff:feee:eeee/64 scope link tentative 
       valid_lft forever preferred_lft forever
284: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default 
    link/ether 02:42:ac:12:00:05 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.5/16 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fe80::42:acff:fe12:5/64 scope link tentative 
       valid_lft forever preferred_lft forever
default via 172.18.0.1 dev eth1 
172.18.0.0/16 dev eth1  proto kernel  scope link  src 172.18.0.5 
192.168.88.0/24 dev cali0  scope link 
192.168.91.0/24 dev cali0  proto kernel  scope link  src 192.168.91.4 
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on cali0, link-type EN10MB (Ethernet), capture size 65535 bytes
11:56:08.632703 IP 192.168.88.9 > 192.168.91.4: ICMP echo request, id 60798, seq 0, length 64
11:56:08.632768 IP 192.168.91.4 > 192.168.88.9: ICMP echo reply, id 60798, seq 0, length 64
11:56:09.636184 IP 192.168.88.9 > 192.168.91.4: ICMP echo request, id 60798, seq 1, length 64
11:56:09.636218 IP 192.168.91.4 > 192.168.88.9: ICMP echo reply, id 60798, seq 1, length 64
^C
PING 192.168.91.4 (192.168.91.4): 56 data bytes
64 bytes from 192.168.91.4: icmp_seq=0 ttl=61 time=0.794 ms
64 bytes from 192.168.91.4: icmp_seq=1 ttl=61 time=0.740 ms
^C

When I tried this back in July, the container would get a single calic0 device, with a default route to that, and it just worked. See for example the second output in calico-docker issue 341.

How can I make make this work with the latest?

Further version details:

[trinity10] Executing task 'info'
[trinity10] run: cat /etc/lsb-release
[trinity10] out: DISTRIB_ID=Ubuntu
[trinity10] out: DISTRIB_RELEASE=14.04
[trinity10] out: DISTRIB_CODENAME=trusty
[trinity10] out: DISTRIB_DESCRIPTION="Ubuntu 14.04.3 LTS"
[trinity10] out: 

[trinity10] Executing task 'docker_version'
[trinity10] run: docker version
[trinity10] out: Client:
[trinity10] out:  Version:      1.9.0
[trinity10] out:  API version:  1.21
[trinity10] out:  Go version:   go1.4.2
[trinity10] out:  Git commit:   76d6bc9
[trinity10] out:  Built:        Tue Nov  3 17:43:42 UTC 2015
[trinity10] out:  OS/Arch:      linux/amd64
[trinity10] out: 
[trinity10] out: Server:
[trinity10] out:  Version:      1.9.0
[trinity10] out:  API version:  1.21
[trinity10] out:  Go version:   go1.4.2
[trinity10] out:  Git commit:   76d6bc9
[trinity10] out:  Built:        Tue Nov  3 17:43:42 UTC 2015
[trinity10] out:  OS/Arch:      linux/amd64
[trinity10] out: 

Flaky test - mainline multi-host

The mainline test is failing intermittently (maybe one in ten runs). This is annoying and should be fixed

======================================================================
FAIL: Run a mainline multi-host test.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tests/st/libnetwork/test_mainline_multi_host.py", line 96, in test_multi_host
    workload_h2n1_1])
  File "/tests/st/utils/utils.py", line 122, in wrapped
    raise e
AssertionError: workload_h1n1_1 cannot ping 10.0.0.2

Question: security code review

Hi,

Did you guys ever did a security review on your code by an external party?
And if yes, are you willing to share these results or can we find it back in some specifc commits?
We are scheduling a code review by an external security company and if you want, we would like to share these results with you/the community as feedback to improve the product :)

Michiel

Port mapping doesn't work

When creating a container, the -p option to publish a port on the container to the host doesn't work on a Calico network.

If created with Docker default networking, then the port mapping survives when the container is added to a Calico network, but as soon as the container is removed from the Docker network, the port mapping disappears.

Docker will accept label names that 'calicoctl apply -f' cannot load

When testing the labeling feature I was able to create labels that docker would accept and could be retrieved with docker inspect but I could not create a policy that used the label. We may want to file a bug against docker because I found https://docs.docker.com/engine/userguide/labels-custom-metadata/ that indicates that only alphanumeric, period, and dashes are allowed though I was able to create labels with special characters and a space.

about the dummy gateway

I’ve upgrade from 0.8.0 to 0.9.0, and my containers became unconnected...Finally i found that the default gateway in my container was 169.254.1.1, which is changed is #51 . i am still not clear about that after reading the whole PR. Could anyone make some explanation for that? And how could i fix the whole cluster, besides rolling back to v0.8.0?

Calico-Libnetwork should fail on unusable opts

From @djosborne on June 8, 2016 20:43

Docker network creation supports passing --opt (as we explain in our documentation). We accept --opt ipip and --opt nat-outgoing as possible options, but they are only used if not using calico-ipam.

From the user's perspective, they might create a network using calico-ipam and pass those two parameters, but Calico will never modify the pool to use them.

Two possible solutions for this issue:

  1. Fail with an error if they use default ipam and still pass one of those two options
  2. Actually apply those options to the pool when they are using Calico IPAM.

We may also want to widen the scope of this issue to fail on all unknown/unused opts passed during creation.

Copied from original issue: projectcalico/calico-containers#1017

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.