networkservicemesh/examples

Network Service Mesh examples repo

Archived

This repo has been archived. Network Service Mesh continues to be very actively developed in multiple repos.

A variety of current examples can be found in the deployment-k8s repo.

examples's Issues

Example make targets and docs don't work well when CRI-O is used as the container runtime

I was attempting to demo some NSM things to some colleagues. The systems I was working on don't use docker as the container runtime, they use CRI-O. Here's an example of what happens when you have CRI-O instead of docker:

suse@tidwellr-dev-1:~/examples> make k8s-icmp-save k8s-icmp-load-images Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post http://%2Fvar%2Frun%2Fdocker.sock/v1.40/build?buildargs=%7B%22VPP_AGENT%22%3A%22artembelov%2Fvpp-agent%3Av2.5.1%22%7D&cachefrom=%5B%5D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=examples%2Fproxy%2Fsidecar-nse%2FDockerfile&labels=%7B%7D&memory=0&memswap=0&networkmode=host&rm=1&session=v7sut2ftomcncr9un32rtyctm&shmsize=0&t=networkservicemesh%2Fproxy-sidecar-nse&target=&ulimits=null&version=1: dial unix /var/run/docker.sock: connect: permission denied make: *** [/home/suse/examples/mk/docker-targets.mk:53: docker-proxy-sidecar-nse-build] Error 1

I encountered this is on openSUSE Tumbleweed running Kubic. Docker is not installed or running in this configuration. The make scripts here assume docker build. Adding support for buildah would be a nice touch. If possible, a little massaging of make targets and docs would be a nice touch for environments where docker is not the container runtime.

icmp example is using non-existing docker image

In icmp example, simple-endpoint.yaml file is using networkservicemesh/proxy-sidecar-nse:latest for the container image.
But the tag latest doesn't exist on docker hub.
Hers is the above mentioned yaml file
https://github.com/networkservicemesh/examples/blob/master/examples/icmp/k8s/simple-endpoint.yaml

I have noticed the same with load balancer example as well.
Can I raise a PR to change it to v0.2.0?

Load-balancer example problem with registration of the endpoint

In load-balancer example the NSMmgr cannot register the newly created load-balancer endpoint. These are the logs for NSMmngr:

{"log":"time="2020-04-21T14:44:03Z" level=info msg="NSM-EndpointMonitor(load-balancerphbwv): Connected"\n","stream":"stderr","time":"2020-04-21T14:44:03.32292345Z"}
{"log":"time="2020-04-21T14:44:03Z" level=info msg="NSM-EndpointMonitor(load-balancerphbwv): Started monitor"\n","stream":"stderr","time":"2020-04-21T14:44:03.323064934Z"}
{"log":"2020/04/21 14:44:03 Reporting span 709295b50ddf86b5:709295b50ddf86b5:0:1\n","stream":"stderr","time":"2020-04-21T14:44:03.323704719Z"}
{"log":"time="2020-04-21T14:44:03Z" level=error msg="NSM-EndpointMonitor(load-balancerphbwv): Connection closed: context canceled"\n","stream":"stderr","time":"2020-04-21T14:44:03.323808357Z"}
{"log":"time="2020-04-21T14:44:03Z" level=info msg="==----\u003e DeleteEndpointWithBrokenConnection() span:43596ab7b463c331:5f4faa0cbf3eda80:3f86b29875e8fdd1:1"\n","stream":"stderr","time":"2020-04-21T14:44:03.324257464Z"}

Load balancer example tcp connection to application server is not working

The nc connection to the application server through the "nc 10.2.2.0 5001" command is not working.

Executing tcpdump commands in application server in the nsm0 interface I observe that the SRC ip of the encapsulated packet is not the 10.70.0.0 but some random IP. Something is not working very well in load balancer plugin when it comes to the TCP connections. The ICMP connections are working fine and the source IPs are 10.70.0.0.

Steps to reproduce:

1. Deploy the load-balancer example
2. Login to the application server pod and execute tcpdump -i nsm0
3. login to load balancer pod and execute "nc 10.2.2.0 5001"
4. Check the Source IPs of the encapsulated packet.

create VETH pair fails when source inft of device1 and dst intf of device2 are same

I tried the topology example use case and it seemed to work for several topologies. But when I create new topologies now, I get an IPAM error since so many NSM requests are hitting the NSE and depleting the IPAM pool. I also noticed that these NSM requests starts coming even before the NSM has started.

Logs at NSE:
Request 1:
time="2020-03-11T05:43:25Z" level=info msg="NSE: channel has been successfully advertised, waiting for connection from NSM..." operation=Endpoint-newtopoceos08-Start span="4db6cd6c7b7a966d:4db6cd6c7b7a966d:0:1"
2020/03/11 05:43:25 Reporting span 4db6cd6c7b7a966d:4db6cd6c7b7a966d:0:1
time="2020-03-11T05:43:28Z" level=info msg=">><<-- GRPC->/networkservice.NetworkService/Request()="" span=6547d7ba2a75324a:6547d7ba2a75324a:0:1"
time="2020-03-11T05:43:28Z" level=info msg=">><<-- request={"connection":{"id":"162","network_service":"newtopoceos08","context":{"ip_context":{"src_ip_required":true,"dst_ip_required":true,"excluded_prefixes":["10.244.0.0/21","10.0.0.0/16","10.244.0.0/21","10.0.0.0/16"]}},"labels":{"link":"net-38","namespace":"default","peerif":"eth1","podName":"deviceceos15-7647c7b9c4-tcwlh"},"path":{"path_segments":[{"name":"k8s-agentpool1-40367033-vmss000002"}]}},"mechanism_preferences":[{"type":"KERNEL_INTERFACE"}]} span=6547d7ba2a75324a:6547d7ba2a75324a:0:1"

Response1:
time="2020-03-11T05:43:28Z" level=info msg=">><<---------- response={"id":"162","network_service":"newtopoceos08","mechanism":{"cls":"LOCAL","type":"KERNEL_INTERFACE","parameters":{"description":"NSM Endpoint","name":"nsmGp6hwFOHW","netnsInode":"4026532964","socketfile":"nsmGp6hwFOHW/memif.sock"}},"context":{"ip_context":{"src_ip_addr":"10.60.8.193/30","dst_ip_addr":"10.60.8.194/30","src_ip_required":true,"dst_ip_required":true,"excluded_prefixes":["10.244.0.0/21","10.0.0.0/16","10.244.0.0/21","10.0.0.0/16"]}},"labels":{"link":"net-38","namespace":"default","peerif":"eth1","podName":"deviceceos16-56c68d5966-bw9hh"},"path":{"path_segments":[{"name":"k8s-agentpool1-40367033-vmss000002"}]}} span=6547d7ba2a75324a:62c7128cff0a3689:20ecd13780b18430:1"

Request2:
time="2020-03-11T05:43:37Z" level=info msg=">><<-- GRPC->/networkservice.NetworkService/Request()="" span=3abb9ad5d81d79ac:3abb9ad5d81d79ac:0:1"
time="2020-03-11T05:43:37Z" level=info msg=">><<-- request={"connection":{"id":"164","network_service":"newtopoceos08","context":{"ip_context":{"src_ip_required":true,"dst_ip_required":true,"excluded_prefixes":["10.244.0.0/21","10.0.0.0/16","10.244.0.0/21","10.0.0.0/16"]}},"labels":{"link":"net-38","namespace":"default","peerif":"eth1","podName":"deviceceos15-7647c7b9c4-tcwlh"},"path":{"path_segments":[{"name":"k8s-agentpool1-40367033-vmss000002"}]}},"mechanism_preferences":[{"type":"KERNEL_INTERFACE"}]} span=3abb9ad5d81d79ac:3abb9ad5d81d79ac:0:1"
time="2020-03-11T05:43:37Z" level=info msg="==--> Endpoint.Request() span:3abb9ad5d81d79ac:664eecb257ab5227:3abb9ad5d81d79ac:1"

Response2:
time="2020-03-11T05:43:37Z" level=info msg=">><<---------- response={"id":"164","network_service":"newtopoceos08","mechanism":{"cls":"LOCAL","type":"KERNEL_INTERFACE","parameters":{"description":"NSM Endpoint","name":"nsmfxDM2ROHL","netnsInode":"4026532964","socketfile":"nsmfxDM2ROHL/memif.sock"}},"context":{"ip_context":{"src_ip_addr":"10.60.8.197/30","dst_ip_addr":"10.60.8.198/30","src_ip_required":true,"dst_ip_required":true,"excluded_prefixes":["10.244.0.0/21","10.0.0.0/16","10.244.0.0/21","10.0.0.0/16"]}},"labels":{"link":"net-38","namespace":"default","peerif":"eth1","podName":"deviceceos16-56c68d5966-bw9hh"},"path":{"path_segments":[{"name":"k8s-agentpool1-40367033-vmss000002"}]}} span=3abb9ad5d81d79ac:56141f7686ad02f5:122dd61c70767e73:1"

and more requests keep coming. I checked the timestamp when NSM started and notice that these requests start coming even before it has started.

time="2020-03-11T05:44:41Z" level=info msg="Starting nsc-sidecar..."
time="2020-03-11T05:44:41Z" level=info msg="Version: "
time="2020-03-11T05:44:41Z" level=info msg="All env variables:"
time="2020-03-11T05:44:41Z" level=info msg="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
time="2020-03-11T05:44:41Z" level=info msg="HOSTNAME=deviceceos15-7647c7b9c4-tcwlh"
time="2020-03-11T05:44:41Z" level=info msg="NSM_CLIENT_SOCKET=/var/lib/networkservicemesh/nsm.client.io.sock"
time="2020-03-11T05:44:41Z" level=info msg="NSM_SERVER_SOCKET=/var/lib/networkservicemesh/nsm.server.io.sock"
time="2020-03-11T05:44:41Z" level=info msg="NSM_DEVICE_PLUGIN=true"
time="2020-03-11T05:44:41Z" level=info msg="INSECURE=true"
time="2020-03-11T05:44:41Z" level=info msg="WORKSPACE=/var/lib/networkservicemesh/"
time="2020-03-11T05:44:41Z" level=info msg="NS_NETWORKSERVICEMESH_IO=newtopoceos08/eth1?link=net-38&peerif=eth1"

Load balancer example check improvement

The load balancer example check script is not verifying the traffic is passing and distributed by the balancer. We need a more sophisticated check that will ensure the balancer is operating as expected.

Simple-client not initialising correctly in examples

Good morning:

When following the instructions on simple-bridge, the containers become stuck in an Init:0/1 state in kubectl get pods, never reaching the "initialised" state.

The command kubectl logs pods/simple-client-ffcdd585c-5mvnl nsm-init-container throws the following errors:

time="2021-03-05T09:45:23Z" level=info msg="==--> RequestNetworkService() span:33b5c79275b34f64:33b5c79275b34f64:0:1"
time="2021-03-05T09:45:23Z" level=info msg="ADVERTISE_NSE_NAME not found."
time="2021-03-05T09:45:23Z" level=info msg="OUTGOING_NSC_NAME not found."
time="2021-03-05T09:45:23Z" level=info msg="ADVERTISE_NSE_LABELS not found."
time="2021-03-05T09:45:23Z" level=info msg="OUTGOING_NSC_LABELS not found."
time="2021-03-05T09:45:23Z" level=info msg="NSC_INTERFACE_NAME not found."
time="2021-03-05T09:45:23Z" level=info msg="MECHANISM_TYPE not found."
time="2021-03-05T09:45:23Z" level=info msg="IP_ADDRESS not found."
time="2021-03-05T09:45:23Z" level=info msg="POD_NAME not found."
time="2021-03-05T09:45:23Z" level=info msg="NSM_NAMESPACE not found."
time="2021-03-05T09:45:23Z" level=info msg="ROUTES not found."
time="2021-03-05T09:45:23Z" level=info msg="NSM_NAMESPACE not found."
time="2021-03-05T09:45:23Z" level=warning msg="global opentracer is already initialized"
time="2021-03-05T09:45:23Z" level=info msg="Creating logger from config: &{nsm-client@simple-client-ffcdd585c-5mvnl false false [] 0xc000184780 0xc00005e3c0 }"

time="2021-03-05T09:50:24Z" level=info msg="FetchX509SVID.Recv failed with rpc error: code = PermissionDenied desc = no identity issued; aborting due to timeout (last success 5m0.018688341s ago)

Any idea what should be done in this case?

Wrong test in secure-intranet example

The check script in secure-intranet tries to connect to port 8080, to see if this is correctly blocked by the ACL filter. There is no service running at port 8080 at the gateway, so this check would fail, regardless of the ACL filter working.

I fixed this by running a simple fileserver at the gateway. Can I just submit a PR for this?

gw-routers example uses incorrect docker images and fails to deploy

In the gw-routers example, docker image networkservicemesh/universal-cnf-vppagent:latest is used for all the containers.
But looking at https://hub.docker.com/r/networkservicemesh/universal-cnf-vppagent/tags; latest tag is not present. Instead it is tagged as master