nicholasjackson/demo-consul-service-mesh

Demos built using Docker Compose showing Consul Service Mesh features

Consul Service Mesh Demos

This repository contains various demonstrations to highlight features and configuration in Consul Service Mesh.

Docker Compose

NOTE: All service registration and central config is automatically applied when running these examples.
To apply configuration manually you can comment out the CENTRAL_CONFIG environment variable in the docker-compose.yml files:

  web_envoy:
    image: nicholasjackson/consul-envoy:v1.6.0-v0.10.0
    environment:
      CONSUL_HTTP_ADDR: 10.5.0.2:8500
      CONSUL_GRPC_ADDR: 10.5.0.2:8502
      SERVICE_CONFIG: /config/web_v1.json
#     CENTRAL_CONFIG: "/central_config/web-defaults.hcl;/central_config/api-defaults.hcl;/central_config/api-v1-defaults.hcl;/central_config/api-v2-defaults.hcl;/central_config/api-router.hcl"
    volumes:
      - "./service_config:/config"
      - "./central_config:/central_config"
    command: ["consul", "connect", "envoy","-sidecar-for", "web-v1"]
    network_mode: "service:web"

Config can then be manually applied using the following command once you have started the example with docker-compose up:

$ consul config write config_file.hcl

All central config is stored in the central_config folder in each example.

Traffic Splitting

This demonstration shows how traffic can be split between two service instances. This feature could be used for a Canary or Dark deployment testing strategy.

Traffic Routing

This demonstration shows how upstream traffic can be routed between two services using HTTP paths.

Metrics / Tracing

This demonstration shows how to configure Consul Service Mesh for Observability.

Service Failover

This demonstration shows how to failover upstream services to a different datacenter. This feature could be used to main uptime during a partial or regional service outage.

Consul Gateways

This demonstration shows how to route traffic to a second Consul Datacenter using Consul Gateways. This feature could be used to route traffic between Virtual Machines and Kubernetes.

Kubernetes

Traffic Splitting

This demonstration shows how traffic can be split between two service instances. This feature could be used for a Canary or Dark deployment testing strategy.

Traffic Routing

This demonstration shows how upstream traffic can be routed between two services using HTTP paths.

Exposing Health Checks

Example showing how to bypass proxy security to expose Service health checks publicly.

demo-consul-service-mesh's Issues

Error communicating with upstream service

Hello! I just pulled your repo and stared example by docker-compose up in gateways folder to check communication in different datacenters. Now when I trying to connect to HTTP service at http://127.0.0.1:9090 I see this:

{ "name": "web", "uri": "/", "type": "HTTP", "start_time": "2021-10-27T16:02:26.979074", "end_time": "2021-10-27T16:02:27.005766", "duration": "26.6926ms", "upstream_calls": [ { "uri": "http://localhost:9091", "code": -1, "error": "Error communicating with upstream service: Get http://localhost:9091/: dial tcp 127.0.0.1:9091: connect: connection refused" } ], "code": 500 }

Is it okey or I doing smith wrong?

Scaling services and their sidecars in docker-compose

Is it possible to scale services in a docker-compose based setup that defines both service and sidecar as separate docker-compose services?

I tried to use docker-compose --scale with the examples but that did not work.

There is an Envoy example which scales well, but they take a different approach, they package up proxy and service in a single Docker image.

Is it possible to make scaling work with separate service and sidecar services in docker-compose?

sidecar injection

I do not want to inject sidecars automatically in k8s.
because I want to customize envoy proxy to do other things like jaeger native tracing and many other things planned.
Please provide example of manual configuration of envoy proxy (using yaml) in k8s instead of automatic injection through annotation.

Docker Swarm Consul Setup

Hi,
I tested the uses cases of this demo, on k8s and on a local docker installation, and all works great as expected.
I need to test it on a Docker Swam also but unfortunately I'm encountered a problem at the first step.
I cannot use the compose on the repo because "network_mode:" isn't supported by swarm.
Then, while in the docker local test I forced to apply the container network configuration manually, it is not clear if is possible to have that automated in SWARM.
For example, how could I manage the static ip for my container on docker swarm using an overlay network? I could implement a script that at the bootstrap will set my eth0 but not seems the best approach, and this it would force me to use a custom docker image.
There is a possibility to configure a sort of auto-registration of the consul-envoy-proxy to the consul server within Docker Swarm? (Something like auto-injection in k8s)Maybe I could install consul CLI and run it at bootstrap?
It is not thinkable for me to imagine a design where I must manage each IP of my service mesh network manually.
If you have tested Consul on SWARM infrastructure and can link me to a documentation/example, will be very appreciated because I cannot find anything about the setup on SWAM infrastructure.

Thanks in advance.
Have you a nice day!

Getting invalid config key "Expose"

I am getting this error Bad request, I uncommented in the docker-compose and ran this command ?

pwd
demo-consul-service-mesh/traffic_routing/central_config
SJC-ML-00028512:central_config shitole$ consul config write payments-defaults.hcl 
Error writing config entry "service-defaults" / "payments": Unexpected response code: 400 (Bad request: Request decoding failed: 1 error occurred:
	* invalid config key "Expose"

)

Try to work out with consul routing

I am a newbie to consul, and I am trying to use consul router to dispatch the requests to different instances, but it does not work as my expectation. May I get your help? Thank you every much :).

Overview of the Issue

I have run 2 services in my local PC, the first service (weather-forecast) is to provide information weather info, and the other (weather-client) is to query weather info from weather forecast service.

Weather forecast service has 2 running instances, each listens to port 5001 and 5002. And weather client is listening to port 5005, it will call weather forecast service through proxy upstream port 5008. We can get the weather forecasts information through following URL.

http://localhost:5005/WeatherClient?location={location}

An example of response message is shown as below:

{ "forecasts": [ { "date": "2020-01-04T15:34:55.49344+08:00", "temperatureC": 30, "temperatureF": 85, "summary": "Bracing" }, { "date": "2020-01-05T15:34:55.493447+08:00", "temperatureC": 1, "temperatureF": 33, "summary": "Chilly" }, { "date": "2020-01-06T15:34:55.493447+08:00", "temperatureC": 10, "temperatureF": 49, "summary": "Hot" }, { "date": "2020-01-07T15:34:55.493447+08:00", "temperatureC": 8, "temperatureF": 46, "summary": "Chilly" }, { "date": "2020-01-08T15:34:55.493447+08:00", "temperatureC": 38, "temperatureF": 100, "summary": "Cool" } ], "received": "weather-5001" }

I would like to route the request for location is "Singapore" to instance 5001, and route location is "GuangZhou" to instance 5002, but it doest not works. Followings are the consul service definition files and L7 configuration files:

If the routing works properly, the service instance that received request should be "weather-5001" if the location in query parameter is set to "Singapore", as above sample response shown.

• weather client (weatherClient.json)
  { "service": { "id": "weather-client", "name": "weather-client", "port": 5005, "connect": { "sidecar_service": { "proxy": { "upstreams": [ { "destination_name": "weather", "local_bind_port": 5008 } ] } } } } }

• weather forecast service 1 (weather1.json)
  { "service": { "id": "weather-5001", "name": "weather", "port": 5001, "meta": { "alias": "weather5001" }, "connect": { "sidecar_service": { } } } }

• weather forecast service 2 (weather2.json)
  { "service": { "id": "weather-5002", "name": "weather", "port": 5002, "meta": { "alias": "weather5002" }, "connect": { "sidecar_service": { } } } }

• weather service defaults (weather-service-defaults.json)
  { "kind": "service-defaults", "name": "weather", "protocol": "http" }

• weather service resolver (weather-service-resolver.json)
  { "kind": "service-resolver", "name": "weather", "subsets": { "5001": { "filter": "Service.Meta.alias == weather5001" }, "5002": { "filter": "Service.Meta.alias == weather5002" } } }

• weather service router (weather-service-router.json)
  { "kind": "service-router", "name": "weather", "routes": [ { "match": { "http": { "query_param": [ { "name": "location", "exact": "Singapore" } ] } }, "destination": { "service": "weather", "service_subset": "5001" } }, { "match": { "http": { "query_param": [ { "name": "location", "exact": "GuangZhou" } ] } }, "destination": { "service": "weather", "service_subset": "5002" } } ] }

Reproduction Steps

Steps to reproduce this issue:

1. Initialise a consul server by command (consul agent -dev -ui -config-dir=./weather) and load the service definition, folder "weather" contains service definition files.
2. Run command (consul connect proxy -sidecar-for weather-5001) to initialise the sidecar proxy for all above 3 instances.
3. Run command (consul config write weather-service-defaults.json) to apply configuration to service weather-forcast.
4. Run command (consul config write weather-service-resolver.json) to apply the service resolver to service weather-forcast.
5. Run command (consul config write weather-service-router.json) to apply the service router to service weather-forcast.

And the service router does not work, the request is not dispatched base on the location. I used consul monitor to above command, there was no error being detected.

Thanks a lot for your precious time :)

k8s branch shows docker-compose example instead of k8s

you have a branch called k8s but it does not contain any k8s configration.