Comments (12)
kand617
commented on May 20, 2024
1
I have repeated the entire set of instructions using us-east-2 and still get the same error.
Rather annoying to be honest.
Here is the full output
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ eksctl create cluster --name appmeshtest --version 1.12 --nodes-min 2 --nodes-max 3 --nodes 2 --auto-kubeconfig --full-ecr-access --appmesh-access
[ℹ] using region us-east-2
[ℹ] setting availability zones to [us-east-2a us-east-2c us-east-2b]
[ℹ] subnets for us-east-2a - public:192.168.0.0/19 private:192.168.96.0/19
[ℹ] subnets for us-east-2c - public:192.168.32.0/19 private:192.168.128.0/19
[ℹ] subnets for us-east-2b - public:192.168.64.0/19 private:192.168.160.0/19
[ℹ] nodegroup "ng-5531c2ff" will use "ami-04ea7cb66af82ae4a" [AmazonLinux2/1.12]
[ℹ] creating EKS cluster "appmeshtest" in "us-east-2" region
[ℹ] will create 2 separate CloudFormation stacks for cluster itself and the initial nodegroup
[ℹ] if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=us-east-2 --name=appmeshtest'
[ℹ] 2 sequential tasks: { create cluster control plane "appmeshtest", create nodegroup "ng-5531c2ff" }
[ℹ] building cluster stack "eksctl-appmeshtest-cluster"
[ℹ] deploying stack "eksctl-appmeshtest-cluster"
[ℹ] buildings nodegroup stack "eksctl-appmeshtest-nodegroup-ng-5531c2ff"
[ℹ] deploying stack "eksctl-appmeshtest-nodegroup-ng-5531c2ff"
[✔] all EKS cluster resource for "appmeshtest" had been created
[✔] saved kubeconfig as "/Users/andalak/.kube/eksctl/clusters/appmeshtest"
[ℹ] adding role "arn:aws:iam::562185760820:role/eksctl-appmeshtest-nodegroup-ng-5-NodeInstanceRole-12WS57YBIBF9W" to auth ConfigMap
[ℹ] nodegroup "ng-5531c2ff" has 0 node(s)
[ℹ] waiting for at least 2 node(s) to become ready in "ng-5531c2ff"
[ℹ] nodegroup "ng-5531c2ff" has 2 node(s)
[ℹ] node "ip-192-168-46-96.us-east-2.compute.internal" is ready
[ℹ] node "ip-192-168-64-68.us-east-2.compute.internal" is ready
[ℹ] kubectl command should work with "/Users/andalak/.kube/eksctl/clusters/appmeshtest", try 'kubectl --kubeconfig=/Users/andalak/.kube/eksctl/clusters/appmeshtest get nodes'
[✔] EKS cluster "appmeshtest" in "us-east-2" region is ready
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ export KUBECONFIG=~/.kube/eksctl/clusters/appmeshtest
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ export MESH_NAME=color-mesh
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ export INJECT_XRAY_SIDECAR=true
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ export ENABLE_STATS_TAGS=true
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ export MESH_REGION=us-east-2
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ curl https://raw.githubusercontent.com/aws/aws-app-mesh-inject/v0.1.3/hack/install.sh | bash
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1164 100 1164 0 0 2443 0 --:--:-- --:--:-- --:--:-- 2440
\nWorking directory at /var/folders/by/_mqxmh194bn7v0whkmd_zk4m0000gn/T/tmp.7d4DSVqh\n
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 68 100 68 0 0 196 0 --:--:-- --:--:-- --:--:-- 196
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3489 100 3489 0 0 9815 0 --:--:-- --:--:-- --:--:-- 9828
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2200 100 2200 0 0 6261 0 --:--:-- --:--:-- --:--:-- 6250
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 622 100 622 0 0 1725 0 --:--:-- --:--:-- --:--:-- 1727
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1076 100 1076 0 0 2988 0 --:--:-- --:--:-- --:--:-- 2988
namespace/appmesh-inject created
creating certs in tmpdir /var/folders/by/_mqxmh194bn7v0whkmd_zk4m0000gn/T/tmp.2ZAfYtHu
Generating RSA private key, 2048 bit long modulus
................................................................+++
..........................................................................+++
e is 65537 (0x10001)
certificatesigningrequest.certificates.k8s.io/aws-app-mesh-inject.appmesh-inject created
NAME AGE REQUESTOR CONDITION
aws-app-mesh-inject.appmesh-inject 2s kubernetes-admin Pending
certificatesigningrequest.certificates.k8s.io/aws-app-mesh-inject.appmesh-inject approved
secret/aws-app-mesh-inject created
processing templates
Created injector manifest at:/var/folders/by/_mqxmh194bn7v0whkmd_zk4m0000gn/T/tmp.7d4DSVqh/_output/inject.yaml
serviceaccount/aws-app-mesh-inject-sa created
clusterrole.rbac.authorization.k8s.io/aws-app-mesh-inject-cr created
clusterrolebinding.rbac.authorization.k8s.io/aws-app-mesh-inject-binding created
service/aws-app-mesh-inject created
deployment.apps/aws-app-mesh-inject created
mutatingwebhookconfiguration.admissionregistration.k8s.io/aws-app-mesh-inject created
waiting for aws-app-mesh-inject to start
Waiting for deployment "aws-app-mesh-inject" rollout to finish: 0 of 1 updated replicas are available...
deployment "aws-app-mesh-inject" successfully rolled out
Mesh name has been set up
The injector is ready
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ kubectl -n appmesh-inject get po
NAME READY STATUS RESTARTS AGE
aws-app-mesh-inject-86859c84f9-7lclq 1/1 Running 0 40s
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ curl https://raw.githubusercontent.com/aws/aws-app-mesh-controller-for-k8s/v0.1.0/deploy/v0.1.0/all.yaml | kubectl apply -f -
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 8419 100 8419 0 0 22104 0 --:--:-- --:--:-- --:--:-- 22097
customresourcedefinition.apiextensions.k8s.io/meshes.appmesh.k8s.aws created
customresourcedefinition.apiextensions.k8s.io/virtualnodes.appmesh.k8s.aws created
customresourcedefinition.apiextensions.k8s.io/virtualservices.appmesh.k8s.aws created
namespace/appmesh-system created
deployment.apps/app-mesh-controller created
serviceaccount/app-mesh-sa created
clusterrole.rbac.authorization.k8s.io/app-mesh-controller created
clusterrolebinding.rbac.authorization.k8s.io/app-mesh-controller-binding created
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ kubectl wait $(kubectl get pods -n appmesh-system -o name) \
> --for=condition=Ready --timeout=30s -n appmesh-system
pod/app-mesh-controller-65897498cb-284qt condition met
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ kubectl api-resources --api-group=appmesh.k8s.aws
NAME SHORTNAMES APIGROUP NAMESPACED KIND
meshes appmesh.k8s.aws false Mesh
virtualnodes appmesh.k8s.aws true VirtualNode
virtualservices appmesh.k8s.aws true VirtualService
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ ls
CODE_OF_CONDUCT.md FAQ.md NOTICE examples
CONTRIBUTING.md LICENSE README.md walkthroughs
SINC02W54EDHTD6:aws-app-mesh-examples andalak$ cd walkthroughs/eks/
SINC02W54EDHTD6:eks andalak$ cd aws-app-mesh-controller-for-k8s/
SINC02W54EDHTD6:aws-app-mesh-controller-for-k8s andalak$ make example
./hack/example.sh
namespace/appmesh-demo created
mesh.appmesh.k8s.aws/color-mesh created
virtualnode.appmesh.k8s.aws/colorgateway created
virtualnode.appmesh.k8s.aws/colorteller created
virtualnode.appmesh.k8s.aws/colorteller-black created
virtualnode.appmesh.k8s.aws/colorteller-blue created
virtualnode.appmesh.k8s.aws/colorteller-red created
virtualservice.appmesh.k8s.aws/colorteller.appmesh-demo created
virtualservice.appmesh.k8s.aws/colorgateway.appmesh-demo created
service/colorgateway created
deployment.apps/colorgateway created
service/colorteller created
deployment.apps/colorteller created
service/colorteller-black created
deployment.apps/colorteller-black created
service/colorteller-blue created
deployment.apps/colorteller-blue created
service/colorteller-red created
deployment.apps/colorteller-red created
SINC02W54EDHTD6:aws-app-mesh-controller-for-k8s andalak$ kubectl -n appmesh-demo \
> get deploy,po,svc,virtualnode.appmesh.k8s.aws,virtualservice.appmesh.k8s.aws
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.extensions/colorgateway 1 1 1 1 42s
deployment.extensions/colorteller 1 1 1 1 40s
deployment.extensions/colorteller-black 1 1 1 1 38s
deployment.extensions/colorteller-blue 1 1 1 1 36s
deployment.extensions/colorteller-red 1 1 1 1 34s
NAME READY STATUS RESTARTS AGE
pod/colorgateway-69cd4fc669-5d5vl 3/3 Running 0 42s
pod/colorteller-845959f54-cwd67 3/3 Running 0 40s
pod/colorteller-black-6cc98458db-5jslx 3/3 Running 0 38s
pod/colorteller-blue-88bcffddb-t7h4f 3/3 Running 0 36s
pod/colorteller-red-6f55b447db-zxmx2 3/3 Running 0 34s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/colorgateway ClusterIP 10.100.146.35 <none> 9080/TCP 44s
service/colorteller ClusterIP 10.100.151.85 <none> 9080/TCP 42s
service/colorteller-black ClusterIP 10.100.68.10 <none> 9080/TCP 40s
service/colorteller-blue ClusterIP 10.100.227.111 <none> 9080/TCP 38s
service/colorteller-red ClusterIP 10.100.82.10 <none> 9080/TCP 36s
NAME AGE
virtualnode.appmesh.k8s.aws/colorgateway 50s
virtualnode.appmesh.k8s.aws/colorteller 49s
virtualnode.appmesh.k8s.aws/colorteller-black 48s
virtualnode.appmesh.k8s.aws/colorteller-blue 47s
virtualnode.appmesh.k8s.aws/colorteller-red 46s
NAME AGE
virtualservice.appmesh.k8s.aws/colorgateway.appmesh-demo 45s
virtualservice.appmesh.k8s.aws/colorteller.appmesh-demo 45s
SINC02W54EDHTD6:aws-app-mesh-controller-for-k8s andalak$ aws appmesh list-meshes --region us-east-2
{
"meshes": [
{
"meshName": "color-mesh",
"arn": "arn:aws:appmesh:us-east-2:562185760820:mesh/color-mesh"
}
]
}
SINC02W54EDHTD6:aws-app-mesh-controller-for-k8s andalak$ aws appmesh list-virtual-services \
> --mesh-name color-mesh \
> --region us-east-2
{
"virtualServices": [
{
"meshName": "color-mesh",
"virtualServiceName": "colorgateway.appmesh-demo",
"arn": "arn:aws:appmesh:us-east-2:562185760820:mesh/color-mesh/virtualService/colorgateway.appmesh-demo"
},
{
"meshName": "color-mesh",
"virtualServiceName": "colorteller.appmesh-demo",
"arn": "arn:aws:appmesh:us-east-2:562185760820:mesh/color-mesh/virtualService/colorteller.appmesh-demo"
}
]
}
SINC02W54EDHTD6:aws-app-mesh-controller-for-k8s andalak$ aws appmesh list-virtual-nodes \
> --mesh-name color-mesh \
> --region us-east-2
{
"virtualNodes": [
{
"meshName": "color-mesh",
"arn": "arn:aws:appmesh:us-east-2:562185760820:mesh/color-mesh/virtualNode/colorgateway-appmesh-demo",
"virtualNodeName": "colorgateway-appmesh-demo"
},
{
"meshName": "color-mesh",
"arn": "arn:aws:appmesh:us-east-2:562185760820:mesh/color-mesh/virtualNode/colorteller-appmesh-demo",
"virtualNodeName": "colorteller-appmesh-demo"
},
{
"meshName": "color-mesh",
"arn": "arn:aws:appmesh:us-east-2:562185760820:mesh/color-mesh/virtualNode/colorteller-black-appmesh-demo",
"virtualNodeName": "colorteller-black-appmesh-demo"
},
{
"meshName": "color-mesh",
"arn": "arn:aws:appmesh:us-east-2:562185760820:mesh/color-mesh/virtualNode/colorteller-red-appmesh-demo",
"virtualNodeName": "colorteller-red-appmesh-demo"
},
{
"meshName": "color-mesh",
"arn": "arn:aws:appmesh:us-east-2:562185760820:mesh/color-mesh/virtualNode/colorteller-blue-appmesh-demo",
"virtualNodeName": "colorteller-blue-appmesh-demo"
}
]
}
SINC02W54EDHTD6:aws-app-mesh-controller-for-k8s andalak$ kubectl -n appmesh-demo \
> run -it curler \
> --image=tutum/curl /bin/bash
kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
If you don't see a command prompt, try pressing enter.
root@curler-5875dfcc64-2x2hj:/#
root@curler-5875dfcc64-2x2hj:/# curl colorgateway:9080/color
curl: (7) Failed to connect to colorgateway port 9080: Connection refused
from aws-app-mesh-examples.
kiranmeduri
commented on May 20, 2024
1
Can you confirm if the following works?
root@curler-5875dfcc64-2x2hj:/# curl http://colorgateway.appmesh-demo:9080/color
from aws-app-mesh-examples.
rothgar
commented on May 20, 2024
1
I appear to be having the same problem (also in us-west-2)
I can dig the endpoint but telnet also gets connection refused. I thought maybe it was a network policy but there aren't any
kubectl get netpol --all-namespaces
No resources found.
from aws-app-mesh-examples.
kiranmeduri
commented on May 20, 2024
You may see errors due to any of the following. But these errors should go away once the configurations is distributed.
- service-discovery is not ready yet. So colorgateway is not resolving to any ip.
- colorgateway envoy is not primed yet (i.e. Envoy config is not distributed or loaded yet).
- colorgateway app container is not ready yet.
I noticed a different behavior that maps to (2) and (3)
root@curler-5b467f98bb-9zq5p:/# curl colorgateway:9080/color
500 - Unexpected Error
...
...
root@curler-5b467f98bb-9zq5p:/# curl colorgateway:9080/color
500 - Unexpected Error
root@curler-5b467f98bb-9zq5p:/# curl colorgateway:9080/color
{"color":"upstream connect error or disconnect/reset before headers", "stats": {"upstream connect error or disconnect/reset before headers":1}}
...
root@curler-5b467f98bb-9zq5p:/# curl colorgateway:9080/color
{"color":"black", "stats": {"black":0.33,"upstream connect error or disconnect/reset before headers":0.67}}
....
root@curler-5b467f98bb-9zq5p:/# curl colorgateway:9080/color
{"color":"blue", "stats": {"black":0.38,"blue":0.29,"upstream connect error or disconnect/reset before headers":0.01,"white":0.32}}
from aws-app-mesh-examples.
mhausenblas
commented on May 20, 2024
@kand617 @rothgar thanks for reporting this! I can't reproduce it but looking into it.
from aws-app-mesh-examples.
nckturner
commented on May 20, 2024
kubectl port-forward -n appmesh-demo \
$(kubectl get pod -n appmesh-demo |grep colorgateway \
|awk '{print $1}') 9901
and then navigate to localhost:9901/config_dump in your browser and copy the output here?
from aws-app-mesh-examples.
rothgar
commented on May 20, 2024
Here you go.
{
"configs": [
{
"@type": "type.googleapis.com/envoy.admin.v2alpha.BootstrapConfigDump",
"bootstrap": {
"node": {
"id": "mesh/color-mesh/virtualNode/colorgateway-appmesh-demo",
"cluster": "mesh/color-mesh/virtualNode/colorgateway-appmesh-demo",
"build_version": "ae8c8aa036e58e39b3d2fba81f5bdc4683a30682/1.9.0/Clean/DEBUG/BoringSSL"
},
"dynamic_resources": {
"lds_config": {
"ads": {}
},
"cds_config": {
"ads": {}
},
"ads_config": {
"api_type": "GRPC",
"grpc_services": [
{
"google_grpc": {
"target_uri": "appmesh-envoy-management.us-west-2.amazonaws.com:443",
"channel_credentials": {
"ssl_credentials": {
"root_certs": {
"filename": "/etc/pki/tls/cert.pem"
}
}
},
"call_credentials": [
{
"from_plugin": {
"name": "envoy.grpc_credentials.aws_iam",
"config": {
"region": "us-west-2",
"service_name": "appmesh"
}
}
}
],
"stat_prefix": "ads",
"credentials_factory_name": "envoy.grpc_credentials.aws_iam"
}
}
]
}
},
"tracing": {
"http": {
"name": "envoy.xray",
"config": {
"daemon_endpoint": "127.0.0.1:2000"
}
}
},
"admin": {
"access_log_path": "/tmp/envoy_admin_access.log",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 9901
}
}
},
"stats_config": {
"stats_tags": [
{
"tag_name": "appmesh.mesh",
"fixed_value": "color-mesh"
},
{
"tag_name": "appmesh.virtual_node",
"fixed_value": "colorgateway-appmesh-demo"
}
]
}
},
"last_updated": "2019-04-29T17:46:45.101Z"
},
{
"@type": "type.googleapis.com/envoy.admin.v2alpha.ClustersConfigDump",
"version_info": "261982128",
"dynamic_active_clusters": [
{
"version_info": "261982128",
"cluster": {
"name": "cds_egress_color-mesh_amazonaws",
"type": "ORIGINAL_DST",
"connect_timeout": "1s",
"lb_policy": "ORIGINAL_DST_LB"
},
"last_updated": "2019-04-30T17:57:42.358Z"
},
{
"version_info": "261982128",
"cluster": {
"name": "cds_egress_color-mesh_colorteller-appmesh-demo_http_9080",
"type": "LOGICAL_DNS",
"connect_timeout": "1s",
"load_assignment": {
"cluster_name": "cds_egress_color-mesh_colorteller-appmesh-demo_http_9080",
"endpoints": [
{
"lb_endpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "colorteller.appmesh-demo.svc.cluster.local",
"port_value": 9080
}
}
}
}
]
}
]
}
},
"last_updated": "2019-04-30T17:57:42.365Z"
},
{
"version_info": "261982128",
"cluster": {
"name": "cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080",
"type": "LOGICAL_DNS",
"connect_timeout": "1s",
"load_assignment": {
"cluster_name": "cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080",
"endpoints": [
{
"lb_endpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "colorteller-black.appmesh-demo.svc.cluster.local",
"port_value": 9080
}
}
}
}
]
}
]
}
},
"last_updated": "2019-04-30T17:57:42.362Z"
},
{
"version_info": "261982128",
"cluster": {
"name": "cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080",
"type": "LOGICAL_DNS",
"connect_timeout": "1s",
"load_assignment": {
"cluster_name": "cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080",
"endpoints": [
{
"lb_endpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "colorteller-blue.appmesh-demo.svc.cluster.local",
"port_value": 9080
}
}
}
}
]
}
]
}
},
"last_updated": "2019-04-30T17:57:42.369Z"
},
{
"version_info": "261982128",
"cluster": {
"name": "cds_ingress_color-mesh_colorgateway-appmesh-demo_http_9080",
"connect_timeout": "0.250s",
"load_assignment": {
"cluster_name": "cds_ingress_color-mesh_colorgateway-appmesh-demo_http_9080",
"endpoints": [
{
"lb_endpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "127.0.0.1",
"port_value": 9080
}
}
}
}
]
}
]
}
},
"last_updated": "2019-04-30T17:57:42.355Z"
}
]
},
{
"@type": "type.googleapis.com/envoy.admin.v2alpha.ListenersConfigDump",
"version_info": "261982128",
"dynamic_active_listeners": [
{
"version_info": "261982128",
"listener": {
"name": "lds_ingress_0.0.0.0_15000",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 15000
}
},
"filter_chains": [
{
"filter_chain_match": {
"destination_port": 9080
},
"filters": [
{
"name": "envoy.http_connection_manager",
"config": {
"http_filters": [
{
"name": "envoy.router"
}
],
"rds": {
"config_source": {
"ads": {}
},
"route_config_name": "rds_ingress_http_9080"
},
"stat_prefix": "ingress",
"tracing": {}
}
}
]
}
],
"listener_filters": [
{
"name": "envoy.listener.original_dst"
}
]
},
"last_updated": "2019-04-30T17:57:42.385Z"
},
{
"version_info": "261982128",
"listener": {
"name": "lds_egress_0.0.0.0_15001",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 15001
}
},
"filter_chains": [
{
"filter_chain_match": {
"destination_port": 9080
},
"filters": [
{
"name": "envoy.http_connection_manager",
"config": {
"http_filters": [
{
"name": "envoy.router"
}
],
"rds": {
"config_source": {
"ads": {}
},
"route_config_name": "rds_egress_http_9080"
},
"stat_prefix": "egress",
"tracing": {
"operation_name": "EGRESS"
}
}
}
]
},
{
"filter_chain_match": {
"destination_port": 443,
"server_names": [
"*.amazonaws.com"
]
},
"filters": [
{
"name": "envoy.tcp_proxy",
"config": {
"stat_prefix": "egress",
"cluster": "cds_egress_color-mesh_amazonaws"
}
}
]
}
],
"listener_filters": [
{
"name": "envoy.listener.original_dst"
},
{
"name": "envoy.listener.tls_inspector"
}
]
},
"last_updated": "2019-04-30T17:57:42.389Z"
}
]
},
{
"@type": "type.googleapis.com/envoy.admin.v2alpha.RoutesConfigDump",
"dynamic_route_configs": [
{
"version_info": "261982128",
"route_config": {
"name": "rds_egress_http_9080",
"virtual_hosts": [
{
"name": "colorteller.appmesh-demo",
"domains": [
"colorteller.appmesh-demo",
"colorteller.appmesh-demo:9080"
],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"weighted_clusters": {
"clusters": [
{
"name": "cds_egress_color-mesh_colorteller-appmesh-demo_http_9080",
"weight": 1
},
{
"name": "cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080",
"weight": 1
},
{
"name": "cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080",
"weight": 1
}
],
"total_weight": 3
}
}
}
]
}
]
},
"last_updated": "2019-04-30T17:57:42.393Z"
},
{
"version_info": "261982128",
"route_config": {
"name": "rds_ingress_http_9080",
"virtual_hosts": [
{
"name": "local_service",
"domains": [
"*"
],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "cds_ingress_color-mesh_colorgateway-appmesh-demo_http_9080"
}
}
]
}
]
},
"last_updated": "2019-04-30T17:57:42.393Z"
}
]
}
]
}
from aws-app-mesh-examples.
rothgar
commented on May 20, 2024
After not touching this for 6 days it appears the problem resolved itself for me. I am now able to curl the color gateway from inside the cluster.
curl http://colorgateway:9080/color
{"color":"black", "stats": {"black":1}}
Next time I see an error I'll enable debug logging on the gateway sidecar and see if I see anything in the logs.
For anyone else that also needs to do this:
To enable debug log level you need to change the appmesh inject hook deployment via
kubectl edit deploy aws-app-mesh-inject -n appmesh-inject
Change the APPMESH_LOG_LEVEL environment variable on the container spec from info to debug
Then restart any pods you want to have debug logging on the envoy sidecar.
from aws-app-mesh-examples.
kiranmeduri
commented on May 20, 2024
Please reopen if you see any more issues. Thanks
from aws-app-mesh-examples.
jicowan
commented on May 20, 2024
@kiranmeduri I am also getting this error.
{
"configs": [
{
"@type": "type.googleapis.com/envoy.admin.v2alpha.BootstrapConfigDump",
"bootstrap": {
"node": {
"id": "mesh/color-mesh/virtualNode/colorgateway-appmesh-demo",
"cluster": "mesh/color-mesh/virtualNode/colorgateway-appmesh-demo",
"build_version": "ae8c8aa036e58e39b3d2fba81f5bdc4683a30682/1.9.0/Clean/DEBUG/BoringSSL"
},
"dynamic_resources": {
"lds_config": {
"ads": {}
},
"cds_config": {
"ads": {}
},
"ads_config": {
"api_type": "GRPC",
"grpc_services": [
{
"google_grpc": {
"target_uri": "appmesh-envoy-management.us-east-1.amazonaws.com:443",
"channel_credentials": {
"ssl_credentials": {
"root_certs": {
"filename": "/etc/pki/tls/cert.pem"
}
}
},
"call_credentials": [
{
"from_plugin": {
"name": "envoy.grpc_credentials.aws_iam",
"config": {
"service_name": "appmesh",
"region": "us-east-1"
}
}
}
],
"stat_prefix": "ads",
"credentials_factory_name": "envoy.grpc_credentials.aws_iam"
}
}
]
}
},
"tracing": {
"http": {
"name": "envoy.xray",
"config": {
"daemon_endpoint": "127.0.0.1:2000"
}
}
},
"admin": {
"access_log_path": "/tmp/envoy_admin_access.log",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 9901
}
}
},
"stats_config": {
"stats_tags": [
{
"tag_name": "appmesh.mesh",
"fixed_value": "color-mesh"
},
{
"tag_name": "appmesh.virtual_node",
"fixed_value": "colorgateway-appmesh-demo"
}
]
}
},
"last_updated": "2019-08-27T16:35:56.090Z"
},
{
"@type": "type.googleapis.com/envoy.admin.v2alpha.ClustersConfigDump",
"version_info": "380526692",
"dynamic_active_clusters": [
{
"version_info": "380509227",
"cluster": {
"name": "cds_egress_color-mesh_amazonaws",
"type": "ORIGINAL_DST",
"connect_timeout": "1s",
"lb_policy": "ORIGINAL_DST_LB"
},
"last_updated": "2019-08-27T16:35:56.339Z"
},
{
"version_info": "380526692",
"cluster": {
"name": "cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080",
"type": "LOGICAL_DNS",
"connect_timeout": "1s",
"load_assignment": {
"cluster_name": "cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080",
"endpoints": [
{
"lb_endpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "colorgateway.appmesh-demo.svc.cluster.local",
"port_value": 9080
}
}
}
}
]
}
]
}
},
"last_updated": "2019-08-27T16:55:17.377Z"
},
{
"version_info": "380509227",
"cluster": {
"name": "cds_ingress_color-mesh_colorgateway-appmesh-demo_http_9080",
"connect_timeout": "0.250s",
"load_assignment": {
"cluster_name": "cds_ingress_color-mesh_colorgateway-appmesh-demo_http_9080",
"endpoints": [
{
"lb_endpoints": [
{
"endpoint": {
"address": {
"socket_address": {
"address": "127.0.0.1",
"port_value": 9080
}
}
}
}
]
}
]
}
},
"last_updated": "2019-08-27T16:35:56.335Z"
}
]
},
{
"@type": "type.googleapis.com/envoy.admin.v2alpha.ListenersConfigDump",
"version_info": "380526692",
"dynamic_active_listeners": [
{
"version_info": "380509227",
"listener": {
"name": "lds_ingress_0.0.0.0_15000",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 15000
}
},
"filter_chains": [
{
"filter_chain_match": {
"destination_port": 9080
},
"filters": [
{
"name": "envoy.http_connection_manager",
"config": {
"http_filters": [
{
"name": "envoy.router"
}
],
"rds": {
"route_config_name": "rds_ingress_http_9080",
"config_source": {
"ads": {}
}
},
"stat_prefix": "ingress",
"tracing": {}
}
}
]
}
],
"listener_filters": [
{
"name": "envoy.listener.original_dst"
}
]
},
"last_updated": "2019-08-27T16:35:56.370Z"
},
{
"version_info": "380509227",
"listener": {
"name": "lds_egress_0.0.0.0_15001",
"address": {
"socket_address": {
"address": "0.0.0.0",
"port_value": 15001
}
},
"filter_chains": [
{
"filter_chain_match": {
"destination_port": 9080
},
"filters": [
{
"name": "envoy.http_connection_manager",
"config": {
"http_filters": [
{
"name": "envoy.router"
}
],
"rds": {
"route_config_name": "rds_egress_http_9080",
"config_source": {
"ads": {}
}
},
"stat_prefix": "egress",
"tracing": {
"operation_name": "EGRESS"
}
}
}
]
},
{
"filter_chain_match": {
"destination_port": 443,
"server_names": [
"*.amazonaws.com"
]
},
"filters": [
{
"name": "envoy.tcp_proxy",
"config": {
"stat_prefix": "egress",
"cluster": "cds_egress_color-mesh_amazonaws"
}
}
]
}
],
"listener_filters": [
{
"name": "envoy.listener.original_dst"
},
{
"name": "envoy.listener.tls_inspector"
}
]
},
"last_updated": "2019-08-27T16:35:56.377Z"
}
]
},
{
"@type": "type.googleapis.com/envoy.admin.v2alpha.RoutesConfigDump",
"dynamic_route_configs": [
{
"version_info": "380526692",
"route_config": {
"name": "rds_egress_http_9080",
"virtual_hosts": [
{
"name": "colorteller.appmesh-demo",
"domains": [
"colorteller.appmesh-demo",
"colorteller.appmesh-demo:9080"
],
"routes": [
{
"match": {
"prefix": "/color"
},
"route": {
"weighted_clusters": {
"clusters": [
{
"name": "cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080",
"weight": 1
}
],
"total_weight": 1
}
}
}
]
}
]
},
"last_updated": "2019-08-27T16:55:17.380Z"
},
{
"version_info": "380509227",
"route_config": {
"name": "rds_ingress_http_9080",
"virtual_hosts": [
{
"name": "local_service",
"domains": [
"*"
],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "cds_ingress_color-mesh_colorgateway-appmesh-demo_http_9080"
}
}
]
}
]
},
"last_updated": "2019-08-27T16:55:17.380Z"
}
]
}
]
}
from aws-app-mesh-examples.
jicowan
commented on May 20, 2024
The logs from the color-gateway envoy container:
Did not find Envoy configuration file at /tmp/envoy.yaml, creating one.
added envoy.xray tracing config to /tmp/envoy_tracing_config.yaml
Appending /tmp/envoy_tracing_config.yaml to /tmp/envoy.yaml
Appending /tmp/envoy_stats_config.yaml to /tmp/envoy.yaml
Starting Envoy.
[2019-08-27 16:35:56.074][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws_Init_Cleanup Initiate AWS SDK for C++ with Version:1.6.39
[2019-08-27 16:35:56.075][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] CurlHttpClient Initializing Curl library
[2019-08-27 16:35:56.075][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::AWSConfigFileProfileConfigLoader Initializing config loader against fileName //.aws/config and using profilePrefix = 1
[2019-08-27 16:35:56.075][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::AWSConfigFileProfileConfigLoader Unable to open config file //.aws/config for reading.
[2019-08-27 16:35:56.075][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::AWSProfileConfigLoader Failed to reload configuration.
[2019-08-27 16:35:56.084][000001][info][main] [source/server/server.cc:206] initializing epoch 0 (hot restart version=10.200.16384.567.options=capacity=16384, num_slots=8209 hash=228984379728933363 size=9863272)
[2019-08-27 16:35:56.084][000001][info][main] [source/server/server.cc:208] statically linked extensions:
[2019-08-27 16:35:56.085][000001][info][main] [source/server/server.cc:210] access_loggers: envoy.file_access_log,envoy.http_grpc_access_log
[2019-08-27 16:35:56.085][000001][info][main] [source/server/server.cc:213] filters.http: envoy.buffer,envoy.cors,envoy.ext_authz,envoy.fault,envoy.filters.http.header_to_metadata,envoy.filters.http.jwt_authn,envoy.filters.http.rbac,envoy.grpc_http1_bridge,envoy.grpc_json_transcoder,envoy.grpc_web,envoy.gzip,envoy.health_check,envoy.http_dynamo_filter,envoy.ip_tagging,envoy.lua,envoy.rate_limit,envoy.router,envoy.squash
[2019-08-27 16:35:56.085][000001][info][main] [source/server/server.cc:216] filters.listener: envoy.listener.original_dst,envoy.listener.proxy_protocol,envoy.listener.tls_inspector
[2019-08-27 16:35:56.085][000001][info][main] [source/server/server.cc:219] filters.network: envoy.client_ssl_auth,envoy.echo,envoy.ext_authz,envoy.filters.network.dubbo_proxy,envoy.filters.network.rbac,envoy.filters.network.sni_cluster,envoy.filters.network.thrift_proxy,envoy.http_connection_manager,envoy.mongo_proxy,envoy.ratelimit,envoy.redis_proxy,envoy.tcp_proxy
[2019-08-27 16:35:56.085][000001][info][main] [source/server/server.cc:221] stat_sinks: envoy.dog_statsd,envoy.metrics_service,envoy.stat_sinks.hystrix,envoy.statsd
[2019-08-27 16:35:56.085][000001][info][main] [source/server/server.cc:223] tracers: envoy.dynamic.ot,envoy.lightstep,envoy.tracers.datadog,envoy.xray,envoy.zipkin
[2019-08-27 16:35:56.085][000001][info][main] [source/server/server.cc:226] transport_sockets.downstream: envoy.transport_sockets.alts,envoy.transport_sockets.capture,raw_buffer,tls
[2019-08-27 16:35:56.085][000001][info][main] [source/server/server.cc:229] transport_sockets.upstream: envoy.transport_sockets.alts,envoy.transport_sockets.capture,raw_buffer,tls
[2019-08-27 16:35:56.097][000001][info][main] [source/server/server.cc:271] admin address: 0.0.0.0:9901
[2019-08-27 16:35:56.100][000001][info][config] [source/server/configuration_impl.cc:50] loading 0 static secret(s)
[2019-08-27 16:35:56.100][000001][info][config] [source/server/configuration_impl.cc:56] loading 0 cluster(s)
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::AWSConfigFileProfileConfigLoader Initializing config loader against fileName //.aws/config and using profilePrefix = 1
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::AWSConfigFileProfileConfigLoader Initializing config loader against fileName //.aws/credentials and using profilePrefix = 0
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] ProfileConfigFileAWSCredentialsProvider Setting provider to read credentials from //.aws/credentials for credentials file and //.aws/config for the config file , for use with profile default
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] EC2MetadataClient Creating AWSHttpResourceClient with max connections2 and scheme http
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] CurlHandleContainer Initializing CurlHandleContainer with size 2
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] InstanceProfileCredentialsProvider Creating Instance with default EC2MetadataClient and refresh rate 300000
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] DefaultAWSCredentialsProviderChain Added EC2 metadata service credentials provider to the provider chain.
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::AWSConfigFileProfileConfigLoader Unable to open config file //.aws/credentials for reading.
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::AWSProfileConfigLoader Failed to reload configuration.
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::AWSConfigFileProfileConfigLoader Unable to open config file //.aws/config for reading.
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::AWSProfileConfigLoader Failed to reload configuration.
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] InstanceProfileCredentialsProvider Credentials have expired attempting to repull from EC2 Metadata Service.
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] CurlHandleContainer Pool grown by 2
[2019-08-27 16:35:56.101][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] CurlHandleContainer Connection has been released. Continuing.
[2019-08-27 16:35:56.107][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] CurlHandleContainer Connection has been released. Continuing.
[2019-08-27 16:35:56.108][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::EC2InstanceProfileConfigLoader Successfully pulled credentials from metadata service with access key ASIA36C6WWEJWDONYVPS
[2019-08-27 16:35:56.108][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] CurlHandleContainer Connection has been released. Continuing.
[2019-08-27 16:35:56.111][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] EC2MetadataClient Detected current region as us-east-1
[2019-08-27 16:35:56.111][000001][info][misc] [source/common/aws/aws_sdk_config.cc:66] Aws::Config::AWSProfileConfigLoader Successfully reloaded configuration.
[2019-08-27 16:35:56.112][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:132] cm init: initializing cds
[2019-08-27 16:35:56.113][000001][info][config] [source/server/configuration_impl.cc:67] loading 0 listener(s)
[2019-08-27 16:35:56.113][000001][info][config] [source/server/configuration_impl.cc:92] loading tracing configuration
[2019-08-27 16:35:56.113][000001][info][config] [source/server/configuration_impl.cc:101] loading tracing driver: envoy.xray
[2019-08-27 16:35:56.113][000001][info][tracing] [source/extensions/tracers/xray/xray_tracer_impl.cc:95] send X-Ray generated segments to daemon address on 127.0.0.1:2000
[2019-08-27 16:35:56.113][000001][info][tracing] [source/extensions/tracers/xray/sampling.cc:114] unable to parse empty json file. falling back to default rule set.
[2019-08-27 16:35:56.113][000001][info][config] [source/server/configuration_impl.cc:112] loading stats sink configuration
[2019-08-27 16:35:56.114][000001][info][main] [source/server/server.cc:463] starting main dispatch loop
[2019-08-27 16:35:56.335][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:495] add/update cluster cds_ingress_color-mesh_colorgateway-appmesh-demo_http_9080 during init
[2019-08-27 16:35:56.339][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:495] add/update cluster cds_egress_color-mesh_amazonaws during init
[2019-08-27 16:35:56.343][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:495] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 during init
[2019-08-27 16:35:56.348][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:136] cm init: all clusters initialized
[2019-08-27 16:35:56.349][000001][info][main] [source/server/server.cc:435] all clusters initialized. initializing init manager
[2019-08-27 16:35:56.370][000001][info][upstream] [source/server/lds_api.cc:80] lds: add/update listener 'lds_ingress_0.0.0.0_15000'
[2019-08-27 16:35:56.377][000001][info][upstream] [source/server/lds_api.cc:80] lds: add/update listener 'lds_egress_0.0.0.0_15001'
[2019-08-27 16:35:56.380][000001][info][config] [source/server/listener_manager_impl.cc:961] all dependencies initialized. starting workers
[2019-08-27 16:35:56.381][000026][info][tracing] [source/extensions/tracers/xray/sampling.cc:114] unable to parse empty json file. falling back to default rule set.
[2019-08-27 16:35:56.384][000025][info][tracing] [source/extensions/tracers/xray/sampling.cc:114] unable to parse empty json file. falling back to default rule set.
[2019-08-27 16:36:16.692][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:36:16.696][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:36:16.700][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:36:16.700][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:36:16.705][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:36:16.706][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:36:16.706][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:36:22.843][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:36:22.844][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:36:22.844][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:36:22.844][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:36:22.848][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:36:31.221][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c0f-9690f7f98045c542f2ae421f","id":"30aab7ac92f5b2ae","start_time":1566923791.219,"end_time":1566923791.221,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"bf04a61628ef37ab","start_time":1566923791.219,"end_time":1566923791.221,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:31.230][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c0f-42162373c8c3aba2e817b927","id":"60801b68312f640d","start_time":1566923791.225,"end_time":1566923791.23,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"0b2cfbb1c71f14be","start_time":1566923791.225,"end_time":1566923791.23,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:32.246][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c10-57908fc8d5bb3c7da1884003","id":"2335d50fee5c7b09","start_time":1566923792.245,"end_time":1566923792.246,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"911e288a0cfc754d","start_time":1566923792.245,"end_time":1566923792.246,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:32.247][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c10-ce43a36f52dedf1d053de6aa","id":"a42d2e44c8970be9","start_time":1566923792.245,"end_time":1566923792.247,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"1cf46b876d50555c","start_time":1566923792.245,"end_time":1566923792.247,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:33.933][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c11-f96ffae8823598deec13f32e","id":"7b1a21207269c8da","start_time":1566923793.932,"end_time":1566923793.933,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"dc765124056c2ac0","start_time":1566923793.932,"end_time":1566923793.933,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:33.937][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c11-d0b1c281991774e4ca7160b6","id":"cd8a13d9d4294be5","start_time":1566923793.937,"end_time":1566923793.937,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"28af1df04469b170","start_time":1566923793.937,"end_time":1566923793.937,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:34.074][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c12-9b5625b8fde4f2a9113df105","id":"8ade8076afe86412","start_time":1566923794.071,"end_time":1566923794.074,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"38216aeed9a65f15","start_time":1566923794.071,"end_time":1566923794.074,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:34.087][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c12-7f394949dfb38dd74485a47d","id":"75c9e5c2a9ea96de","start_time":1566923794.086,"end_time":1566923794.086,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/v2/swagger","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"bdfa873de1dc4030","start_time":1566923794.086,"end_time":1566923794.086,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/v2/swagger","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:36.647][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c14-b393f94784998cb191072d02","id":"389c69c14e7509c9","start_time":1566923796.646,"end_time":1566923796.647,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"3e6a12ed22814fb0","start_time":1566923796.646,"end_time":1566923796.647,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:36.650][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c14-d65a4607d00d98895342b46a","id":"f004800ddf662762","start_time":1566923796.65,"end_time":1566923796.65,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"0444236bb564d431","start_time":1566923796.65,"end_time":1566923796.65,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:41.011][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c19-3f1365eafc113a9834c3003d","id":"85f24a9209ecb425","start_time":1566923801.01,"end_time":1566923801.011,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"b20b46092ab50b63","start_time":1566923801.01,"end_time":1566923801.011,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:41.016][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c19-d178e04b06cc35bdbbd93088","id":"c487e98ff604535b","start_time":1566923801.016,"end_time":1566923801.016,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"77242094b456e046","start_time":1566923801.016,"end_time":1566923801.016,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:46.921][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c1e-2f8877dd09541c5ef01232f9","id":"098bc7fe083e9855","start_time":1566923806.921,"end_time":1566923806.921,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"551a7f19db1121e9","start_time":1566923806.921,"end_time":1566923806.921,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:46.925][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c1e-181ea0f505ab291cc40620e9","id":"ba40882ce8e47935","start_time":1566923806.924,"end_time":1566923806.925,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"7bfa4c56b851ea73","start_time":1566923806.924,"end_time":1566923806.925,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:47.379][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c1f-2d93ec03ed290fef53c5679c","id":"82d443aeab25c815","start_time":1566923807.378,"end_time":1566923807.379,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"61066e7a8cff2e97","start_time":1566923807.378,"end_time":1566923807.379,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:47.384][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c1f-1ac6518eb758890c34b21358","id":"6ece48aeaed5829c","start_time":1566923807.384,"end_time":1566923807.384,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"da0daf3222b659fd","start_time":1566923807.384,"end_time":1566923807.384,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v1","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:57.115][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c29-408a902e02a3206c08aee235","id":"21a9e0161ce07082","start_time":1566923817.114,"end_time":1566923817.115,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"d16e8f575786396f","start_time":1566923817.114,"end_time":1566923817.115,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:58.175][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c2a-6a22b00c59bf010ac7e4d52d","id":"a2220a597eb6b50d","start_time":1566923818.174,"end_time":1566923818.175,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"ced55b99f7bf4105","start_time":1566923818.174,"end_time":1566923818.175,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:36:58.182][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c2a-4b389c8d89527b99aaaca981","id":"034d266b612f3cfb","start_time":1566923818.181,"end_time":1566923818.182,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v2","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"e292b3ae8635d4d9","start_time":1566923818.181,"end_time":1566923818.182,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v2","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:37:11.444][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c37-9da982261c8771fac5d1c559","id":"2d979c1084109d7b","start_time":1566923831.443,"end_time":1566923831.444,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"67a801e65d2c1860","start_time":1566923831.443,"end_time":1566923831.444,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:37:11.451][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c37-11e6da533ba1244e22c28402","id":"d55b29228545b217","start_time":1566923831.45,"end_time":1566923831.45,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v2","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"27c53c2228a171e5","start_time":1566923831.45,"end_time":1566923831.451,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger/docs/v2","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:37:15.205][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655c3b-4f5fa5a703e8ceeab1122325","id":"f3bb02660042a546","start_time":1566923835.205,"end_time":1566923835.205,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"12ac2774026b141d","start_time":1566923835.205,"end_time":1566923835.205,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/swagger.json","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:38:18.555][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:38:18.560][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:38:18.564][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:38:18.564][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:38:18.570][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:38:18.570][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:38:18.571][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:38:50.963][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:38:50.963][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:38:50.963][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:38:50.963][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:38:50.967][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:39:25.561][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655cbd-773aaf902bfc5f45c6e23dfd","id":"c7054ee4b5764622","start_time":1566923965.557,"end_time":1566923965.561,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}},"fault":true,"subsegments":[{"id":"93d59b29fa677fa8","start_time":1566923965.557,"end_time":1566923965.561,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}}}]}
[2019-08-27 16:39:59.847][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655cdf-457474a7a6ce1ee7d5f3fc9e","id":"0d18d773c8464405","start_time":1566923999.846,"end_time":1566923999.847,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}},"fault":true,"subsegments":[{"id":"5c51600a7c27d6ca","start_time":1566923999.846,"end_time":1566923999.847,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}}}]}
[2019-08-27 16:40:05.475][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655ce5-8d263ffe7d3192dbc6de3599","id":"23fb523576030e4a","start_time":1566924005.474,"end_time":1566924005.475,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway:9080/","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":404,"content_length":19}},"error":true,"subsegments":[{"id":"3ef9a9a8dd31b855","start_time":1566924005.474,"end_time":1566924005.475,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway:9080/","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":404,"content_length":19}}}]}
[2019-08-27 16:41:12.556][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655bfc-756e155007b07e0a07ce401b","id":"3b11288019966597","start_time":1566923772.557,"end_time":1566924072.556,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","method":"POST","user_agent":"grpc-go/1.13.0"},"response":{"status":200,"content_length":0}},"subsegments":[{"id":"b66fa3df617bd770","start_time":1566923772.557,"end_time":1566924072.556,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","method":"POST","user_agent":"grpc-go/1.13.0"},"response":{"status":200,"content_length":0}}}]}
[2019-08-27 16:42:44.265][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:42:44.268][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:42:44.272][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:42:44.272][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:42:44.277][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:42:44.278][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:42:44.278][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:42:51.243][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:42:51.244][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:42:51.244][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:42:51.244][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:42:51.252][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:45:21.407][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:45:21.411][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:45:21.414][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:45:21.415][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:45:21.420][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:45:21.420][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:45:21.420][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:45:44.949][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:45:44.949][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:45:44.950][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:45:44.950][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:45:44.961][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:46:12.561][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655d28-ef3a285d80f6070e3fef9c7b","id":"22e259786d0ce813","start_time":1566924072.556,"end_time":1566924372.561,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","method":"POST","user_agent":"grpc-go/1.13.0"},"response":{"status":200,"content_length":0}},"subsegments":[{"id":"d7378a2306a465b3","start_time":1566924072.556,"end_time":1566924372.561,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","method":"POST","user_agent":"grpc-go/1.13.0"},"response":{"status":200,"content_length":0}}}]}
[2019-08-27 16:46:15.217][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:46:15.221][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:46:15.225][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:46:15.225][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:46:15.229][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:46:15.230][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:46:15.230][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:46:20.324][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:46:20.325][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:46:20.325][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:46:20.325][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:46:20.336][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:47:16.965][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:47:16.968][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:47:16.972][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:47:16.973][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:47:16.977][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:47:16.977][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:47:16.978][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:47:48.463][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:47:48.463][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:47:48.464][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:47:48.464][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:47:48.468][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:48:48.893][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:48:48.897][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:48:48.901][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:48:48.901][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:48:48.912][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:48:48.913][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:48:48.913][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:49:16.953][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:49:16.953][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:49:16.953][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:49:16.953][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:49:16.958][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:49:26.701][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655f16-155f79dbc0adff965f8314e7","id":"8f48dc82fb677629","start_time":1566924566.701,"end_time":1566924566.701,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorteller.appmesh-demo:9080/","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":0}},"error":true,"subsegments":[{"id":"5b031f10ddef6e9f","start_time":1566924566.701,"end_time":1566924566.701,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorteller.appmesh-demo:9080/","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":0}}}]}
[2019-08-27 16:49:26.701][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655f16-dfa361e11e658bed8a9e3d6c","id":"caa1b6c481370778","start_time":1566924566.699,"end_time":1566924566.701,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}},"fault":true,"subsegments":[{"id":"e66c18d57d270102","start_time":1566924566.699,"end_time":1566924566.701,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}}}]}
[2019-08-27 16:50:56.381][000001][info][main] [source/server/drain_manager_impl.cc:63] shutting down parent after drain
[2019-08-27 16:51:15.096][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655e57-87786acbd920320f26d5c87a","id":"f461b214f287c51c","start_time":1566924375.095,"end_time":1566924675.096,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","method":"POST","user_agent":"grpc-go/1.13.0"},"response":{"status":200,"content_length":0}},"subsegments":[{"id":"d273e95e3ff64083","start_time":1566924375.095,"end_time":1566924675.096,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","method":"POST","user_agent":"grpc-go/1.13.0"},"response":{"status":200,"content_length":0}}}]}
[2019-08-27 16:51:51.103][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:51:51.107][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:51:51.111][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:51:51.111][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:51:51.115][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:51:51.115][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:51:51.116][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:52:19.077][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:52:19.078][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:52:19.078][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:52:19.079][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:52:19.083][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:52:32.357][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655fd0-87f42b34ce51fcb34dd6fd10","id":"8040acb300bba569","start_time":1566924752.356,"end_time":1566924752.356,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorteller.appmesh-demo:9080/","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":0}},"error":true,"subsegments":[{"id":"43bcccab587d6c42","start_time":1566924752.356,"end_time":1566924752.356,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorteller.appmesh-demo:9080/","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":0}}}]}
[2019-08-27 16:52:32.357][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655fd0-16c0ad4e24c5c6f6966a0736","id":"4eb8c192bf5cf6fc","start_time":1566924752.354,"end_time":1566924752.357,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}},"fault":true,"subsegments":[{"id":"98a7e7dd8789bb76","start_time":1566924752.354,"end_time":1566924752.357,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}}}]}
[2019-08-27 16:53:53.581][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d656021-e4f02793a18722535759d4a7","id":"6bc2b28fe7518b0d","start_time":1566924833.58,"end_time":1566924833.581,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}},"fault":true,"subsegments":[{"id":"381e6f3f2c235fce","start_time":1566924833.58,"end_time":1566924833.581,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}}}]}
[2019-08-27 16:53:55.674][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d656023-7155c0d71b9faa5f5e07cd48","id":"3976d6afae0419ae","start_time":1566924835.674,"end_time":1566924835.674,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorteller.appmesh-demo:9080/","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":0}},"error":true,"subsegments":[{"id":"9c4b68c2d75ee1f0","start_time":1566924835.674,"end_time":1566924835.674,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorteller.appmesh-demo:9080/","method":"GET","user_agent":"Go-http-client/1.1"},"response":{"status":404,"content_length":0}}}]}
[2019-08-27 16:53:55.676][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d656023-06247d57018f3ae88638801c","id":"7f707cb31ffec53b","start_time":1566924835.674,"end_time":1566924835.676,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}},"fault":true,"subsegments":[{"id":"b718d2a93aea3843","start_time":1566924835.674,"end_time":1566924835.676,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}}}]}
[2019-08-27 16:54:46.993][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:54:46.997][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:54:47.000][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:54:47.001][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:54:47.005][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:54:47.005][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:54:47.006][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:55:17.377][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:55:17.377][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:55:17.377][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:55:17.378][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:55:17.515][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:56:15.101][000025][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d655f83-f591df891429b9da55021758","id":"64f1b44f6c8bb2f3","start_time":1566924675.096,"end_time":1566924975.101,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","method":"POST","user_agent":"grpc-go/1.13.0"},"response":{"status":200,"content_length":0}},"subsegments":[{"id":"fcd63c72e1565330","start_time":1566924675.096,"end_time":1566924975.101,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/grpc.reflection.v1alpha.ServerReflection/ServerReflectionInfo","method":"POST","user_agent":"grpc-go/1.13.0"},"response":{"status":200,"content_length":0}}}]}
[2019-08-27 16:56:20.323][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:56:20.327][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:56:20.330][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:56:20.331][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:56:20.335][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:56:20.335][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:56:20.336][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:56:45.453][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:56:45.453][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:56:45.453][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:56:45.453][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:56:45.457][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:56:51.633][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:56:51.637][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:56:51.641][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:56:51.641][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:56:51.646][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:56:51.646][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:56:51.647][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:57:16.283][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:57:16.284][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:57:16.284][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:57:16.284][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:57:16.289][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:57:22.133][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 starting warming
[2019-08-27 16:57:22.137][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 starting warming
[2019-08-27 16:57:22.153][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 starting warming
[2019-08-27 16:57:22.154][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080
[2019-08-27 16:57:22.159][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080 complete
[2019-08-27 16:57:22.159][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080 complete
[2019-08-27 16:57:22.160][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080 complete
[2019-08-27 16:57:47.066][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:501] add/update cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 starting warming
[2019-08-27 16:57:47.066][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-black-appmesh-demo_http_9080
[2019-08-27 16:57:47.066][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-blue-appmesh-demo_http_9080
[2019-08-27 16:57:47.066][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:554] removing cluster cds_egress_color-mesh_colorteller-appmesh-demo_http_9080
[2019-08-27 16:57:47.070][000001][info][upstream] [source/common/upstream/cluster_manager_impl.cc:513] warming cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080 complete
[2019-08-27 16:58:07.930][000026][info][tracing] [source/extensions/tracers/xray/xray_core_types.cc:295] xray generated segments document: {"format":"json","version":1}
{"trace_id":"1-5d65611f-c568eba00bb1f2d892fe0662","id":"a2d82a7c05f56a00","start_time":1566925087.927,"end_time":1566925087.93,"name":"color-mesh/colorgateway-appmesh-demo","origin":"AWS::AppMesh::Proxy","aws":{"app_mesh":{"mesh_name":"color-mesh","virtual_node_name":"colorgateway-appmesh-demo"}},"http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}},"fault":true,"subsegments":[{"id":"e6a675bd7b3fc26e","start_time":1566925087.927,"end_time":1566925087.93,"namespace":"remote","name":"color-mesh/colorgateway-appmesh-demo","http":{"request":{"url":"http://colorgateway.appmesh-demo.svc.cluster.local:9080/color","method":"GET","user_agent":"curl/7.35.0"},"response":{"status":500,"content_length":22}}}]}
from aws-app-mesh-examples.
Y0Username
commented on May 20, 2024
@jicowan Envoy seems to removing app clusters: removing cluster cds_egress_color-mesh_colorgateway-appmesh-demo_http_9080.
Can you paste the logs for your colorgateway and colorteller? Are they healthy and server comes up?
from aws-app-mesh-examples.
Related Issues (20)
- [BUG] Enoy Proxy container fails to initialize while trying to create an app mesh using the dj-app HOT 2
- [BUG] App Mesh Envoy version 1.17+ does not work with SPIRE for mTLS HOT 1
- Having 502 or https redirect loop (hard-mesh-time) HOT 4
- Walkthrough examples to deploy on Graviton instances
- [BUG] Fix howto-k8s-alb walkthrough HOT 1
- [BUG] Have a public ECR repo from common images generated in walkthrough
- Samples using deprecated APPMESH_VIRTUAL_NODE_NAME HOT 1
- Provide mTLS walkthrough for east-west traffic
- [BUG] Custom Envoy image does not startup in mTLS example
- [BUG] TLS examples don't work with default bash IFS HOT 1
- [QUESTION] Why an Envoy Service infront of Services with Envoy Sidecar? Why an NLB?
- "/bin/sh: yum: command not found" while implementing "howto-mutual-tls-file-provided" in the walkthroughs HOT 1
- [BUG] Can not remove the ending / in k8s manifest HOT 2
- [BUG] INSTANCE_PROFILE_NAME is empty because no prefix matches
- [BUG] HOT 5
- Grpc Load Balancing is not working
- [BUG] Internal SMTP connection fails with App Mesh HOT 1
- [BUG] wrktool in Circuit Breaker doesn't work successfully
- [BUG] Metrics related to Outbound Traffic are not exported after enabling the appmesh metric extension HOT 1
- [BUG] aws-app-mesh-examples/walkthroughs/howto-k8s-mtls-sds-based/v1beta2/manifest.yaml.template has issue
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aws-app-mesh-examples.