Question: Micro Service Discovery over WAN using Consul

[ramukima]

Question: Micro Service Discovery over WAN using Consul

[ramukima]

I see a warning like below on the consul server running on machine A. Not sure if that can be the cause.

2016/12/03 19:51:54 [WARN] Service name "foo.example" will not be discoverable via DNS due to invalid characters. Valid characters include all alpha-numerics and dashes.

[asim]

The warning is non-issue but if you want to avoid that then you need to specify a service name that matches consul's requirements.

The timeout error mentioned above could be any number of reasons. micro list services and micro get service foo.example only query consul while micro query makes a request to the service itself. It's likely that you have dead entries in consul and its attempting to request one of these. To avoid such an issue you should use the register ttl and interval options. Example here https://github.com/micro/micro/blob/master/examples/greeter/server/main.go#L24L25.

[asim]

The other likely issue could be that your other service is not accessible or registering a local address.

[ramukima]

Thanks for the reply, will try to define the TTL as well as refresh consul completely. I will close the issue once I have it working if that is OK.

[asim]

That's fine, please test and respond asap if looking to leave the issue open until resolved. Lingering issues unrelated to the code itself is not ideal.

[ramukima]

No luck, after trying the following -

1. Add TTL per your feedback
2. Cleared consul data and started with clean slate on both machines A and B
3. The server is accessible using public IP, may be the RPC port used for communication is being blocked somehow (checking, not sure) - Can it relate to proxy_address ?

[asim]

Please ensure your service is binding to the publicly accessible IP and attempt to manually connect to it. Are these two machines accessible on a private network? Are they two machines in completely different networks with public interfaces? What address is actually being registered in the service registry? Is that address accessible?

Also note, this is a machine to machine configuration issue rather than Micro itself. You need to ensure that both machines can talk to each other and the address you're binding to or advertising is available from other machines.

[ramukima]

Both nodes of the consul cluster reside on public facing network and the node2node communication is over WAN gossip. Regarding "What address is actually being registered in the service registry? Is that address accessible?" - Looks like this is the issue -

#micro stats foo.example
service  foo.example

version latest

node		address:port		started	uptime	memory	threads	gc
foo.example-19129b57-bc4a-11e6-b092-0cc47ab587da		x.x.x.x:34588					0

In the result above, I see that x.x.x.x is actually used from a private network and not from the public network.

I agree that it is non-issue for the micro toolkit.

[ramukima]

I am reopening this as a question:

I have a Consul Cluster (Server) in a DC say SFO and another Consul Cluster (Server) in another DC say BOS. Both these are joined over wan (consul join -wan ). I can see them connected fine.

I then start a micro service in SFO with registry_address as the the address of SFO consul server.
Using 'http://bos-server:8500/v1/catalog/services?dc=sfo', I can see that the micro service was registered to consul in SFO dc from BOS dc.

When I run 'micro list services' in SFO, I see my micro service listed.
When I run 'micro list services' in BOS, I do not see my micro service listed.

Is there a way I can list/call services running in SFO dc from the server in BOS dc ? Apologies if this has been answered before.

[asim]

I'm sorry but this is a consul specific issue. Micro only abstract consul away as a registry, it does not control the underlying functionality of consul. micro list services just uses consul.Catalog().Services(nil). You may need to talk to hashicorp about how WAN clustering works.

[pawmart]

FYI i have similar problems on NATS, so that's not only consul specific...

[ZiyunYang]

@pawmart Hi, I meet same problems on NATs, have you found any solutions?