Does the Zigbee2MQTT server and MQTT Broker need to be on the same host?

[lyance]

Does the Zigbee2MQTT server and MQTT Broker need to be on the same host?

[andreypopov]

• [email protected]

[stefanheule]

I found the issue: The server config page has a "Use TLS" checkbox, which is ignored by the code. Specifically, server.js line 52 should also check if TLS is enabled. Right now TLS is always used, even if the user does not enable it.

[oxivanisher]

Also with the newest release (0.6.0) this problem still exists. Can anyone with some JS node knowhow take a look at the code again? Please? ^^ @stefanheule <3

[oxivanisher]

You probably also have the problem, that it tries to reconnect all the time, but the connection is not getting up.

[...]
Aug 27 08:32:47 nodered Node-RED[1998]: 27 Aug 08:32:47 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Reconnect
Aug 27 08:32:47 nodered Node-RED[1998]: 27 Aug 08:32:47 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Close
Aug 27 08:32:48 nodered Node-RED[1998]: 27 Aug 08:32:48 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Reconnect
Aug 27 08:32:48 nodered Node-RED[1998]: 27 Aug 08:32:48 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Close
Aug 27 08:32:49 nodered Node-RED[1998]: 27 Aug 08:32:49 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Reconnect
Aug 27 08:32:49 nodered Node-RED[1998]: 27 Aug 08:32:49 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Close
Aug 27 08:32:50 nodered Node-RED[1998]: 27 Aug 08:32:50 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Reconnect
Aug 27 08:32:50 nodered Node-RED[1998]: 27 Aug 08:32:50 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Close
Aug 27 08:32:51 nodered Node-RED[1998]: 27 Aug 08:32:51 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Reconnect
Aug 27 08:32:51 nodered Node-RED[1998]: 27 Aug 08:32:51 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Close
Aug 27 08:32:52 nodered Node-RED[1998]: 27 Aug 08:32:52 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Reconnect
Aug 27 08:32:52 nodered Node-RED[1998]: 27 Aug 08:32:52 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Close
Aug 27 08:32:53 nodered Node-RED[1998]: 27 Aug 08:32:53 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Reconnect
Aug 27 08:32:53 nodered Node-RED[1998]: 27 Aug 08:32:53 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Close
[..]

I am also working with eclipse-mosquitto an I am unable to get node-red connecting. My MQTT server was also on a different host, but even if I try it with a local container, it is not working. I also tried disabling user authentication and played with all options for an evening, without any success. The mosquitto log shows corresponding errors:

[..]
mqtt-bridge    | 1598510077: New connection from 172.18.0.1 on port 1883.
mqtt-bridge    | 1598510077: Socket error on client <unknown>, disconnecting.
mqtt-bridge    | 1598510078: New connection from 172.18.0.1 on port 1883.
mqtt-bridge    | 1598510078: Socket error on client <unknown>, disconnecting.
mqtt-bridge    | 1598510078: New connection from 172.18.0.1 on port 1883.
mqtt-bridge    | 1598510078: Socket error on client <unknown>, disconnecting.
mqtt-bridge    | 1598510079: New connection from 172.18.0.1 on port 1883.
mqtt-bridge    | 1598510079: Socket error on client <unknown>, disconnecting.
mqtt-bridge    | 1598510079: New connection from 172.18.0.1 on port 1883.
mqtt-bridge    | 1598510079: Socket error on client <unknown>, disconnecting.
[..]

But sadly, there is no more information on what is going on exactly. Even enabling debug for mosquitto does not log more infos.

My guess is, that there is a problem with the used MQTT library. The built-in MQTT nodes are working without any problems. I am no "node guy", but is it possible that the this version

node-red-contrib-zigbee2mqtt/package.json

Line 12 in 87c3c69

"mqtt": "^2.18.8",

is very outdated? When I google and search the node mqtt package, the current Version of mqtt would be 4.2.1.

[oxivanisher]

Sorry, @andreypopov this is somewhat hidden here, that's why I am mentioning you.

[andreypopov]

do you use docker ?

[oxivanisher]

Yes for MQTT, both instances where docker (eclipse-mosquitto, tried with 1.5 and 1.6), but node red is installed native on a RPI 3 (bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered).

[andreypopov]

did you open ports in docker?

[oxivanisher]

Yes, other MQTT nodes from node red (and many, many home IOT devices) work without a problem. And I see the dropped connection in the MQTT log. So networking problems are pretty much out of the picture.

[andreypopov]

I dont use docker, so I cant tell you what to do
As for subject of this issue:

Does the Zigbee2MQTT server and MQTT Broker need to be on the same host?

NO

My configuration.yaml looks like:

_

homeassistant: false
permit_join: false
mqtt:
base_topic: zigbee2mqtt
server: 'mqtt://192.168.0.2'
serial:
port: /dev/ttyMOD3
advanced:
rtscts: false
last_seen: epoch
log_directory: /var/log
devices:

_

[oxivanisher]

Which MQTT Broker are you using?

[andreypopov]

zigbee2mqtt:
mosquitto version 1.4.15 (build date Thu, 07 Jun 2018 19:59:46 +0300)
mosquitto is an MQTT v3.1.1/v3.1 broker.

pi4 with node-red:
mosquitto version 1.5.7
mosquitto is an MQTT v3.1.1 broker.

[oxivanisher]

Can you post your mosquitto config?

[andreypopov]

default config, I didnt install it manually. Everything is out of the box

[oxivanisher]

so you use anonymous for auth?

[andreypopov]

yes, no user

[oxivanisher]

Ok, I will try some more things with that information. but as i mentioned, I tried that in general already.
How xomlicated would it be for you to update to a newer mqtt dependency?

[andreypopov]

you can update and check
mqtt has backward compatibility, will not affect

[oxivanisher]

what is your node red version?

[andreypopov]

always the latest

[oxivanisher]

After updating mqtt to the current version 4.2.1, I get more info on the error:

Aug 28 11:11:05 nodered Node-RED[26345]: 28 Aug 11:11:05 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Close
Aug 28 11:11:06 nodered Node-RED[26345]: 28 Aug 11:11:06 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Reconnect
Aug 28 11:11:06 nodered Node-RED[26345]: 28 Aug 11:11:06 - [info] [zigbee2mqtt-server:IOT MQTT] MQTT Error
Aug 28 11:11:06 nodered Node-RED[26345]: Error: read ECONNRESET
Aug 28 11:11:06 nodered Node-RED[26345]:     at TLSWrap.onStreamRead (internal/stream_base_commons.js:205:27) {
Aug 28 11:11:06 nodered Node-RED[26345]:   errno: 'ECONNRESET',
Aug 28 11:11:06 nodered Node-RED[26345]:   code: 'ECONNRESET',
Aug 28 11:11:06 nodered Node-RED[26345]:   syscall: 'read'
Aug 28 11:11:06 nodered Node-RED[26345]: }

... I don't know why TLS is involved, since I do not use TLS. This might be a node or node red bug. Does yours still work when you update your mqtt dependency to 4.2.1?

Edit: Also, what is your node version?

[andreypopov]

node: 12
i didnt try to update

I suppose, you have docker settings issue.

[oxivanisher]

No, I don't think so ... why should it be docker?
All other things are able to talk to the MQTT containers. Other node red plugins, several different arduino or esp8266 devices, and so on.

[oxivanisher]

So I updated some dependenciesto the current version (requests to 2.88.2 and eventsource to 1.0.7) and also updated all node packages to the current versions npm update -g, but the error still persists.

FYI: The node page for requests tells us in a big red banner, it is deprecated and no longer maintainet: https://www.npmjs.com/package/request

But I managed to get it connecting. I configured the local "test mqtt container" with TLS based on https://mcuoneclipse.com/2017/04/14/enable-secure-communication-with-tls-and-the-mosquitto-broker/ and then it was able to connect. So somehow if node is using TLS (as said before, only for node-red-contrib-zigbee2mqtt, the other mqtt connections are still "normal") it works.

[oxivanisher]

@andreypopov did you see the message above from @stefanheule ?
it would be really nice if this problem could be solved.

[oxivanisher]

This was probably not the only thing broken with that. It's still only working when connecting to the TLS broker for me.

[ptath]

same problem

no TLS, no auth, no docker, local machine, with node-red-contrib-zigbee2mqtt enabled default mqtt stops working, "red dot"

[andreypopov]

is not true for me, both work fine

[ptath]

can you please show both nodes configs?

[oxivanisher]

For me, its like this:

node-red-contrib-zigbee2mqtt working

... with MQTT bridge in local container, which itself connects to the "real" MQTT server.

With this cfg in the MQTT bridge on localhost:

connection myowndomain
address 192.168.199.100:1883
start_type automatic
try_private false
cleansession true
port 8883
cafile /mosquitto/certs/m2mqtt_ca.crt
certfile /mosquitto/certs/m2mqtt_srv.crt
keyfile /mosquitto/certs/m2mqtt_srv.key
#tls_version tlsv1

# Authentication
remote_username myowniot
remote_password myownpasswordandnotpastingithere

# Topics
topic zigbee2mqtt/# out 0
topic zigbee2mqtt/# in 0

node-red-contrib-zigbee2mqtt not working

... connecting directly to the "real" MQTT sever, also a docker container but on another host

persistence true
persistence_location /mosquitto/data/
log_timestamp_format %Y-%m-%dT%H:%M:%S
log_dest stdout
connection_messages true
password_file /mosquitto/config/mosquitto.passwd

[oxivanisher]

In node red the current version is 0.6.2 available, but I am still having the same problem.

[omarecd]

Hi. I can't seem to get this to work. The Zigbee2MQTT bridge node always has a RED DOT at the bottom which probably means it can't connect. I have other nodes using the basic MQTT node-red and are able to connect to the Mosquitto broker. My Mosquitto broker is on a different machine (IP) from the Raspberry-Pi running the Zigbee2MQTT server. So for the Host IP configuration, which IP (Mosquitto or RaspPi) should I put there? Any way I tried both both none of them connected. Let me know what other info you need.

Thank you.

Hi @lyance
No, you can use any MQTT Broker, no matter where it is :)
Just enter the MQTT address (IP or URL), user and password if needed and that's all.

[oxivanisher]

I don't think this is correct, @omarecd . As I mentioned in all my comments above, my connection issues still persists.
This is why I removed node-red-contrib-zigbee2mqtt completely due to the frustration that this issue here is just ignored, and replaced it with a JS function in NodeRed. This issue is now open since more than six months and the problem with TLS connections still persists.

[omarecd]

I don't think this is correct, @omarecd . As I mentioned in all my comments above, my connection issues still persists.
This is why I removed node-red-contrib-zigbee2mqtt completely due to the frustration that this issue here is just ignored, and replaced it with a JS function in NodeRed. This issue is now open since more than six months and the problem with TLS connections still persists.

Hi @oxivanisher !

To the question indicated in the subject "Does the Zigbee2MQTT server and MQTT Broker need to be on the same host?" I can reply with confidence that the answer is no :) I'm Very confident about it. Why? Basically because for me Zigbee2MQTT server and MQTT Broker are not on the same host and everything works fine.

Indeed, there is a little bit of extra delay if you need to jump from machine to machine (instead of just doing a simple localhost) but, does it work? For sure :)

Regarding the Node-RED extension for ZigBee2MQTT I tried to use that and then Node-RED was not working anymore, a lot of nodes like MQTT, etc were unusable so (as I am not an Node-RED expert) I just decided to reinstall Node-RED and do not use the ZigBee2MQTT "extension". Everything I do it by hand and so far it works fine :)

I hope it helps.

[oxivanisher]

Hi @omarecd

I did not open this issue, I just found it, because I had the same (at least initial) behavior as @lyance. If he has the same problem as me I don't know, he/she never wrote an update to the situation. But obviously others have/had the problem too.
And to my very extensive debugging, please read the whole thread to understand it, it still exists.

To the question indicated in the subject "Does the Zigbee2MQTT server and MQTT Broker need to be on the same host?" I can reply with confidence that the answer is no :) I'm Very confident about it. Why? Basically because for me Zigbee2MQTT server and MQTT Broker are not on the same host and everything works fine.

Indeed, there is a little bit of extra delay if you need to jump from machine to machine (instead of just doing a simple localhost) but, does it work? For sure :)

But sure: Does the Zigbee2MQTT server and MQTT Broker need to be on the same host?
No, as long as you don't run into the same problem others do, described in this very thread. ;)

Regarding the Node-RED extension for ZigBee2MQTT I tried to use that and then Node-RED was not working anymore, a lot of nodes like MQTT, etc were unusable so (as I am not an Node-RED expert) I just decided to reinstall Node-RED and do not use the ZigBee2MQTT "extension". Everything I do it by hand and so far it works fine :)

Good for you. :)

[andreypopov]

try 2.0.0 version