Finishing MQTT integration

[emoncms]

Finishing MQTT integration

[janecker]

Good point, did not think about that. That could be good solution and I played around with an new php 7 based container running the mqtt script. That seems to be working. I will try to polish this version in the upcoming days.

[JumpMaster]

I'm personally running emoncms in 3 (or maybe 4) docker containers.

• emoncms web app
• mqtt server (shared with other system so not part of emoncms)
• MariaDB
• mqtt listener

I also followed the mantra of one process per container. It's a shame that both the mqtt listener and emoncms server both have a full copy of the code from GitHub.

I'll upload my code to GitHub later today.

It's a shame

[glynhudson]

as I understand it best practice would be to have a single container for each service/process

Yes, this is also what I understand to be best practice.

new container could be created that is added to the docker-compose. This would require an additional repository with its own Dockerfile

No, additional repository would not be required. Just a sub-folder in this repo. We already use docker-compose to bring up multiple containers e.g. web, db, and redis. Its just that for db and redis we just the off-the-shelf image without any customisation therefore dockerfile is not needed. See: https://nickjanetakis.com/blog/docker-tip-10-project-structure-with-multiple-dockerfiles-and-docker-compose

I think creating another php (not php-apache) container is the best way to run the MQTT service. The user could then choose to start this container or not depending if MQTT input is required. Although it would't use much resources.

[mattiarossi]

Hi, I am picking up this enhancement request, as I am redeploying my monolithic emoncms install and would like to move it to docker composer.
I am ok with building an additional mqtt container if needed, but I am not sure how to tackle the phpmqtt_input task.
In theory it should go into another container, using php-cli as base image, but it would need to share the data volumes with the main web container for access to the timeseries data, and the locking could break when the underlying volumes are mounted on an nfs share (guess, my case)
Another possibility would be to spawn it in the existing web container, but it would break the 'one purpose per container' pattern
I think I will go the separate container, shared volumes way, but comments are welcome
Regards,
Mattia

[glynhudson]

Hi Mattia,

Thanks for your comment. After considering this issue we have created an all-in-on Emoncms docker based on the ubuntu image. This simplifies everything, even though maybe this is not docker best practice I think it's the way forward for deployment. Here is a development repo, it's not yet on docker hub. Please let us know how you get on with it: https://github.com/TrystanLea/emoncms-docker

[mattiarossi]

Thanks Glyn,
I agree the monolithic solution simplifies things, that's where I am coming from, a single container with emoncms, mosquitto and redis, I don't think I want to keep going that way, for me emoncms is part of a broader home automation setup that includes openhab,node-red,influxdb, grafana, you name it ... and I am reworking all of it to use docker/docker-compose. Do I have gone the separate container way with shared volumes (mindful of all risks involved in flock() potentially misbehaving when accessing time series data).

[haniham]

Hey guys - nearly 1 Year has passed - have you meanwhile figured out a solution, cause i would like to use it (-;

[TrystanLea]

I've got this working here https://github.com/emoncms/emoncms-docker/pull/27/files
Does that match your approach @JumpMaster ? any tips to improve?

[jamesfidell]

I came to the conclusion that the web app and the script to subscript/publish to MQTT are so tightly coupled in terms of shared code and configuration that there's a fair case to be made for running them in the same container. I have therefore gone the route of using supervisord to start both Apache and the script. It seems to work fine so far, modulo the issues with the script and Redis.

[TrystanLea]

Thanks @jamesfidell I'd be interested to hear more about your solution, would you be able to make a pull request with your implementation?

[jamesfidell]

Sure, I'll try to sort something out. The files are in a bit of a mess at the moment as whilst I am using a Docker container for emoncms, I'm not using the database container (already have a database running elsewhere). I'll clone a fresh copy and tidy it all up for a PR once I have my new templates working.