emoncms/emoncms-docker

Emoncms docker container(s)

Emoncms docker

An easy way to deploy a simple Emoncms installation.

Note: This docker installation is not quite a complete Emoncms installation. In general we recommend building an Emoncms installation using our EmonScripts installation script on a Debian/Ubuntu/RaspberryPi based system, this said this docker image does provide a useful alternative approach to get a simple but functional emoncms installation up and running.

Latest image hosted on docker hub: openenergymonitor/emoncms:latest

Quickstart

docker pull openenergymonitor/emoncms
git clone https://github.com/emoncms/emoncms-docker
cd emoncms-docker
docker-compose up

That's it! Emoncms should now be running: http://localhost:8080

---

Install Docker

• Offical Docker install guide

If running Linux Docker-compose will also need to be installed separately. Docker-toolbox on Mac and Windows includes Docker-compose

• Offical Docker Compose install guide

If running on Linux highly recommended to create Docker group to avoid having to use sudo with Docker commands:

sudo groupadd docker
sudo usermod -aG docker $USER
docker run hello-world

After restarting terminal (logout & log back in), you should now be able to docker run hello-world without sudo.

Setup dev environment

If you want to edit the emoncms file (dev) it's best to clone them externally to docker then mount volume into the container:

Uncomment in docker-compose.override.yml:

volumes:
  ##mount emoncms files from local FS for dev
  - ./emoncms:/var/www/emoncms

Then clone the repos into ./emoncms

./bin/setup_dev_repositories
docker-compose pull
docker-compose up

If you get an error bind: address already in use, this means there is already a process on the host machine listening on port 8080. You can check what processes are listening on ports by running sudo netstat -plnt. There are two options, either change the emoncms web container port to use a different port then rebuild the container or kill the process currently running on the host machine using the same port.

---

Build Emoncms Docker Containers

Git Clone

Note: If Emoncms Docker being used for production / testing (i.e modifying the Emoncms files at run-time is not required) there is no need to clone emoncms core & modules since by default the Emoncms git master branch is cloned when the containers are built (see Dockerfile), this cloned 'snapshot' is then overwritten by the Emoncms files mounted from local file-system when running development docker-compose (default)

Run the following script to clone the emoncms repository along with the dashboard and graph modules:

./bin/setup_dev_repositories

Further modules can be found in the emoncms git repo e.g. backup, WiFi etc.

The file structure should look like:

+-- emoncms-docker
¦   +-- Dockerfile
¦   +-- docker-compose.yml
+-- emoncms
¦   +-- <emoncms-core files>
¦   +-- e.g index.php
¦   +-- Modules
¦       +-- <core-modules> e.g. admin, feed, input
¦       +-- <optional-modules> e.g dashboard

Customise config

MYSQL Database Credentials

For development the default settings in default.docker.env are used. For production a .env file should be created with secure database Credentials. See Production setup info below.

PHP Config

Edit config/php.ini to add custom php settings e.g. timezone (default Europe)

Build / update Docker container

Required on first run or if Dockerfile or Docker-compose.yml are changed:

docker-compose build

Start Emoncms containers

Start as foreground service:

docker-compose up

Stop with [CTRL + c]

That's it! Emoncms should now be runnning in Docker container, browse to http://localhost:8080

Start as background service:

docker-compose up -d

Stop the containers

docker-compose down 

Stop the containers and delete the volumes (remove database and all stored data)

docker-compose down -v

Docker compose up will start two containers:

1. emon_web which is based on the official Docker PHP-Apache image
2. emon_db which is based on the official Docker MySQL image

---

---

How Docker Compose works...

Infomation about how docker-compose working in Emoncms Docker.

Development Vs Production

There are three docker compose files:

1. docker-compose.yml
2. docker-compose.override.yml
3. docker-compose.prod.yml

The first docker-compose.yml defines the base config; things that are common to both development and production.

The second file docker-compose.override.yml defines additional things for development environment e.g. use default.docker-env environment variables and mount emoncms files from local file-system instead of copy into container.

The third file docker-compose.prod.yml defines production specific setup e.g. expose port 80 and copy in files instead of mount. The production docker-compose file is docker-compose.prod.yml instead of docker-compose.override.yml when the Emoncms Docker contains are ran as 'production'

This setup is based on the recommended Docker method, see Docker Multiple Compose Files Docs.

Development

The development enviroment with the base + override compose is used by default e.g:

docker-compose up

Production

To run the production compose setup run:

docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

Files, storage & database info

Infomation about how files and databases work in Emoncms Docker.

Files

By default when running the development compose enviroment (see above) Emoncms files are mounted from the host file system in the emon_web Docker container at startup. This is desirable for dev. For production / deployment / clean testing run production docker compose (see above). This will copy emoncms files into the docker container on first run. Any changes made to the files inside the container will be lost when the container is stopped.

Storage

Storage for feed engines e.g. var/lib/phpfiwa are mounted as persistent Docker file volumes e.g.emon-phpfiwa. Data stored in these folders is persistent if the container is stopped / started but cannot be accessed outside of the container. See below for how to list and remove docker volumes.

Database

Database storage /var/lib/mysql/data is mounted as persistent Docker volumes e.g.emon-db-data. Database data is persistent if the container is stopped / started but cannot be accessed outside of the container.

---

Useful Docker commands

List running containers

docker ps

List all containsers

docker ps -a

Stop / kill all running containers: mysql e.g. emon_web, emon_db

	docker rm $(docker ps -a -q)

List all base images:

docker image ls

Remove all images:

e.g. Base images: php-apache, mysql, Ubuntu pulled from Dockerhub

docker rmi $(docker images -q)

List docker volumes (where data is stored e.g database)

docker volume ls

Remove single or all docker volumes

docker volume rm <name_of_volume>
docker volume prune

Attach a shell to a running container:

docker exec -it emoncms-docker_web_1 /bin/bash

---

Pushing to docker hub

From: https://docs.docker.com/docker-hub/repos/

docker login --username=yourhubusername [email protected]
docker tag openenergymonitor/emoncms:<tag-name>
docker push openenergymonitor/emoncms:<tag-name>

Tag name should be the Emoncms version e.g 10.x.x

Also push the latest version using latest tag

docker tag openenergymonitor/emoncms:latest
docker tag openenergymonitor/emoncms:latest

emoncms-docker's Issues

when i run docker-compose pull it gives error

Error -> Unsupported config option for services service: 'db'

Using emoncms-docker.

Continued from emoncms/emoncms#540

(I hope you don't mind me answering here, I can move it to the forum if you prefer)

no I haven't yet tried emoncms-docker, I am not familiar with docker and how it works, I would like to try it, but not on my live server, yet!

I understand from very basic knowledge that the concept behind docker is to compartmentalize, but I would need to run it on a test instance first, until I was familiar and had confidence in it. reading the guide it is still unclear to me how it is structured, for example it says

"We have taken a multi-container approach with php-apache running in one container and the MYSQL database running in another. The containers are linked using docker-compose."

Does that mean each of the individual emoncms instances each have php-apache and mysql containers? or do they share those same 2 containers? are those containers in addition to any existing live LAMP? Is the data and tables kept separate for each instance? How does traffic get directed? I currently use "https://myserver" rather than "https://myserver/emoncms" so how could that work?

Plus, for a test setup how would I configure port forwarding? could I redirect all non-live traffic from my live server to a docker test server until I am happy to combine them?

As much as I think it COULD be a great tool, I do not know that for sure and cannot tell from the guide if the "in minutes" installation would suit my needs unchanged and unfortunately at the moment I do not have the time for a long steep learning curve in Docker to adapt it if it doesn't, as you know my application is quite different from the single home emonPi user.

I think I need more Pi's setup, I should really setup a good VM environment, but it's the same issue of finding the time to research, audition and learn about VM setups.

I have also replied on the https://community.openenergymonitor.org/t/emoncms-dev-in-a-browser-using-cloud9-ide/1756 forum thread, mainly in response to

"Off-topic: I woul really like to create a testing enviroment as part of the emoncms-docker which automaticlly loads in a decent amount of test data and some complex dashboards at launch for testing and also a service to hit the API hard with a large volume of incoming data. However this is not the righ location to discuss this...."

Modules/group/group_model.php missing in the Docker image ?

Hi!

I'm getting the following errors while trying to edit or view dashboards.

Notice: Undefined index: groups in /var/www/html/Modules/graph/widget/graph_widget.php on line 21

Warning: Invalid argument supplied for foreach() in /var/www/html/Modules/graph/widget/graph_widget.php on line 21

I looked at the faulty piece of code

if (group_module_installed()) {
    require_once "Modules/group/group_model.php";
    $group = new Group($mysqli, null, null, null, null);
}
else
    $group = null;

.......

function group_module_installed() {
    global $mysqli;
    $result = $mysqli->query("SHOW TABLES LIKE 'groups'");
    if ($result->num_rows > 0)
        return true;
    else
        false;
}

Based on the content of my database, the query in group_module_installed() returns 0 rows and the function returns true.
However, my image doesn't seem to have group_model.php anywhere, an error occurs and $group is not defined when used later on, causing the message.

Is it a bug or am I doing something wrong ? :-/

Note that the dashboard and widgets are working perfectly fine despite that error message!

Docker Hub not up to date

The image needs to be pushed to Docker Hub as it isn't an automated build. The Hub version is a few commits out of date.

Error on registration attempt: "Database error, you may need to run database update"

I just followed the instructions for setting up for dev work, which seemed to go fine, docker-compose brings everything up etc. However, when I try to register a new user, I get the error:

Database error, you may need to run database update

Any pointers on what I might be doing wrong would be helpful!

Restore function not working

On the latest v2 branch the restore function within backup module is not working.
By default it simply refuses to upload the file. I figured we were missing php.ini upload limits.

Within php subfolder where yoy have the Dockerfile, i created the config/php.ini file

php/Dockerfile has to add the COPY routine:
`COPY ./config/php.ini /usr/local/etc/php/

then
docker-compose build

After uploading the archive

Warning: move_uploaded_file(BACKUP_LOCATION/uploads/emoncms-backup-2020-10-05.tar.gz): failed to open stream: No such file or directory in /opt/emoncms/backup/backup-module/backup_controller.php on line 120

Warning: move_uploaded_file(): Unable to move '/tmp/phpdWe3ZM' to 'BACKUP_LOCATION/uploads/emoncms-backup-2020-10-05.tar.gz' in /opt/emoncms/backup/backup-module/backup_controller.php on line 120

Script emoncms-import.sh also doesn't appear to be configured/docker ready.

root@fb01b90d0e64:/opt/emoncms/backup# ./emoncms-import.sh 
=== Emoncms import start ===
2020-10-05-10:10:31
Backup module version:
cat: ./backup/module.json: No such file or directory
EUID: 0
Reading ./config.cfg....
Location of data databases: DATABASE_PATH
Location of emonhub.conf: EMONHUB_CONFIG_PATH
Location of Emoncms: EMONCMS_LOCATION
Backup destination: BACKUP_LOCATION
Backup source path: BACKUP_SOURCE_PATH
Starting import from BACKUP_SOURCE_PATH to BACKUP_LOCATION...
./emoncms-import.sh: line 26: systemctl: command not found
./emoncms-import.sh: line 27: systemctl: command not found
./emoncms-import.sh: line 28: systemctl: command not found
./emoncms-import.sh: line 29: systemctl: command not found
old image
Error: BACKUP_SOURCE_PATH does not exist, nothing to import

How can we import backups for using docker containers?

Thanks

Outdated quickstart instructions

The quickstart instructions at https://hub.docker.com/r/openenergymonitor/emoncms/ are out of date resulting in a 403.

The additional step to run bin/setup_dev_repositories from https://github.com/emoncms/emoncms-docker#quick-start is required.

error apache

this is the error i get when i try to run the openenergymonitor/emoncms
system:
intel NUC I3

OS | Resin OS 2.3.0+rev1
Kernel version | 4.8.17-yocto-standard

AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.4. Set the 'ServerName' directive globally to suppress this message
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.4. Set the 'ServerName' directive globally to suppress this message
[Sun Dec 02 12:15:11.689003 2018] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.10 (Debian) PHP/5.6.31 configured -- resuming normal operations
[Sun Dec 02 12:15:11.689029 2018] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'

use of deprecated Links in docker-compose file

I think Links should not be used anymore - https://docs.docker.com/network/links/

The containers should use a defined network, and their name should match the environnement variables *_HOST

I have had issues because of this.

Thanks

Volume for local dev environment point the wrong place

I just spent a couple of time to understand why changing files on dev environnement just was doing anything. I even thought I was pointing on the wrong server.

There is a bug in documentation and in docker-compose.override.yml file

Volumes local mount for dev

should be

volumes:
  ##mount emoncms files from local FS for dev
  - ./emoncms:/var/www/emoncms

and not

volumes:
  ##mount emoncms files from local FS for dev
  - ./emoncms:/var/www/html

if it may avoid headaches to other, can't believe I'm the only one using this mode :-)

Sending data to docker emoncms from openevse

The docker image works well on Windows 10. I was able to get it running and create a login and start configuring my dashboards in just a few minutes. Good job!

How exactly would you configure OpenEVSE for accessing the emoncms running on http://localhost:8080 ? I tried replacing localhost with the machine ip address - can browse to it fine.

In OpenEVSE I tried adding the Emoncms Server, Node and write-apikey info, but Connected: No prevails. Presumably using the docker image, would be one of the use cases for testing OpenEVSE to Emoncms.

Any advice would be appreciated. Thank you.

What branch is the most active/recent?

I see there is a v2 branch with MQTT support - should I be using this? Or is the master branch the most up-to-date?

I have got EmonCMS running by building my own image from the master branch but MQTT is not working and there is a type on the devices.git clone in Dockerfile - so unsure if it is actually in-use?

Database: Connection Refused

Receive the above issue during new implementation.. Any thoughts ? Thank you!!

few changes for running the container rpi4

Can you add some instructions for raspberry pi (rpi4) owners? To run the container, the following changes are required:

• change docker-compose.yml to use the armv7 flavor for mariadb tobi312/rpi-mariadb:latest_
• docker-compose build before docker-compose up to prevent web-1 from throwing exec error.

thanks for the great work!

fatal: Remote branch python3 not found in upstream origin

Building emonhub
Step 1/7 : FROM python:3.8-slim
---> 07ea617545cd
Step 2/7 : ARG EMONHUB_DIR
---> Using cache
---> f360cdd4fe32
Step 3/7 : RUN apt-get update && apt-get install -y git python3-serial python3-configobj python3-pip
---> Using cache
---> 33c77f32bd28
Step 4/7 : RUN pip3 install paho-mqtt requests configobj pyserial
---> Using cache
---> 8af4255a3c62
Step 5/7 : RUN git clone -b python3 https://github.com/openenergymonitor/emonhub.git ${EMONHUB_DIR}
---> Running in 5dbd9938e079
Cloning into '/opt/emonhub'...
fatal: Remote branch python3 not found in upstream origin

Setup CI to automate pushes to docker-hub

Would you be interested in setting up automation so that it automatically publishes to docker hub?

e.g. you tag a release from the emoncms repo and then that triggers a build on here to build the image, do some basic sanity checks and then pushes?

I can try and help with that if you're interested.

Multiple errors on image update

I was using 10.8.5 and spotted the update - updated image and left it 2 days.

Tried to login, and found (from the error messages) I can authenticate, but am receiving multiple errors about MQTT and then PHP errors. Reverted to 10.8.5 and all is well again. I don't appear to have suffered any data loss - but if there is an upgrade path from 10.8.5 to "latest" is it published anywhere?

Screenshot available if requested.

emoncms access

I installed on ubuntu 20.04 server in docker emoncms and I always get the error at access (with localhost:8080) -> Forbidden, you don't have the permission to access/ on this server (Apache 2.4.38 (Debian Server). I stopped apache on the ubuntu server already, stopped the firewall, changed port settings to 8070/70 but I always get the same issue. Before I used this setup on a Synology Nas and it worked... (with an older Emoncms Docker image).

during docker-compose up I get these messages:
root@ubuntu1:/etc/emoncms-docker# docker-compose up
Creating emoncms-docker_redis_1 ... done
Creating emoncms-docker_db_1 ... done
Creating emoncms-docker_web_1 ... done
Attaching to emoncms-docker_db_1, emoncms-docker_redis_1, emoncms-docker_web_1
db_1 | 2020-08-30 22:17:18+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.5+mariafocal started.
db_1 | 2020-08-30 22:17:18+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
db_1 | 2020-08-30 22:17:18+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.5+mariafocal started.
redis_1 | 1:C 30 Aug 2020 22:17:18.439 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1 | 1:C 30 Aug 2020 22:17:18.439 # Redis version=5.0.9, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1 | 1:C 30 Aug 2020 22:17:18.439 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
db_1 | 2020-08-30 22:17:19 0 [Note] mysqld (mysqld 10.5.5-MariaDB-1:10.5.5+mariafocal) starting as process 1 ...
redis_1 | 1:M 30 Aug 2020 22:17:18.442 * Running mode=standalone, port=6379.
redis_1 | 1:M 30 Aug 2020 22:17:18.442 # Server initialized
redis_1 | 1:M 30 Aug 2020 22:17:18.442 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1 | 1:M 30 Aug 2020 22:17:18.442 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Using Linux native AIO
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Uses event mutexes
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Number of pools: 1
redis_1 | 1:M 30 Aug 2020 22:17:18.443 * DB loaded from disk: 0.002 seconds
redis_1 | 1:M 30 Aug 2020 22:17:18.444 * Ready to accept connections
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Using SSE4.2 crc32 instructions
db_1 | 2020-08-30 22:17:19 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Completed initialization of buffer pool
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: 128 rollback segments are active.
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Creating shared tablespace for temporary tables
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: 10.5.5 started; log sequence number 45139; transaction id 21
db_1 | 2020-08-30 22:17:19 0 [Note] Plugin 'FEEDBACK' is disabled.
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
db_1 | 2020-08-30 22:17:19 0 [Note] InnoDB: Buffer pool(s) load completed at 200830 22:17:19
db_1 | 2020-08-30 22:17:19 0 [Note] Server socket created on IP: '::'.
db_1 | 2020-08-30 22:17:19 0 [Warning] 'proxies_priv' entry '@% root@3721b0d4f36c' ignored in --skip-name-resolve mode.
db_1 | 2020-08-30 22:17:19 0 [Note] Reading of all Master_info entries succeeded
db_1 | 2020-08-30 22:17:19 0 [Note] Added new Master_info '' to hash table
db_1 | 2020-08-30 22:17:19 0 [Note] mysqld: ready for connections.
db_1 | Version: '10.5.5-MariaDB-1:10.5.5+mariafocal' socket: '/run/mysqld/mysqld.sock' port: 3306 mariadb.org binary distribution
web_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.4. Set the 'ServerName' directive globally to suppress this message
web_1 | AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.18.0.4. Set the 'ServerName' directive globally to suppress this message
web_1 | [Sun Aug 30 22:17:20.308514 2020] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.38 (Debian) PHP/7.3.9 configured -- resuming normal operations
web_1 | [Sun Aug 30 22:17:20.308574 2020] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND'
web_1 | localhost:80 172.18.0.1 - - [30/Aug/2020:22:18:05 +0000] "GET / HTTP/1.1" 403 503 "-" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:80.0) Gecko/20100101 Firefox/80.0"

Any idea? Thank you very much for your help!

Permissions Error

You don't have permission to access / on this server.

Finishing MQTT integration

I wanted to start a small discussion how the MQTT integration could completed. For receiving inputs through docker, the mqtt_input script has to run in the background, which means an additional process is required. I am quite new to docker but as far as I understand it best practice would be to have a single container for each service/process. I could think of using multiple approaches for integrating the MQTT service:

1. A new container could be created that is added to the docker-compose. This would require an additional repository with its own Dockerfile. Additionally the emoncms files have to be cloned again or the files could be moved to there own volume that is re-used in the new mqtt service container.
2. The mqtt process could somehow be started in its own process on the startup of the emoncms web container in the background. This is a simple but very dirty way of doing it.
3. Maybe supervisord could be used to start multiple services (apache and the mqtt service). I am not 100% sure yet how this is done when the php-apache base image is used. This would not require an additional repository and the emoncms files could stay where they are.

I think it would be good if it would be possible to start the mqtt service only if mqtt is also enabled in the settings. I am not sure if this is possible with all three proposed variants. Any opinions on my proposed variants? Are there other, maybe better way to solve this issue?

PHP error

Hi, I'm running docker for Mac and I receive the following error after connecting to localhost:8080:
"settings.php file error

settings.php config file has new settings for this version. Copy default.settings.php to settings.php and modify the later.

homedir is not configured or directory does not exists, check settings: homedir

To fix, check that the settings are set in settings.php or try re-creating your settings.php file from default.settings.php template"

Connecting to the web container, there is no home dir and none of the docker files creates the home dir. In the default.settings.php where should the $homedir point to?

If i manually create the home dir in the container, then I am still left with the error:
"settings.php file error

settings.php config file has new settings for this version. Copy default.settings.php to settings.php and modify the later.

To fix, check that the settings are set in settings.php or try re-creating your settings.php file from default.settings.php template"

docker image for a complete emoncms installation

I've tried to run the emoncms-docker. There is a lot of configuration files to manage. Apparently, you have to run the mqtt broker separatly. As I am new to docker, I had difficulties to make it work....

I wanted to share another way to consider things, with a single image including mariadb, redis, apache and php, totally seamless with the EmonScripts and the emonpi I think

here is the link to the dockerfile for that :

https://github.com/dromotherm/sandbox/blob/master/emoncms/emoncms_lamp/Dockerfile

and the readme :

https://github.com/dromotherm/sandbox/tree/master/emoncms

To test without having to build or touch any file :

docker pull alexjunk/emoncms:0.0.3
docker run --rm -it alexjunk:emoncms:0.0.3

To get the container IP address : docker network inspect bridge

To send mqtt datas from the host if 172.17.0.2 is the container IP : mosquitto_pub -h 172.17.0.2 -u "emonpi" -P "emonpimqtt2016" -t 'emon/test/t3' -m 15

if you want persistant sql and timeseries datas :

docker volume create mariadb_datas
docker volume create ts_datas
docker run --rm -v mariadb_datas:/var/lib/mysql -v ts_datas:/var/opt/emoncms -it alexjunk/emoncms:0.0.3

everything is working fine :

• mariadb, redis-server, apache2 and mosquitto are started as services
• the 3 workers : emoncms_mqtt, feedwriter and service-runner are started by supervisord
• sync and backup modules work OK

maybe one can say it is not in the spirit of docker as things are not isolated, etc, etc, but I do think it is in the spirit of emoncms to have its stack mariadb, redis, apache and php in a single container...

@TrystanLea , @chaveiro : any thoughts on that ?

docker ENV port MySQL port is not present in settings.php

currently hardcorded to 3306, should also map to MYSQL_PORT environment variable

Raspberry Pi Docker dev

Run Emoncms docker on a Raspberry Pi

- Requirement to run root file-system in read-only for long term SD card life span as mentioned on Hypriot docker issues hypriot/image-builder-rpi#119 (comment)

Two options:

1. Start with Hypriot RasPi SD card

• SD card image: http://blog.hypriot.com/downloads/

Either build up Emoncms in a single image using resin/rpi-raspbian/ as base

• + Better for production
• • Have to workout how to make RO FS
• https://hub.docker.com/r//resin/rpi-raspbian/

Or use docker-compose multi container setup with resin/rpi-raspbian/ or nidorpi/rpi-php as the base

• + Better for development
• • Requires docker-compose
• https://hub.docker.com/r/nidorpi/rpi-php
• https://hub.docker.com/r/hypriot/rpi-mysql/
• https://hub.docker.com/r/hypriot/rpi-redis/

2. Start with Raspbian Jessie image

• + Already have working setup with RO FS
• /- Not optimised for Docker like the Hypriot pre-built SD card will be

Instructions to install Docker on Raspbian Jessie :

• with RO root FS and RW partition in /home/pi/data

In the Debian Jessie main repository there is not yet an installation package for Docker , so we will download the package docker-hypriot_1.10.3-1_armhf.deb from hypriot.com. They provide binary packages for latest version on Docker.
We will install the package with the following command :
sudo dpkg –i docker-hypriot_1.10.3-1_armhf.deb
They provide binary packages for latest version on Docker. After installation we have checked Docker daemon functionality with the command:
 sudo docker info
with the output:
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: 1.10.3
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Plugins:
 Volume: local
 Network: bridge null host
Kernel Version: 4.1.21-v7+
Operating System: Raspbian GNU/Linux 8 (jessie)
OSType: linux
Architecture: armv7l
CPUs: 4
Total Memory: 925.8 MiB
Name: raspberrypi
ID: TMKH:55WA:6KVM:SDMD:PIBU:O2YZ:KAK6:IH73:P5XT:UVCJ:HZIW:FW5Y
Debug mode (server): true
 File Descriptors: 11
 Goroutines: 20
 System Time: 2016-04-10T22:23:40.196987813+03:00
 EventsListeners: 0
 Init SHA1: 0db326fc09273474242804e87e11e1d9930fb95b
 Init Path: /usr/lib/docker/dockerinit
 Docker Root Dir: /var/lib/docker
WARNING: No memory limit support
WARNING: No swap limit support
WARNING: No oom kill disable support
WARNING: No cpu cfs quota support
WARNING: No cpu cfs period support
We will fix the warnings by adding the text  “cgroup_enable=memory swapaccount=1” to the file /boot/cmdline.txt.
and the requirement for read-only filesystem is to move the docker root dir on the writable image:  /home/pi/data/docker 
with these operations:
sudo service docker stop
next backup the original location with:
tar -zcC /var/lib/docker > /home/pi/data/docker/varlibdocker-backup-$(date +%s).tar.gz
move the folder in the new location: /home/pi/data/docker
mkdir /home/pi/data/docker
change permissions with
sudo chmod -R a+w /home/pi/data/docker
sudo chown -R pi /home/pi/data/docker
sudo chgrp -R pi /home/pi/data/docker
sudo mv /var/lib/docker /home/pi/data/docker
next make a symlink:
sudo ln -s /home/pi/data/docker /var/lib/docker   
and restart the docker engine:
sudo systemctl start docker
We will get a basic image that contained Java Docker that was adapted for ARM7 architecture. We found this Docker image on Docker Hub and we downloaded it with the following command:
docker pull sdhibit/rpi-java for a Java image
and
docker pull hypryot/rpi-node for a NodeJs image
To allow acces from Docker container to the host port 3000 (Rest Api) add the following line to iptables:
-A INPUT -i docker0 -p tcp --dport 3000 -j ACCEPT
and reload the iptables settings with : 
 sudo netfilter-persistent reload 
docker0 is the Docker bridge interface with the address 172.17.0.1. We will use this ip address instead of localhost to access the host applications. 

Issue with docker-compose up from Quick start

I have

pi@pi3:~/emoncms-docker $ docker-compose up
Starting emoncmsdocker_redis_1
Starting emoncmsdocker_db_1
Starting emoncmsdocker_web_1

ERROR: for web  Cannot start service web: driver failed programming external connectivity on endpoint emoncmsdocker_web_1 (672aa8242faa8703bd26ee0f950b147095dd971350d693828490e0078ea3fea1): Error starting userland proxy: listen tcp 0.0.0.0:8080: bind: address already in use
ERROR: Encountered errors while bringing up the project.

Any ideas what it can be?

EmonCMS' dockerised Web service fails when starting up

Kia ora,

I'm working on getting EmonCMS up and running in a Docker container running on my Raspberyy Pi 3 Model B Plus. I've had to modify the docker-compose.yml file to pull in a MariaDB image that works with the ARMv7 (ARMHF) instruction set, from:

pi@XENON:~/Containers/Github/EmonCMS/EmonCMS-Docker $ git diff
diff --git a/docker-compose.yml b/docker-compose.yml
index a063ca8..e9b4ea6 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -22,7 +22,7 @@ services:
   db:
     # Official MYSQL docker image
     # To do: make compatiable with 5.7
-    image: mariadb:latest
+    image: linuxserver/mariadb:latest
     environment:
       - "MYSQL_HOST=localhost"
     volumes:

Other than that, everything else is as per committed git.

When I fire up the Dockerised services/containers, I am getting the following error reported by the Web service:

pi@XENON:~/Containers/Github/EmonCMS/EmonCMS-Docker $ docker-compose up
Starting emoncms-docker_redis_1 ... done
Starting emoncms-docker_db_1    ... done
Starting emoncms-docker_web_1   ... done
Attaching to emoncms-docker_redis_1, emoncms-docker_db_1, emoncms-docker_web_1
redis_1  | 1:C 18 Aug 2020 00:40:51.849 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
<snip>
web_1    | standard_init_linux.go:211: exec user process caused "exec format error"
<snip>
emoncms-docker_web_1 exited with code 1

^CGracefully stopping... (press Ctrl+C again to force)
Stopping emoncms-docker_db_1    ... done
Stopping emoncms-docker_redis_1 ... done
pi@XENON:~/Containers/Github/EmonCMS/EmonCMS-Docker $```

I am not sure how to proceed checking out how to fix this.

MQTT integration

Hi,

i have mqtt server and set the mqtt in emocms properties to point to it.

When i publish a message with mqttlense app to the topic i.e. "emon/2/test" value "100" i would expect that this creates an input in emoncms but nothing happens.

If i subscribe to this topic i can see that the message is published correctly.

Props in emoncsm docker:

MQTT_ENABLED | true
MQTT_HOST | xx.xx.xx.xx
MQTT_USER | mqtt
MQTT_PASSWORD | xxxx
MQTT_BASETOPIC | emon

Mod rewrite missing when mounting www

When uncomment volumes in override i get apache rewrite module missing and can't login.
What is the best approach to have the app module installed without loosing it every time i need restart the docker?

Zoom charts does not work.

I have just moved to this docker version of emoncms. Everything seems ok but zoom graphs (really mandatory for power monitoring) are all empty. I have migrated about 4GB of php storage and also complete database into it. If I look at feeds I see updates in SQL feeds. When I look at chart from feed page I can see data updated. But no zoom and simplezoom widget works with these feeds.

And it looks like this project is dead. I expected that it will move to latest version as soon as possible. And just replacing image of emoncms docker container will give me update. This would be great as I have modified making process so everything about data is outside the containers on the host system. I can delete database image, main image, make it fresh and it runs where it was just before it.

I will investigate this issue more but it is quite slow as I am not linux guru and I do not have 50 hours a day time.

Sensitive data publicily exposed via php-info.php

Issue: environment variables containing sensitive info (passwords) are publicily exposed in the following scenario:

• Emoncms is installed in Docker using the documented method via docker-compose
• Emoncms is behind a reverse proxy, e.g. jwilder/nginx-proxy

Example dockerfile (docker-compose.prod.yml w/ VIRTUAL_HOST w/o ports):

# production / deployment / clean testing enviroment setup

version: '2'
services:

  web:
    #ports:
      # Expose Emoncms to the world on port 80
      #- 80:80
    env_file:
     # Use .env with secure db credentials (needs creating), ignored by git
     - '.docker-env'
    restart: always
    environment:
      # Set production enviroment variable flag, not currently used for anything but might be of use in the future
      PRODUCTION: 'true'
      # Used by nginx-proxy
      VIRTUAL_HOST: 'my-emoncms.company.com'
    
  db:
    restart: always
    env_file:
     - '.docker-env'

By accessing the url http://my-emoncms.company.com/php-info.php interally or externally, all info from .docker-env is exposed:

Variable	Value
MYSQL_HOST	db
MYSQL_PORT	3306
MYSQL_DATABASE	emoncms
MYSQL_USER	emoncms
MYSQL_PASSWORD	YOUR_SECURE_PASSWORD
MYSQL_RANDOM_ROOT_PASSWORD	yes
REDIS_ENABLED	true
REDIS_HOST	redis
REDIS_PORT	6379
REDIS_PREFIX	'emoncms'
MQTT_ENABLED	false
MQTT_HOST	localhost
MQTT_USER	YOUR_MQTT_USER
MQTT_PASSWORD	YOUR_MQTT_PASSWORD
MQTT_BASETOPIC	emon
VIRTUAL_HOST	my-emoncms.company.com

This works externally due to nature of reverse proxy (requests are always originating from a local ip).

I would recommend removing php-info.php from the docker installation set. But, rather remove it from the general deployment set because it isn't necessary for normal operation and users could point /var/www/html to a currently checked out release (with php-info.php).

To do

• Allow docker container to startup with a large amount of test data & demo dashboards etc.
• Create ability to simulated large number of inputs
• Simulate emoncms.org multi-server
• Add MQTT
• Backup module
• Add Redis
• EmonHub
• Investigate running Docker on RaspberryPi
• Use Composer instead of PEAR to install PHP extensions
• Use Pickle to install PECL Extensions
• Update to php 7, potential performace improvements

Fixing quick start

If I'm reading it right, the current quick start won't quite work because ./bin/setup_dev_repositories is not there, so the default mount in docker-compose.override.yml leaves Apache with nothing to serve, resulting in 403's.

Not sure which solution is preferable - modify the quick start to set up dev repositories, or modify docker-compose.override.yml so this isn't needed. I'll open a PR that just updates the README since that won't impact anyone who is used to the way it currently works.

mysql on aarch64?

I tried with latest too, but no go. What' missing?

Pulling db (mysql:5.6)...
5.6: Pulling from library/mysql
ERROR: no supported platform found in manifest list
karlp@odroid64:/src/emoncms-docker$ vi docker-compose.yml
Pulling db (mysql:latest)...
latest: Pulling from library/mysql
ERROR: no supported platform found in manifest list
karlp@odroid64:/src/emoncms-docker$