david00/rpi-power-monitor

Power Monitor (for Raspberry Pi)

Power Monitor HAT (for Raspberry Pi)

This project is a combination of custom hardware and software that will allow you to monitor your unique power situation in real time, including accurate consumption, generation, and net-production. The data are stored to a database and displayed in a Grafana dashboard for monitoring and reporting purposes.

This project is derived from and inspired by the resources located at https://learn.openenergymonitor.org.

---

Where can I get it?

I am offering DIY kits, factory-assembled PCBs, and a variety of current transformers to use with this project.

Please see https://power-monitor.dalbrecht.tech/ for more information.

---

How do I install?

There are two main ways to install - either flash the custom OS image to your microSD card, or perform the manual installation process on your existing image. Both processes are documented here:

See Software Installation in the docs.

After installation, see the configuration section in the docs for further information on setting up the power monitor.

---

What does it do?

This code accompanies DIY circuitry that supports monitoring of up to 6 current transformers and one AC voltage reading. The individual readings are then used in calculations to provide real data on consumption and generation, including the following key metrics:

• Total home consumption
• Total solar PV generation
• Net home consumption
• Net home generation
• Total current, voltage, power, and power factor values
• Individual current transformer readings
• Harmonics inspection through a built in snapshot/plotting mechanism.

The code takes tens of thousands of samples per second, corrects for phase errors in the measurements, calculates the instantaneous power for the tens of thousands of sampled points, and uses the instantaneous power calculations to determine real power, apparent power, and power factor. This means the project is able to monitor any type of load, including reactive, capacitive, and resisitve loads.

---

Documentation

https://david00.github.io/rpi-power-monitor/docs/general/index.html

---

Contributing

Would you like to help out? Shoot me an email at [email protected] to see what items I currently have pending.

---

Credits

• OpenEnergyMonitor and forum member Robert.Wall for guidance and support

• The spidev project on PyPi for providing the interface to read an analog to digital converter

---

Like my project? Donations are welcome!

BTC: 1Go1YKgdxAYUjwGM1u3JRXzdyRM938RQ95

Last Updated: April 2023

rpi-power-monitor's Issues

Backup & Restore

Hi David

I'm having a few problems with backing up and restoring the power_monitor database and I wonder whether you have any insights which may help?

After a fair amount of browsing I attempted the following backup and restore process:

I created an internal docker directory '/backups':

$ docker exec -it influx bash
# mkdir /backups
# exit

Create backup:
$ docker ps (gets influxdb container ID)
$ docker exec -it <container ID> influxd backup -portable /backups

Copy the backed up databases to a network share mapped in fstab:
$ docker cp <container ID>:/backups /home/pi/influxdata/backup<backup date>

Restore the data:
Copy the backup to the /backups container folder:
$ docker cp /home/pi/influxdata/backup<backup date> <container ID>:/backups

Start a bash session in the influx docker container and then restore the backed up database into a new database called pm_bak.
$ docker exec -it influx bash
# influxd restore -portable -db "power_monitor" -newdb "pm_bak" /backups/backup<backupdate>
# exit

Sideload into the power_monitor database and drop the temporary pm_bak database:

Open up an influxdb shell:
$ docker exec -it influx influx

> USE pm_bak
> SELECT * INTO power_monitor..:MEASUREMENT FROM /.*/ GROUP BY *
> DROP DATABASE pm_bak
> exit

I think this should work as it's almost verbatim from the online backup and restore instructions on the Influxdb website (https://docs.influxdata.com/influxdb/v1.8/administration/backup_and_restore/) , modified to work with a container. However, the sideload command ('Select *' etc.) results in errors in the influx docker logs and terminates with a 'ERR: no data received'.

$ docker logs influx 2>&1 | grep -i 'error'
ts=2021-12-31T15:06:11.114892Z lvl=warn msg="Error while freeing cold shard resources" log_id=0YlFfLY0000 service=store error="shard is disabled" db_shard_id=36
ts=2021-12-31T15:06:17.807313Z lvl=info msg="Cannot retrieve measurement cardinality" log_id=0YlFfLY0000 service=store db_instance=pm_bak error="engine is closed"
ts=2021-12-31T15:06:19.276092Z lvl=warn msg="Error while freeing cold shard resources" log_id=0YlFfLY0000 service=store error="shard is disabled" db_shard_id=25
plus several more lines, ending with
fatal error: out of memory

It's possible I have a corrupt database but I've also tried the same process with a 'clean' database and I get the same types of errors, so I'm pretty sure I'm doing something stupid. Permissions, possibly?

How do you do your backups?

Help!

event analyser

I just found this project, and finds it very interesting. I have not read all the documentation, so this might be an obsolete question.

Have you considered any event recorder, for analysis of voltage sags (duration and magnitude) in ms time scale?

How to correct measurements for 240v circuits, i.e. air conditioner, range, dryer, etc

David

first of all thank you for this project.

How should I go about If I want to measure 120v and 240v circuits with the same pi hat?

Are there specific settings on the config.py if, for example, I am measuring a single 15Amps breaker on CT-0 and a 60Amps 240v circuit on CT-1.

Regards

Pablo

Switch from InfluxDB to VictoriaMetrics storage?

It would be great if VictoriaMetrics could be used in the project instead of InfluxDB. It supports data ingestion over Influx line protocol, so there is no need to change anything from data ingestion side. It provides MetricsQL - PromQL-inspired query language, which is better suited for building typical dashboards in Grafana comparing to Flux or InfluxQL. See these docs for details. VictoriaMetrics usually requires lower amounts of resources (CPU, RAM, persistent storage) comparing to InfluxDB - see this article for details. Docker images for VictoriaMetrics are available here.

With DIY kit Raspberry PI will not boot

When i connect my power monitor to the Raspberry PI the PI will not boot up and no power light. I remove the monitor and it boots fine. I have doubled check components and soldering not finding no problems. Any ideas?

Grafana "Home Power Usage" & "Amperage" graphs don't display data for CT5.

Problem: Data for CT5 is not stored to the database, so it's not displayed in the charts in Grafana.

Help identifying component L1.

The L1 component in the schematic just says 1m with no units. Package type 0.4 axial. I looked at the assembled board picture. I want to check my assumptions. Is that a high frequency ferrite choke?

Calibration issues

when i set this up i missed a bunch of the calibration steps and just trying to get them done but something doesnt seem to be working correctly... its using your custom image on a pi4 and the kernel version appears to be the correct 4.19 one

my issue is CT1 is the main feed coming into the house (in NZ, so on 240v single phase), and for some reason its showing that CT3 which is our swimming pool is using more power than what is actually coming into the house which suggests somethings not quite right

trying to do the automatic calibration following the wiki comes up with the following suggestions on what to enter in the phase adjustments but they dont seem to be what they should be

CT1 (Main incoming feed) 2.54774229
CT2 (Hot water) 4.74891988
CT3 (Swimming pool) 4.1974763

"database not found" when starting power-monitor.py

Hello,

Everything seems to have gone well when installing according to the wiki. I did have to start the Docker containers manually after restart because they did not start on their own.

When trying to run power-monitor.py I get the following exception for, "Database not found". Can you tell where I went wrong?

Voltage tuning - AC multiplier seems to have no effect

I purchased the kit from you, and bought the 9v transformer you suggested.

I'm having an odd issue on the voltage tuning step. https://github.com/David00/rpi-power-monitor/wiki/Voltage-Tuning

Steps 1-6, i've measured the voltages, and edited the config file. My values are:
GRID_VOLTAGE = 124.2
AC_TRANSFORMER_OUTPUT_VOLTAGE = 10.4

When I get to steps 7 and 8, this is the output I get:

2020-08-14 18:06:24 :

+---------+---------+---------+---------+---------+---------+---------+
|         |   CT0   |   CT1   |   CT2   |   CT3   |   CT4   |   CT5   |
+---------+---------+---------+---------+---------+---------+---------+
|  Watts  | 548.882 | 538.409 | 530.817 | 532.487 | 542.351 | 548.233 |
| Current |  2.682  |  2.629  |  2.591  |  2.601  |  2.648  |  2.678  |
|   P.F.  |  0.915  |  0.916  |  0.916  |  0.916  |  0.916  |  0.916  |
| Voltage | 223.561 |         |         |         |         |         |
+---------+---------+---------+---------+---------+---------+---------+

The wattage numbers look absurd given nothing is connected. The voltage is also wildly inaccurate. But, carrying on, I got the multiplier, and in my case it's 0.5343. I add that to the AC line at the bottom, and my output is the same.

2020-08-14 18:07:50 :
+---------+---------+---------+---------+---------+---------+---------+
|         |   CT0   |   CT1   |   CT2   |   CT3   |   CT4   |   CT5   |
+---------+---------+---------+---------+---------+---------+---------+
|  Watts  |  557.58 | 547.525 | 539.154 | 540.746 | 550.063 | 555.691 |
| Current |  2.703  |  2.652  |  2.613  |  2.623  |  2.667  |  2.694  |
|   P.F.  |  0.916  |  0.917  |  0.916  |  0.915  |  0.916  |  0.916  |
| Voltage | 225.222 |         |         |         |         |         |
+---------+---------+---------+---------+---------+---------+---------+

I have absolutely no idea what could be wrong. I'm 99% sure I assembled the board right.

Potential of using PI PoE HAT

Is there the potential of using a PoE HAT with the GPIO pin headers passed through to the monitor board. This way you only need a single cable run to the meter cupboard.

Documentation Clean Up

I've received feedback from a few people that the documentation can be difficult to follow along as it is sometimes unclear how to proceed after finishing a page. Thank you for all your feedback!

I'm opening this issue have something to reference commits to as I work towards improving the documentation.

If, at any time, you find something that is unclear or difficult to follow, please leave a comment here (even if the issue is closed in the future, feel free to reopen it), and I will address it.

Start4.elf is not compatible

Please do help me with this issue as per the photo has shown, i have tried multiple methods with the newest imager and even with the imager that was downloaded from the guide's link.
I am still having the same issue over and over again.

Setup compatibiliy question

I have a digital 2 way Energy counter and in a different room an additional Power Generation counter.

Could this be sanely monitored with your solution?

Non Linear calibration?

Hey David,
First of all, this is an awesome project and my HAT is off to you! (pun very...VERY much intended).

I do have a question about final calibration. If noise induced on the CT shows a value above zero when not even clamped around a wire that has current flowing, wouldn't that imply a non-linear scaling factor? Maybe in the future you could make it possible to graph several values compared to a clamp meter and scale accordingly?
Even 3-5 points on a graph would make this quite a bit more accurate.

Thanks again for all you work!

Enhancement: export data to MQTT

I was wondering if anyone would be interested about exporting crunched data into MQTT to be able integrate it into Home Assistant Energy consumption? I think that would be a nice feature to add.

Log tariff alongside current data

My electricity supply is dual rate (ie) I'm charged a day rate and a night rate. I've been looking into how to (fairly) accurately calculate how much a particular piece of equipment is costing me. I need to apply the correct tariffs, which won't necessarily be the same as the current ones if I'm looking at historical data. My other variable is the times of day when the tariffs cut in and out.
My researches so far suggest that the best way to do this would be to record the tariff data alongside the current measurements, making grafana display quite simple (which is a must as far as I'm concerned(!)). Is this something that could be realised in the code or would I be better doing something in Grafana?

Power Factor not being stored to DB

The calculated power factor for individual current transformers is not being stored to the database. The calculations are already done, but the value was overlooked in the DB storing function. This will be fixed in an upcoming update.

Grafana password frequency and SD Card fail

Hi David!

Grafana (v7.1.3) is bugging me much more with password requests than I remember when first installing my system last fall.

Today, in the middle of a session of examining some data, it interrupted me and demanded a password again!

Now, I use chrome to store the password, so it is a few clicks, but I would like to reduce the frequency, or possibly go password free on Grafana.

Since the pi is behind a router NAT, it is not like the world has direct access to the pi. "ShieldsUp" shows my ports are in stealth, but that is the extent of my network security knowledge.

In stumbling about on Grafana, I found nothing in "preferences", but some mysterious variables in Server Admin->settings->auth. The comment is "These system settings are defined in grafana.ini or custom.ini (or overridden in ENV variables). To change these you currently need to restart grafana."

Have you accepted the defaults or can you suggest changes that might be less annoying?

Thanks.

Local time and DayLightSaving offset

Hello. Based on your experience, how will DayLight Saving will be handeled ?

if I understand weel, the python program does use the local time to store the date/time in the database.
then Grafana will use this timing
I made a Cost-Of-Use dashboard and I had to substract (for some reason that is not really clear for the moment) my time difference with the UTC (here 1h because I am in France).
But because this delay will change in the PI/database, I will need to change in Grafana this offset to -2h.

Should we leave then the PI in UTC timezone and change the local zone in Grafana only ? This is what I read in some forum online.

Phase calibration does not work even with the correct kernel version

So, I've followed the setup instructions, the only difference is I remade the circuit on a breadboard and used some bigger capacitors.
The current transformer I used is YHDC SCT006, the burden resistor is 22 Ohms.

The issue is, when I am doing the phase calibration, even with the correct kernel version (4.19.118-v7l+), the results are still larger than 1.x. I have attached the results from your program's debug mode and phase calibration mode, please help!
(The sampling rate reported by the debug mode is around 28 KSPS, is that normal? I don't know... I am using Raspberry Pi 4B with 4GB ram)
phase_calibration_problem.zip

The load I used was an electric heater and its power factor is 1.00, confirmed by a store-bought power meter.

I have also attached photos of my setup, I am clueless what to do next... Any help is much appreciated, thanks a lot.

Current measuring will not read zero

So, I've done everything in the wiki, only difference is my board is completed on a bread board instead of a PCB.

Now, when I'm running the power monitor in terminal mode, I see there is always a current reading of about .5 Amps even when the CTs are not connected, in fact even if I ground CT_input to GND, there is still a .5 Amps reading. When there is actually load, the current measurement is roughly accurate.

This 0.5A messes with the power consumption calculation, because this means there will be a 100W consumption even when nothing is connected, and that's unacceptable. I don't know what is the problem, the ADC, the capacitors I used, the OpAmp, I do not know, please advise!

requirements.txt

Upgrade the python-apt packages. For the stable distribution (buster), this problem has been fixed in version 1.8.4.2.
new ver 12/20/20

Is this project also avaible without the 9V AC transformer?

hello there, i live in the Netherlands and i want to use this project to watch my home power usage. Is it also possible to use this project without the 9V AC transformer (because it cant be shipped to here)?

Service occasionally fails: Failed with result 'start-limit-hit'

Once every couple of weeks I notice my power data isn't being updated. I can ssh into the Rpi successfully but the power-monitor service isn't running.

Here is the error message

pi@powermon:~ $ sudo systemctl status power-monitor
● power-monitor.service - Raspberry Pi Power Monitor
   Loaded: loaded (/etc/systemd/system/power-monitor.service; enabled; vendor preset: enabled)
   Active: failed (Result: start-limit-hit) since Tue 2021-10-19 19:29:18 PDT; 19h ago
  Process: 23982 ExecStart=/usr/bin/python3.7 /home/pi/rpi-power-monitor/power-monitor.py (code=exited, status=0/SUCCESS)
 Main PID: 23982 (code=exited, status=0/SUCCESS)

Oct 19 19:29:18 powermon systemd[1]: power-monitor.service: Service RestartSec=1s expired, scheduling restart.
Oct 19 19:29:18 powermon systemd[1]: power-monitor.service: Scheduled restart job, restart counter is at 6.
Oct 19 19:29:18 powermon systemd[1]: Stopped Raspberry Pi Power Monitor.
Oct 19 19:29:18 powermon systemd[1]: power-monitor.service: Start request repeated too quickly.
Oct 19 19:29:18 powermon systemd[1]: power-monitor.service: Failed with result 'start-limit-hit'.
Oct 19 19:29:18 powermon systemd[1]: Failed to start Raspberry Pi Power Monitor.

I issue sudo systemctl start power-monitor and it starts up just fine.

Are there any changes we can make to /etc/systemd/system/power-monitor.service to avoid this failure? Maybe increasing the number of retries or increasing the delay between retries?

For reference, here is what my /etc/systemd/system/power-monitor.service file looks like:

[Unit]
Description=Raspberry Pi Power Monitor
After=docker.service
Requires=docker.service

[Service]
Restart=always
RestartSec=1
StartLimitInterval=120
StartLimitBurst=5
User=pi
ExecStart=/usr/bin/python3.7 /home/pi/rpi-power-monitor/power-monitor.py

[Install]
WantedBy=multi-user.target

Suggestion: Channel Input Board

I found this interesting project here which resembles something quite similar but with
the additional functionality of monitoring every circuit seperately because of more possible inputs.
You can then monitor after every fuse seperately and make statistics for every room in the house.
Maybe it would be an idea to make something similar as voluntary upgrade which is compatible software and hardware-wise to your project:
https://github.com/crjens/PiPowerMeter

Image: https://github.com/crjens/PiPowerMeter/blob/master/Documentation/DSC_0009.JPG

Incorrect Capacitor Listing in assembly/schematic

Capacitor C4 in the schematic and component list on the assembly page lists C4 as 100nf but the footprint and image of assembled pcb on the assembly page indicate that it is a 1nf capacitor. Also L1 isn't mentioned in the assembly page anywhere. I'm still pretty new to anything beyond basic mcu projects so it may just be obvious to everyone else but the assembly page image seems to look like a wire link? May be beneficial just to clarify that on the assembly page.

Speed Optimizations

I'm opening this issue to have a specific place for discussion about the measurement/calculation speed, sample rates, and ideas for optimizing the code. I've already began work towards offloading some of the more CPU intensive tasks to Cython, but @kobuki brought an idea of using a native Python library like NumPy that has already done a lot of the underlying optimizations.

Here are some benchmarks I ran on my Pi 3B and my Pi 4 to compare the native Python code as it currently exists in 9375380 (latest master commit) to the Cython optimization that I've been working on for the last week. The measurements were performed only on the collect_data() function, so this is only comparing the speed of reading the ADC and storing the values to a Python list or C array. The more pressing measurement will be from the calculate_power() and rebuild_waves() functions as that's where the majority of maths are performed.

---

For each of the different tests, I ran about 2 dozen loops and then averaged the results to get an average sampling speed for each test. Only a few lines of output from each test are included for brevity (instead of all 24 lines)

Pi 3B - Python Sampling Rate

Collected 14000 samples in 0.4837 seconds | Overall Sample Rate: 28944.3/s | SPS Per Channel: 4134.9
Collected 14000 samples in 0.4933 seconds | Overall Sample Rate: 28382.9/s | SPS Per Channel: 4054.7
Collected 14000 samples in 0.5253 seconds | Overall Sample Rate: 26649.4/s | SPS Per Channel: 3807.1

Average Per-Channel Sampling Rate: 4088 SPS

Pi 3B - Cython Sampling Rate

Collected 14000 samples in 0.4366 seconds | Overall Sample Rate: 32062.7/s | SPS Per Channel: 4580.4
Collected 14000 samples in 0.4569 seconds | Overall Sample Rate: 30641.9/s | SPS Per Channel: 4377.4
Collected 14000 samples in 0.4524 seconds | Overall Sample Rate: 30945.4/s | SPS Per Channel: 4420.8

Average Per-Channel Sampling Rate: 4514 SPS

---

Pi 4B - Python Sampling Rate

Collected 14000 samples in 0.4517 seconds | Overall Sample Rate: 30992.6/s | SPS Per Channel: 4427.5
Collected 14000 samples in 0.4586 seconds | Overall Sample Rate: 30529.7/s | SPS Per Channel: 4361.4
Collected 14000 samples in 0.4505 seconds | Overall Sample Rate: 31074.7/s | SPS Per Channel: 4439.2

Average Per-Channel Sampling Rate: 4406 SPS

Pi 4B - Cython Sampling Rate

Collected 14000 samples in 0.4343 seconds | Overall Sample Rate: 32235.4/s | SPS Per Channel: 4605.1
Collected 14000 samples in 0.4340 seconds | Overall Sample Rate: 32258.9/s | SPS Per Channel: 4608.4
Collected 14000 samples in 0.4350 seconds | Overall Sample Rate: 32184.1/s | SPS Per Channel: 4597.7

Average Per-Channel Sampling Rate: 4592 SPS

So, you can see there is a pretty significant improvement on the Pi 3B when using Cython for collecting samples, but not so much on the Pi 4B. There's also not too much of a different between the Cython sampling speeds on the Pi 3B and the Pi 4B.

---

Still to do...

• Benchmark the actual calculation functions, calculate_power() and rebuild_waves() (ideally, separately, to get an idea of the performance gain for each function between Python and Cython)
• Benchmark a completely different optimization strategy using NumPy instead of Cython (and compare the two).

Remote influx

Is it possible to send data to a remote influx server?.

CT3 Current missing from total

Line 478 in power-monitor.py is missing CT3 from the calculation.

Latest influx ? Latest Grafana ?

influx is moving to a v2.0, however it doesn;t appear that the raspi 3B+ is supported just yet. Know anything different ?

Grafana too is up to docker container 8.2.6 - going to give this a try.

Different ADC ? > covered in issue 16 "Raspberry Pi version"

Hello David.
I am doing now a different board (I target a RPI0W size) and am looking for an ADC available at LCSC.
I do see the MCP3208 that is pin-pin compatible. However the resolution is higher (12b instead of 10b) and the max clock is lower.
I guess that the resolution is not a big issue as this can be ajusted with a different calibration value (factor divided by 4) but the speed may be more complicated to overcome. The max SPI clock is lower, then the conversion rate also.
I see that in you code you specify the max SPI at 1M75 but the max SPI clock frequency is at 1MHz for 2,7V.
In the other speed optimisation issue, the SPI interface was listed as a possible limitation.
Would you see any issue to limit the SPI clock and sampling rate ? Can we easily update this parameter ? or would we also need to change the number of sample to get per period (i think that you ask 2000samples per measurement) ?

Raspberry Pi version

I've bought the kit and have just received it but I have a question on the Raspberry Pi.
Would this work on the raspberry pi 1 for just monitoring the power values and grafana etc were hosted on another server ?
If not, Is there a minimum ram spec you think is needed for the raspberry pi?

Thanks.

Monitor intake circuit or just sum the individual circuits?

Tried to find this in a FAQ or the documentation to no avail.

I have an apartment with a 40A intake / main circuit breaker. In addition I have 5 smaller circuits ranging from 10A to 20A.

I plan to order the kit, but should I get a 6 clamp version (ie, monitor the intake + 5 sub circuits) or just go for the 5 clamp version, and just sum the 5 sub circuits together to get the total power draw?

For all I know, the software already has a function to sum the circuits together :)

What would you people do?

Influxdb error log: 'cannot allocate memeory' & 'fatal error: out of memory'

This isn't strictly an RPi-power-monitor issue, so I hope you'll forgive me for raising this issue here in the hope you'll tell me it's a me-only problem (ie) there isn't a general InfluxDB memory allocation bug or something equally intractable at work here!

Whilst trouble-shooting influxdb backup and restore, I noticed the docker influxdb log is recording quite a large number of 'cannot allocate memory' errors and a few 'fatal error: out of memory' events.

I'm running a new build (following the Wiki instructions for a manual install) with a couple of minor changes - I've installed 32 bit Bullseye and I've had to tweak one of the startup commands to reference Python 3.9 (I can't remember which one at the moment).
My computing hardware is a Pi 4 with 2GB RAM

I get at the logged errors using "docker logs influx 2>&1 | grep -i 'fatal|cannot allocate memory' "

The docker log is 8MB (the container has only been running a couple of days or so). I've attached it plus an extract to illustrate the sort of error messages I'm getting.

The errors only started appearing some hours after the initial start up. The reason I started off with a new build was that I noticed similar errors in the logs on my long running power monitor build and assumed it had become corrupted by a recent power outage. Unfortunately, this now looks like it wasn't the case.

A quick look on the web threw up this little gem:
https://community.influxdata.com/t/fatal-error-out-of-memory/16103
which made me wonder whether what I'm seeing may be inherent with only having 2GB RAM to play with. The proposed solution changes indexing to disk from RAM and this doesn't sound good in terms of SD wear.
Any thoughts?

Regards,
Mike

Influxdb errors.txt
influxdb.log

[Edit 1 day later]: Have burned a new SD with the image file linked to in the software install instructions and am giving that a run. So far (1 hour in) no error messages. Keeping my fingers crossed!

I have got some issue

The adc which is mcp3008 is overheated, I am not using pcb, I use your early method at https://community.openenergymonitor.org/t/diy-current-monitor-on-raspberry-pi-my-power-calculation-isnt-accurate-help-please/13266/50 by using breadboard and gpio breakout broad, did the post installation setup until the "file written"

Refactoring into a Class

I wanted to share and see if there's any interest in further development. In order to make installing and using this library easier, I turned your code into a Class, as well as created a setup.py to allow easy installing via pip, at https://github.com/kizniche/rpi-power-monitor. It can then be installed as a package with:

python -m virtualenv ./venv
./venv/bin/python -m pip install git+https://github.com/kizniche/rpi-power-monitor.git

Then, for example:

from rpi_power_monitor import power_monitor

rpm = power_monitor.RPiPowerMonitor()
rpm.run_main()

This allows me to import it in a new input module I created for Mycodo, which uses it like this:

from rpi_power_monitor import power_monitor

phase_correction = {
    'ct1': self.ct1_phase_correction,
    'ct2': self.ct2_phase_correction,
    'ct3': self.ct3_phase_correction,
    'ct4': self.ct4_phase_correction,
    'ct5': self.ct5_phase_correction,
    'ct6': self.ct6_phase_correction,
}

accuracy_calibration = {
    'ct1': self.ct1_accuracy_calibration,
    'ct2': self.ct2_accuracy_calibration,
    'ct3': self.ct3_accuracy_calibration,
    'ct4': self.ct4_accuracy_calibration,
    'ct5': self.ct5_accuracy_calibration,
    'ct6': self.ct6_accuracy_calibration,
    'AC': self.ac_accuracy_calibration,
}

self.sensor = power_monitor.RPiPowerMonitor(
    grid_voltage=self.grid_voltage,
    ac_transformer_output_voltage=self.transformer_voltage,
    ct_phase_correction=phase_correction,
    accuracy_calibration=accuracy_calibration)

board_voltage = self.sensor.get_board_voltage()

samples = self.sensor.collect_data(2000)

rebuilt_waves = self.sensor.rebuild_waves(
    samples,
    self.sensor.ct_phase_correction['ct1'],
    self.sensor.ct_phase_correction['ct2'],
    self.sensor.ct_phase_correction['ct3'],
    self.sensor.ct_phase_correction['ct4'],
    self.sensor.ct_phase_correction['ct5'],
    self.sensor.ct_phase_correction['ct6'])

results = self.sensor.calculate_power(rebuilt_waves, board_voltage)

self.logger.info(f"Voltage: {board_voltage}")

chan = 1
for ct in range(1, 7):
    self.logger.info(f"Power {chan}: {results[f'ct{ct}']['power']} W")
    self.logger.info(f"Current {chan}: {results[f'ct{ct}']['current']} A")
    self.logger.info(f"Power Factor {chan}: {results[f'ct{ct}']['pf']}")
    chan += 3

Here's the module being configured and running in the web UI (only the first channel is enabled):

Input Page

Live Page

Dashboard Graph Widget

Current spike suppression?

Not so much as an issue specifically, as much as it is seeing if anyone has a clever method for eliminating spikes, either through software or hardware changes? I have a panel that I'm monitoring which includes a well pump, and every time the well pump kicks on, you can see spikes in all of the channels due to induction. CT2 is monitoring one leg of the pump circuit (240V pump). CT4 and CT5 are monitoring my mains, so I would expect to see a spike there, but the others (0, 1, 3) have nothing to do with the pump. Thoughts? I was thinking of using shielded wire, but I'm having a hard time finding it. I'm also considering buying a series of chokes. But I'm wondering if there's anything to do for it in software?

Buster vs. Bullseye.

I was trying to upgrade the unit to Bullseye and followed the instruction to in PiPower again, but it fails to monitor.
It was really a while ago and I could not remember the exact details.

Do you have any plans to upgrade the software to be able to run under Bullseye, with the latest Docker and Grafana?

Thank you.

database not saving data?

Hey,

i noticed something strange where when i restarted the pi it appears to have either lost a bunch of data or it was showing the incorrect data in the dashboard... not sure which

after restarting CT1 and 2 (the only 2 im using so far) appear to have lost around 30KWH worth of records, and was wondering if theres something that needs to be run to make the database save or something?

Docker images wont start

Hi,
I've tried both the image and installing from scratch (on bullseye) but when i do docker ps status is always restarting.
239e96bd1c45 influxdb:1.8.3 "/entrypoint.sh infl…" 3 hours ago Restarting (139) 17 seconds ago influx
8a945ecb7074 grafana/grafana:7.3.7 "/run.sh" 3 hours ago Restarting (139) 13 seconds ago grafana

I'm on a Pi zero, hoping you can help

Thanks
Mark

Panel plugin not found: timeseries

first off, great project! absoultley love it

have just set it up following the wiki, and using the individual CT dashboard im getting the following error on the CT listings
Panel plugin not found: timeseries

do you know how i can fix it? have never used grafana before and google doesnt seem to answer it

Wiki instructions not working to run Docker containers as services

I'm opening this issue as I've received several reports via email about the influxdb part of the Software 2. Running as a Service instructions in the project Wiki not working for some folks.

This issue will track the status of my investigation into the problem and the eventual resolution.

The problem is usually encountered when starting the software in terminal or regular mode, because both of these modes save to the database. The traceback that Python provides is extremely verbose, so I will clean that up too, but if you see a ConnectionRefusedError: [Errno 111] Connection refused in your output, you are likely hitting this issue. You may also see the following from sudo systemctl status influx:

pi@raspberrypi:~ $ sudo systemctl status influx
● influx.service - Influx container
   Loaded: loaded (/etc/systemd/system/influx.service; enabled; vendor preset: enabled)
   Active: failed (Result: start-limit-hit) since Thu 2020-09-03 10:21:35 PDT; 5 days ago
  Process: 896 ExecStart=/usr/bin/docker start influx (code=exited, status=0/SUCCESS)
 Main PID: 896 (code=exited, status=0/SUCCESS)

Sep 03 10:21:35 raspberrypi systemd[1]: influx.service: Service RestartSec=100ms expired, scheduling restart.
Sep 03 10:21:35 raspberrypi systemd[1]: influx.service: Scheduled restart job, restart counter is at 5.
Sep 03 10:21:35 raspberrypi systemd[1]: Stopped Influx container.
Sep 03 10:21:35 raspberrypi systemd[1]: influx.service: Start request repeated too quickly.
Sep 03 10:21:35 raspberrypi systemd[1]: influx.service: Failed with result 'start-limit-hit'.
Sep 03 10:21:35 raspberrypi systemd[1]: Failed to start Influx container.

A quick workaround is to disable the InfluxDB service, then restart the Docker container manually:
sudo systemctl disable influx
docker restart influx

... but a more permanent fix along with revised Wiki instructions will be coming soon.

I will also be adding some error handling in the code to provide easier/more readable output instead of the multiple level traceback.

Kw reading disparity

Hi David,

I recently built the kit from you and run all 6 CTs in my Australian 240v main switchboard. I got the 100A CT due to its wire size capacity but I would be better suited with a 60A to be a closer match to my power requirements - to this end I appear to be getting a disparity between the two dashboards I have installed in Grafana. The individual CT dashboard will show a constantly updated usage that is entirely correct (I have checked with clamp meters and a subpanel meter in my garage for relevant devices). When I swap dashboards to the Home power dashboard I appear to get a Kw usage for the day of almost exactly double my actual usage. I.e, if I consumed 10Kw in the day measured with my utility mains supply power meter or load tests of running the lights or AC my CT dashboard reflects this but the Home Power dashboard shows a 20Kw usage. I don't believe this to be a 100A CT range problem as I get almost exact readings to the utility meter when load testing.

I'm wondering if this is due to (somewhere in the Wiki) you discuss using a CT on your two legs of incoming power (I'm assuming you have 110v) which is different to the Australian standard of a single input of 240v.

I've had a basic poke at the code behind the Home power dashboard but it is far beyond my knowledge. I'm assuming it's a set of code to 'double' your usage based on having two inputs in the US?

All big assumptions on my side so I might just be doing something wrong. I'm happy to provide screenshots or such if that would help.

Not a big deal if it can't be done or if I'm doing something wrong as I'm extremely happy with everything and I'm very impressed at what it can show and how easy it was to build and install. It's just a convenience thing instead of dividing the readout on the Home Power dashboard in my head to see my daily usage.

Thanks,

Rowan

3 Phase Power

Starting an issue to discuss the possibility (and perhaps track the progress) of supporting 3 phase power.

Influx DB gets too big, producing communication issues

I've been running the power monitor service for about 9 months, and recently I've seen a lot of communication issues with the Influx Db. The Influxd server was utilizing about 300% of the CPU, and the pi slowed to a crawl. I moved the influx directory, recreated a fresh power monitor db, and everything went back to normal. I think influx has an issue with large dbs on the pi.

I think there needs to be some kind of slimming process that thins out the old data - say keeping only 1 min or 5 min interval data for data older than 6 months.

30A Current Transformers

This could be a documentation suggestion or possibly expanded config parameters.

Currently using the ECS1030-L72 with a 75ohm resistor as suggested for 30A circuits.

The SCT-013 (100A) transformers needed a accuracy factor of 0.9. In comparison before I changed the accuracy calibration parameter for the ECS1030-L72 I get 82A for a 22A load measured by a clamp multimeter. That's a correction factor of 0.26. Initially this struck me as odd as I had interpreted the purpose of the accuracy calibration to only correct any variation between current transformers, burden resistors, etc. I realized the back end math isn't taking into account the size of the burden resistor or CT turns and just relying on this calibration factor. Looks like it's set up for the PZCT-02 10ohm and 1000/1 so a ECS1030 at 1000/.5 and 22ohm is 10% high like I saw. Voltage across the 75ohm burden resistor at 1000/.5 would therefore be 82/(75/(10*2))=22A, spot on.

I'm wondering if you might want to add config parameters for current transformer turn count, as well as burden resistor size, and do that bit of math in the back end. This way "calibration" would be more true to description and optional for those that don't have a clamp meter. Since you sent 1% burden resistors I'm guessing the accuracy would already be acceptable for many.

Also, kind of a side note, if there is a 30A CT on a 15/20A circuit is there any reason not to do the burden resistor math using the circuit breaker size as max? Obviously inrush current could go over 3.3v Vref but the ADC can obviously handle 5v+, although I couldn't find a spec for breakdown. Even with a 150ohm for a 15A circuit it looks like 30A of inrush would still be under 5v and slightly negative on the other half. This would realistically be the same ADC input range as a 75ohm on my 30A air conditioner circuit which has a 60A locked rotor current.

-Nick

Config files enhancements

Here is some idea of improvement I would suggest to implement:

• Friendly name / Alias name for a specific CT in config file.

• Enable - Disable for each CT in config file -- do not perform calculation for CT not enabled.

• Consumption or Production Value for each CT.

• Dynamically calculate the Net production, etc. based on values in config file.

Phase correction mode halts with a NameError traceback regarding report_title

When running the phase correction mode, the process doesn't finish all the way and instead throws the following traceback:

2020-08-31 11:25:36 : Please wait... building HTML plot...
Traceback (most recent call last):
  File "power-monitor.py", line 738, in <module>
    plot_data(rebuilt_wave, report_title, ct_selection)
NameError: name 'report_title' is not defined

Default voltage value

Yesterday, I lost power to my AC-AC adapter for a few hours. During that time the current sensors and Pi were still churning along gathering data. (The Pi is powered by a POE hat and the GFCI breaker on the AC-AC tripped)

Is it possible for the voltage value to default to 120 if the measured voltage is low (<20)?