Giter Site home page Giter Site logo

everblu-meters-esp32's Introduction

everblu-meters-esp32 - Water usage data for MQTT

Fetch water/gas usage data from Cyble EverBlu meters using RADIAN protocol on 433Mhz. Integrated with MQTT.

Meters supported:

  • Itron EverBlu Cyble Enhanced

Software original code (but also all the hard work to get thingd working was originaly done here then put on github by @neutrinus here and then forked by psykokwak

I added some changes to the original firmware to work with my custom esp32-c3 board (including RGB led control, battery monitoring, deepsleep, battery charger), just plug, and play.

Hardware

The project runs on ESP32 with an RF transreciver (CC1101).

Hardware can be any ESP32+CC1101 with correct wiring.

I used my open source one cc1101-e07-pi, you can check it out for the already made hardware or look at the original repo if you want to build your own.

Configuration

All configuration is done thru platformio.ini file

You need to get your meter serial and year, it can be found on the meter label itself

Cyble Meter Label

Then put values in platformio.ini removing leading 0 from serial (mine was 0828979 I put 828979

    -D METER_YEAR=22
    -D METER_SERIAL=828979

Set the hour/min and retries mode of wakeup to read data

    -D WAKE_HOUR=08
    -D WAKE_MIN=15
    -D RETRIES=6        
    -D RETRIES_DELAY=15 

Here, each day at 08H15, if read fails it will retry 15min later and do that until it worked or 6 times maximum.

So in case of all reads fail, last retry will be at 09H45 and then stop retries.

Process will start back at 08H15 next day.

Set your WiFi credentials and time zone see timezone.h

    -D WIFI_SSID=\"YOUR-SSID\"
    -D WIFI_PASS=\"YOUR-PASSWORD\"
    -D TIME_ZONE=TZ_Europe_Paris ; See timezone.h of ./src folder

And then MQTT related

    -D MQTT_SERVER=\"192.168.1.8\"
    -D TOPIC_BASE=\"everblu/\"
    -D MQTT_USER=\"\"
    -D MQTT_PASS=\"\"
    -D MQTT_PORT=1883
    ;-D PUBLISH_RAW  ; Enable to publish also RAW values (keeping JSON)

If you need MQTT auth please do like that

    -D MQTT_SERVER=\"192.168.1.8\"
    -D TOPIC_BASE=\"everblu/\"
    -D MQTT_USER=\"MQTT_USER\"
    -D MQTT_PASS=\"MQTT_PASS\"
    -D MQTT_PORT=1883
    ;-D PUBLISH_RAW  ; Enable to publish also RAW values (keeping JSON)

MQTT Topics and data

Base Topic

Base topic is everblu followed by serial number (with leading 0 this time) followed by -espabcd, because I've also device with PI so I need to know who is sending data, of course you can remove it if needed.

everblu/cyble-22-0828979-espbf84

Reading OK

When read suceeded, results are sent in JSONformat to the base topic plus json

Example received on everblu/cyble-22-0828979-espbf84/json

{
    "ts": 1689656420,
    "date": "Tue Jul 18 07:00:20 2023",
    "esp_battery": {
        "percent": 97,
        "vbat": 4155 },
    "liters": 467932,
    "battery": 173,
    "read": 228,
    "rssi": -87,
    "lqi": -120,
    "hours":"06:18"
}

Values can also be sent in raw format enabling setting PUBLISH_RAW in platformio.ini

    -D PUBLISH_RAW  ; Enable to publish also RAW values (keeping JSON)
  • everblu/cyble-22-0828979-espbf84/liters
  • everblu/cyble-22-0828979-espbf84/battery
  • everblu/cyble-22-0828979-espbf84/read
  • everblu/cyble-22-0828979-espbf84/date
  • everblu/cyble-22-0828979-espbf84/ts
  • everblu/cyble-22-0828979-espbf84/hours
  • everblu/cyble-22-0828979-espbf84/rssi
  • everblu/cyble-22-0828979-espbf84/lqi

if board has battery built in chip (my esp32-c3 CC1101 boards)

  • everblu/cyble-22-0828979-espbf84/esp_bat_pc
  • everblu/cyble-22-0828979-espbf84/esp_bat_mv

Read fail

When read failed, results are sent in JSON

Example received on everblu/cyble-22-0828979-espbf84/error

{
    "ts": 1689626877,
    "date": "Mon Jul 17 22:47:57 2023",
    "esp_battery": {
        "percent": 90,
        "vbat": 4094 },
    "type": "No Data",
    "retries": 4
}

When read fail, device will follow RETRIES and RETRIES_DELAY parameters of platformio.ini (see above)

Scanning

When scanning, results are sent in real time to the base topic plus scanning

Example received on everblu/cyble-22-0828979-espbf84/scanning

the format is JSON

{ 
    "ts": 1689416825,
	"date": "Sat Jul 15 12:27:05 2023", 
	"frequency": "433.8226", 
    "register", "0x10AE7C",
    "rssi": -87,
    "lqi": -120,
	"result": 115
}

result is below 0 on fail, -2 if data can't be read, -1 if read but saw an error else it's #reads of the counter (so positive)

End of Scan

When scanning done results are sent to the base topic plus scan

Example received on everblu/cyble-22-0828979-espbf84/scan

If frequency is 0 this mean no working frequency has been found.

{ 
    "ts": 1689419164,
	"date": "Sat Jul 15 13:06:04 2023", 
	"frequency": "0.0000",
}

If frequency is found it will be the center frequency of the working frequencies boundaries. Boundaries will be sent also.

{ 
    "ts": 1689419164,
	"date": "Sat Jul 15 13:06:04 2023", 
	"frequency": "433.8000", 
	"min": "433.7900", 
	"max": "433.8100"
}

Next Wake

Before going into deep sleep until next try or programmed hour WAKE_HOUR of platformio.ini the device will send a JSON message to topic sleep_until

Example received on everblu/cyble-22-0828979-espbf84/sleep_until

{
    "seconds": 29526,
    "ts": 1689656406,
    "date": "Tue Jul 18 07:00:06 2023"
}

Troubleshooting

Frequency adjustment

Your transreciver module may be not calibrated correctly, please find working frequency enabling a scan by setting FORCE_SCAN in platformio.ini

    -D FORCE_SCAN ; // Force a frequency scan on boot

Once found device will save frequency and use it on each boot.

Business hours

Your meter may be configured in such a way that is listens for request only during hours when data collectors work - to conserve energy. If you are unable to communicate with the meter, please try again during business hours.

{
    "hours":"06:18" 
}

Mine is working from (06-18) and this results looks like LocalTime (tested in summer) because working form 06:00 to 18:59 but yours may be different. I would suggest on first try to test between 10:00 and 16:00.

Anyway mine can't be read on Saturday/Sunday but works on off day such as July 14th.

Serial number starting with 0

Please ignore the leading 0, provide serial in configuration without it.

Save power

The meter has internal battery, which should last for 10 years when queried once a day.

As basic advice, Please do not query your meter more than once a day

According Water manager here they need to change about 10/15 on each measure session, my previous one from 2017 was not working anymore, now they came and put a new one.

Origin and license

This code is based on code from lamaisonsimon

The license is unknown, citing one of the authors (fred):

I didn't put a license on this code maybe I should, I didn't know much about it in terms of licensing. this code was made by "looking" at the radian protocol which is said to be open source earlier in the page, I don't know if that helps?

Links

There is a very nice port to ESP8266/ESP32: https://github.com/psykokwak-com/everblu-meters-esp8266

Origin and License

Software original code (but also all the hard work to get thingd working was originaly done here then put on github by @neutrinus here.

The license is unknown, citing one of the authors (fred):

I didn't put a license on this code maybe I should, I didn't know much about it in terms of licensing. this code was made by "looking" at the radian protocol which is said to be open source earlier in the page, I don't know if that helps?

Misc

See news and other projects on my blog

everblu-meters-esp32's People

Contributors

davvarrr avatar gszabados avatar hallard avatar neutrinus avatar psykokwak-com avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Forkers

amm0nite

everblu-meters-esp32's Issues

Problème de configuration pour ESP32WROOM32

Bonjour,

Je fais des recherches sur la relève des compteurs avec everblu depuis quelques jours et les améliorations que vous avez apportées pour le déploiement sur ESP32 m'intéressent.
J'ai tester le code de neutrinus sur github, je pense avoir le bon câblage au niveau des pins, le programme avait l'air de bien tourner, wifi et mqtt ok, pas de soucis pour lancer le scan, mais je n'ai jamais réussi a trouver la bonne fréquence.

J'ai comme matériel un ESP32WROOM32 (de Az-delivery), et pour le module RF une E07 (M1101D-SMA).

J'ai réussi à installer votre code cet après-midi et j'ai quelques questions si ça ne vous dérange pas et que vous avez le temps d'y répondre.

Pour installer le code sur une autre carte, esp32dev dans mon cas, dans le fichier platformio.ini ligne 20 il faut bien modifier le default_envs = c3-cc1101 par default_envs = esp32dev ?
c'est ce que j'ai fait, mais pendant la compilation j'ai eu des erreurs dans everblu-meters.cpp et cc1101.cpp, concernant des variables non déclarées dans le scope.

Dans everblu-meters.cpp j'ai mis en commentaire :

 /*   // this resets all the neopixels to an off state
    strip.Begin();
    strip.Show();
    delay(100); // Needed to initialize
    DotStar_Clear();
    DotStar_SetBrightness( MY_RGB_BRIGHTNESS );   
    Wire.begin(I2C_SDA, I2C_SCL);
*/

Dans cc1101.cpp, GDO0 était considéré comme non déclaré
J'ai rajouté la configuration des pins dans platformio.ini :

[env:esp32dev]
extends = env:common-esp32
board = esp32dev
lib_deps = ${env:common-esp32.lib_deps}
build_flags = ${env:common-esp32.build_flags}
    -D SerialDebug=Serial
    -D GDO0=2 -D GDO2=4         ; CC1101 wiring
    -D SPI_SS=5 -D SPI_MOSI=23   ; SPI 
    -D SPI_CSK=18 -D SPI_MISO=19  ; SPI 

Après ces modifications j'ai pu compiler et charger le code.
Je pense avoir réussi a trouver la bonne fréquence, j'ai eu 2 confirmations via mqtt :
{"date":"10/10/2023 16:06:20","ts":1696946780,"min":"433.8599","max":"433.8663","frequency":"433.8631"}
{"date":"10/10/2023 16:27:06","ts":1696948026,"min":"433.8524","max":"433.8647","frequency":"433.8583"}

Je suis un peu dubitatif sur la différence des résultats obtenus ...

Pour le moment le module est débranché, quand je le redémarrerai va-t-il recommencer un scan ou va-t-il utiliser la fréquence trouvée ? le paramètre -D FORCE_SCAN (ligne 42 dans platformio.ini) étant toujours actif dans mon ESP32 sans rechargement du code.

J'ai refais la manipulation en reprenant le code d'origine, j'ai juste changé default_envs = esp32dev et renseigné les infos pour le wifi, le mqtt, et le serial pour l'everblu.
Ca m'affiche ces erreurs dans VisualCode.
Capture

EDIT 11/10/23

J'ai rebrancher mon ESP ce matin sur un chargeur usb, il a relancé automatiquement le scan, je l'ai donc laissé faire.
Une fois la fréquence trouvée j'ai reçu ce message mqtt :
{"date":"11/10/2023 10:00:48","ts":1697011248,"min":"433.8353","max":"433.8706","frequency":"433.8528"}

Puis l'ESP est entré en mode sleep (le programme considère que l'esp fonctionne sur batterie comme l'espc3).
j'ai relancer l'esp avec le bouton RST, et là j'ai reçu ce message mqtt :
{"ts":1697011251,"date":"Wed Oct 11 10:00:51 2023","esp_battery":{"percent":0,"vbat":0},"liters":344078,"battery":141,"read":66,"rssi":-76,"lqi":-119,"hours":"06:18"}

La relève s'est donc bien passé, c'est bon signe ! mais ce dernier message mqtt aurait du être envoyé avant le mode sleep en même temps que la confirmation de la fréquence je pense ?
Ensuite l'esp a recommencé à faire un scan !

J'ai donc désactivé le paramètre ;-D FORCE_SCAN (ligne 42 dans platformio.ini) et changé la fréquence dans everblu-meters.cpp ligne 423 (c'est bien le bon endroit pour modifier la fréquence ?)
float frequency = preferences.getFloat("frequency", 433.85280f);

Compilation, recharge du code dans l'ESP, et là pas de lecture possible ...
{ "ts": 1697012483, "date": "Wed Oct 11 10:21:23 2023", "esp_battery": { "percent": 0, "vbat": 0 }, "type": "No Data", "retries": 5 }

Voilà où j'en suis pour le moment, j'ai pas envie de vider la pile de l'everblu donc j'évite de faire trop de tests inutiles.
Je vais réessayer dans la semaine un scan de fréquence en plaçant l'ESP au plus proche du compteur.

EDIT 12/10/23

Aujourd'hui nouveau test de relève ==> Echec

Reload du firmware avec -D FORCE_SCAN activé.
J'ai branché l'ESP sur un powerbank pour le poser juste au dessus du regard du compteur d'eau.

Test de fréquence OK, mais relève NOK

{"date":"12/10/2023 11:34:28","ts":1697103268,"min":"433.7691","max":"433.8329","frequency":"433.8008"}
{"ts":1697103272,"date":"Thu Oct 12 11:34:32 2023","esp_battery":{"percent":0,"vbat":0},"type":"No Data","retries":4}

Il a alors relancé automatiquement un nouveau scan OK, et cette fois-ci une relève OK

{"date":"12/10/2023 11:50:38","ts":1697104238,"min":"433.7691","max":"433.7992","frequency":"433.7841"}

{"ts":1697104242,"date":"Thu Oct 12 11:50:42 2023","esp_battery":{"percent":0,"vbat":0},"liters":344367,"battery":141,"read":98,"rssi":-24,"lqi":-128,"hours":"06:18"}

J'ai réussi à faire 2 relèves, uniquement en mode "scanning", avec 2 fréquences trouvées ... 433.8528Mhz et 433.7841Mhz.

Ensuite j'ai rechargé le firmware en mode "normal" (;-D FORCE_SCAN) et en remettant la fréquence par défaut 433.82000Mhz.

--- Terminal on COM5 | 115200 8-N-1
--- Available filters and text transformations: colorize, debug, default, direct, esp32_exception_decoder, hexlify, log2file, nocontrol, printable, send_on_enter, time
--- More details at https://bit.ly/pio-monitor-filters
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H
18:51:05.060 > ts Ju[  2026][E][Preferences.cpp:503] getBytesLength(): nvs_get_blob len fail: frequency NOT_FOUND
18:51:05.060 >
18:51:05.060 > ================================
18:51:05.060 > Device Name   : cyble-20-0507012-espxxxxx
18:51:05.060 > Wakeup by     : not from sleep:0
18:51:05.060 > Meter Year    : 20
18:51:05.060 > Meter Serial  : 507012
18:51:05.060 > SPI Speed     : 1.0MHz
18:51:05.060 > Frequency     : 433.8200MHz
18:51:05.060 > Retries left  : 5/6
18:51:05.060 > Retries delay : 15
18:51:05.060 > ===========================
18:51:05.060 > CC1101 Version : 0x0014
18:51:05.060 > Found CC1101
18:51:05.060 > Trying to connect WiFi.
18:51:05.060 > WiFi: Connecting to wifixxx ... (2.936000s)
18:51:05.060 > ..WiFi: Connected (3.938000s), ip : 192.168.1.11
18:51:05.060 > [  3944][E][WiFiServer.cpp:91] begin(): ---------------- IPv6
18:51:05.060 > [  3946][E][WiFiServer.cpp:86] begin(): ---------------- IPv4
18:51:05.060 > WEB: Updater ready, open http://cyble-20-0507012-espxxxxx.local in your browser and login with username 'xxxxx' and password 'xxxxx'.
18:51:05.060 > *MQTT: Connecting to broker "192.168.1.46" with client name "cyble-20-0507012-espxxxxx" and username "xxxxx" ... (4.480000s) - ok. (4.604000s)
18:51:05.060 > ****** Connected to MQTT Broker *****
18:51:05.060 > *
18:51:05.060 > Connected in 3s
18:51:05.060 > Waiting for time sync
18:51:05.834 > ***** Got time adjustment from NTP *****
18:51:05.839 > Next wake tomorrow at October 13 2023 15:00:00 (in 72535 seconds)
18:51:05.991 > 
18:51:05.992 > Synced in 3s
18:51:05.994 > Setting to 433.820007.4fMHz
18:51:05.998 > Trying Reading #1 out of 6 from meter
18:51:06.007 > MARCSTATE : raw:0x13  0x13 free_byte:0x0F sts:0x02 sending 2s WUP...
18:51:07.997 > txbuffer:
18:51:08.001 > 0x50, 0x00, 0x00, 0x00, 0x03, 0xFF, 0xFF, 0xFF, 0xFF, 0x64, 0x70, 0x47, 0x00, 0x75, 0x17, 0x14,
18:51:08.008 > 0x77, 0x07, 0x1E, 0xF1, 0x0F, 0x00, 0x75, 0x17, 0x02, 0x72, 0x87, 0x05, 0x71, 0x47, 0x00, 0x72,
18:51:08.017 > 0x87, 0x01, 0x77, 0xEF, 0x6A, 0xFF, 0xFF,
18:51:08.189 > 204 free_byte:0x0C sts:0x07
18:51:08.219 > 
18:51:08.220 > size_byte=18  l_radian_frame_size_byte=25
18:51:08.369 > TMO on REC
18:51:08.399 > 
18:51:08.401 > size_byte=124  l_radian_frame_size_byte=171
18:51:09.100 > TMO on REC
18:51:09.103 > No data, are you in business hours?
18:51:09.108 > MQTT << [everblu/cyble-20-0507012-espxxxxx/error] {"ts":1697129468,"date":"Thu Oct 12 18:51:08 2023","esp_battery":{"percent":0,"vbat":0},"type":"No Data","retries":1}
18:51:09.576 > 5 retries left out of 6
18:51:12.610 > MQTT << [everblu/cyble-20-0507012-espxxxxx/sleep_until] {"seconds":15,"ts":1697129487,"date":"Thu Oct 12 18:51:27 2023"}18:51:12.661 > Going to deep sleep mode for 15 seconds

	*************************J'ai enlevé le milieu du serial monitoring************************************************************

18:53:04.303 > MQTT << [everblu/cyble-20-0507012-espxxxxx/sleep_until] {"seconds":15,"ts":1697129598,"date":"Thu Oct 12 18:53:18 2023"}18:53:04.353 > Going to deep sleep mode for 15 seconds
18:53:20.596 > ets Jul 29 2019 12:21:46
18:53:20.601 >
18:53:20.601 > rst:0x5 (DEEPSLEEP_RESET),boot:0x17 (SPI_FAST_FLASH_BOOT)
18:53:20.605 > configsip: 0, SPIWP:0xee
18:53:20.607 > clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
18:53:20.614 > mode:DIO, clock div:2
18:53:20.617 > load:0x3fff0030,len:184
18:53:20.715 > load:0x40078000,len:12732
18:53:20.715 > ␛[33m=> 0x40078000: ?? ??:0␛[0m
18:53:20.715 > ho 0 tail 12 room 4
18:53:20.715 > load:0x40080400,len:2908
18:53:20.998 > ␛[33m=> 0x40080400: _init at ??:?␛[0m
18:53:20.998 > entry 0x400805c4
18:53:20.998 > ␛[33m=> 0x400805c4: ?? ??:0␛[0m
18:53:22.678 > [  2027][E][Preferences.cpp:503] getBytesLength(): nvs_get_blob len fail: frequency NOT_FOUND
18:53:22.688 >
18:53:22.688 > ================================
18:53:22.692 > Device Name   : cyble-20-0507012-espxxxxx
18:53:22.696 > Wakeup by     : timer
18:53:22.697 > Meter Year    : 20
18:53:22.699 > Meter Serial  : 507012
18:53:22.703 > SPI Speed     : 1.0MHz
18:53:22.705 > Frequency     : 433.8200MHz
18:53:22.705 > Retries left  : 1/6
18:53:22.707 > Retries delay : 15
18:53:22.709 > ===========================
18:53:22.711 > CC1101 Version : 0x0014 
18:53:22.713 > Found CC1101
18:53:22.957 > Trying to connect WiFi.
18:53:23.515 > WiFi: Connecting to wifixxx ... (2.860000s) 
18:53:23.961 > ..WiFi: Connected (3.861000s), ip : 192.168.1.11 
18:53:24.519 > [  3867][E][WiFiServer.cpp:91] begin(): ---------------- IPv6
18:53:24.525 > [  3868][E][WiFiServer.cpp:86] begin(): ---------------- IPv4
18:53:24.531 > WEB: Updater ready, open http://cyble-20-0507012-espxxxxx.local in your browser and login with username 'xxxxx' and password 'xxxxx'.
18:53:25.003 > *MQTT: Connecting to broker "192.168.1.46" with client name "cyble-20-0507012-espxxxxx" and username "xxxxx" ... (4.403000s) - ok. (4.846000s) 
18:53:25.947 > ****** Connected to MQTT Broker *****
18:53:26.449 > *
18:53:26.501 > Connected in 3s
18:53:26.503 > Waiting for time sync
18:54:26.756 > Unable to sync time in 60s
18:54:54.607 > Going to deep sleep mode for 3600 seconds

J'ai aussi essayé le mqtt.enableHTTPWebUpdater(); // Enable the web updater., ça fonctionne bien.
Par contre mqtt.enableDebuggingMessages(); // Enable debugging messages sent to serial output est bien censé envoyé les messages de serial monitor par MQTT ? je ne crois pas que ça fonctionne pas.
//mqtt.enableLastWillMessage(String(topic + "lastwill").c_str(), "offline", true); empêche la connexion au broker mqtt.

mqtt.enableOTA(); // Enable OTA (Over The Air) updates. Je ne comprends pas comment ça fonctionne.

HA MQTT no discovery

HEllo @hallard ,

Beau travail sur la réalisation de ce code. J'aurais cependant une question.
Je récupère bien les data en heures ouvrés

15:14:13.020 >
15:14:13.021 > Synced in 0s
15:14:13.021 > Setting to 433.820007.4fMHz
15:14:13.025 > Reading data from meter
15:14:16.087 > frame received (689)
15:14:16.089 > Consumption : 135 Liters
15:14:16.093 > Battery left : 175 Months
15:14:16.094 > Read counter : 19 times
15:14:16.097 > Working hours : from 06H to 18
15:14:16.100 > Local Time : Thu Oct 26 15:14:18 2023
15:14:16.103 > RSSI / LQI : -67dBm / -128
15:14:19.645 > Going to deep sleep mode for 85545 seconds

Mon ESP se connecte bien à "Mosquitto broker"

2023-10-28 01:08:55: New client connected from 192.168.1.16:49211 as cyble-23-0303414-esp517c (p2, c1, k15, u'homeassistant').

J'arrive bien à ecouter un sujet via MQTT
image

Mais aucune entrée ne se créé dans MQTT ( à l'inverse du code de psykokwak-com)
J'ai beau retourner/tester dans tous les sens, rien n'apparait.
Aurais tu une idée ? Merci

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.