Not all sensors discovered by HA

[briis]

Not all sensors discovered by HA

[GlennGoddard]

Ok, I will work on that once you finish your testing and I can get all the MQTT topics

[GlennGoddard]

I will test when I get home in a few hours.

[briis]

I assume this is related to issue #4 ?
To get data in to MQTT using MQTT Discovery, you need to have a Config and a State Topic. There will a Config topic per sensor like this for the Solar Radiation: homeassistant/sensor/weatherflow2mqtt/solar_radiation/config and then the states are pushed through obs_air, obs_sky etc with updates for several sensors. The reason there is not 1 state topic or one per sensor, is that I push the data out when they arrive. I am not sure about the Tempest unit, but for the old AIR and SKY, they arrive per device, and I then just split the Tempest unit in the same way.

For the Solar Radiation sensor from above the State will be published in the obs_sky topic, as this is where the data is on the old SKY devices. The topic would look like this:

{
 "illuminance": 6203,
 "uv": 0.07, 
 "rain_today": 0.0,
 "rain_yesterday": 0.1,
 "rain_duration_today": 0,
 "rain_duration_yesterday": 3,
 "rain_start_time": "2021-06-04T13:50:25",
 "wind_lull": 0.0,
 "wind_speed_avg": 0.5,
 "wind_gust": 1.5,
 "wind_bearing_avg": 102,
 "wind_direction_avg": "ESE",
 "battery": 3.28,
 "solar_radiation": 51,
 "precipitation_type":
 "None",
 "rain_rate": 0.0
}

If you still have these issues after resolving Issue #4, I will look at it.

[chasut]

Thanks for the info, makes sense.
At this point I am going to say the two issues are unrelated. I removed the MQTT integration; removed the container; deleted the ".storage.json" file for good measure; restarted HA; re-added the MQTT integration; rebuilt and started the container. Results are that MQTT integration created 20 entities, same as before. All of the created entities appear to be working as expected.

Since I don't know diddly about MQQT, don't know if this is important or not. But just as an observation from MQTT explorer, I am seeing a Config topic for each of the sensors that gets created, but none for those that do not. see attached.

[briis]

Agree, it seems to be unrelated.
All sensors. you don't get are calculated values, except for the last 3. I believe it must be one sensor that is not reporting, and then the program stops adding, without throwing an error. It is. evening here in Denmark, but I will look in to it tomorrow morning, and figure out how we can debug this.

[chasut]

Just poking around this morning while I can't find anything better to do :) Thought I would pass on a couple more observations that I hope will help. If not, well then please ignore me and carry on...

1. In the obs_air topic, none of the values for "lightning_strike_" ever change, they are always 0, and the timestamp is the default value
2. Forecast Attributes are cut-off (incomplete) about a third way through the hourly forecast. This is probably just the MQTT explorer truncating the message, but thought I would mention just in case.
3. Sea Level Pressure is slightly different than what is reported in SmartWeather integration as well as the Tempest app. (e.g. 29.804 inHg v. 29.846) Could be a timing difference, or could be some AI nonsense that WeatherFlow does on their servers. I checked my environment variable for Elevation, and using the reported Elevation + Station Height in meters. That is correct, no?

Ok, sun is finally coming out, so will find something better to do than pester you! Thanks again.

[briis]

I will look at it. Enjoy your day.

[GlennGoddard]

I have similar issues with my Tempest.
Reported firmware by MQTT is 126, reported by app is 143 for hub and 150 for the tempest
Rain always reports 0 for all rain sensors via MQTT. Precip Type is not listed
I had a more extensive post yesterday but computer lost power before actually posting.
Lightning (yesterday) said last strike was 1967, today sensors say 15 today but only one that I can see in weatherflow app (I know they compare other stations for the lightning data for app). I think my lightning count is catching a motion light.
With WireShark I get 4 different message sizes, the most frequent being the rapid wind.
If I get the chance today, I will stop the docker container and run a script that outputs the UDP data.
I have humidity and temperature reported but no dewpoint.
If helpful, I can go thru all sensors like above and list which sensors are visible and reporting.

[briis]

First of all, I just received a Tempest Device today, and will mount it in a few hours, that means that I can finally start testing. Because everything both of you are reporting I have not been able to reproduce here with an AIR and SKY unit.

So give me a few days to test with my new device, and see if I can reproduce some of the errors. I will soon release an updated version, that does better error handling and reporting, so that we hopefully can find out where it fails.

[chasut]

Congrats on the new acquisition! I am still amazed that you got as far as you did without having a real Tempest device to test. Enjoy your new device.

[GlennGoddard]

I only got mine 3 days ago, so I can forgive you for now.

On a separate note, do you think it would be of benefit to add the MQTT topic to the chart in the readme? I kinda started doing that for my own troubleshooting, that is how I came across the issues in the chart that were in the last PR.

[briis]

Thanks both.
@GlennGoddard Yes, I think it would be beneficial to add the different MQTT topics and what they do. Thanks the suggestion.

[briis]

I have now just added more and better error handling, so hopefully that will help in debugging. Not published yet, but will do tomorrow morning, once I tested locally.

Now off to mounting my new Tempest device😬

[GlennGoddard]

Are you utilizing the error codes that are published in the UDP API at all? I could look, but it is easier (for me) to just ask.

[briis]

If you talk about the Sensor Status, yes I record these and output them the error log in Docker.

[briis]

Hi both,
I believe I have found and fixed a few errors.
Fortunately I could replicate the missing sensor issue, so I have adjusted a few MQTT settings, and now it runs at my station. Hope this works for you to.

Also the Forecast Topic is now complete. There is a limit to how much data I can pass in one topic. So I adjusted the number of hours to 36 (from 48), and now it is a complete message.

There is an updated container for Intel, so if you run that, just pull the updated container. Else pull the source, and build.

Let me know how this goes.

[simonranson]

Its looking good on the Pi. I have 30 sensors now showing. The only two missing are Uptime and Feels Like Temperature

Cheers
Simon

[chasut]

Fantastic! I am up and running, still missing feelslike, and the weather (forecast). I turned on debugging but didn't see anything remarkable in the logs. All three show up in the logs with "Setting up..." but nothing else. I cannot find a "config" record in the MQTT for these (all others have one), although the state information is coming across just fine ( feels like in the obs_air topic, and weather in forecast.

Any suggestions on what to try?
Thanks

[briis]

It is a parameter on the MQTT client, I know where to put a number, just not how high it has to be. - I have just made an adjustment again, as I was also missing the weather forecast, and now I have that back.

The issue is, it seems inconsistent, as I have actually been running fine for a while, but after a restart of the container, some sensors were gone.

So, sorry, but please try updating again.

[chasut]

That did it, thank you. I now have all the expected sensors.

Is the parameter related to the message/topic size? That weather topic is large and probably varies greatly with each update. Maybe break it into two sensors (Daily/Hourly)?

In any case, well done!

[briis]

It is a parameter called max_inflight_messages_set which sets the maximum number of messages with QoS>0 that can be part way through their network flow at once.

The big difference between AIR & SKY and TEMPEST, is that Tempest delivers more data at once, than the other two, and that might cause the message Queue to clutter.

The default for this parameter is 20, and I tried 40 (which was what you also tried) and now I bumped it up to 240. That should be enough 😁

Let it run for a few days, and if stable, just close the issue.

[GlennGoddard]

From personal experience I have found adding a few milliseconds delay is required if publishing a lot to MQTT. I think I have that updated at least one of my repos (either light or salt tank); but I am not even coming close to this amount of data.

Ok, just opened a another GitHub tab, I just sleep for 0.01 between publishes and that ensured all MQTT data was published. It was hit and miss before the sleep delay and it took me a long time to actually figure it out. A delay that short should not have any impact, I can't guarantee that is the issue here but it has been for me in the past; now I just build delays into any MQTT publishes from the begining (mainly since I just plagiarize my own code for new projects).

[briis]

Hi Glenn,
I do believe this could be a good solution. As I wrote above, the big difference between the old devices and a Tempest, is the timing of received data. I had no issues with the sensors, before I installed the Tempest device, and I can see data is comming much more compressed in time.

I have now added sleep after each MQTT publish, so let us see if this gives the stability we need. The Container is updated as of now.
Thanks for the feedback.

[chasut]

On the latest and things look good so far. Will continue to monitor for a day or two as you suggest. Hopefully will get some rain and lightning by then to see how those work. Soon we will be in our summer weather pattern and won't see much of either until October. It is no fun having new toys and not getting to see the lights flash.

btw, going to open a separate issue on the Sea Level Pressure comment above.

[briis]

New toys are great 🤓
The sea level issue is my formula. It is close, but...
Yes please open a separate issue on that. Thanks

[GlennGoddard]

So, let that be a lesson to me to make sure I specify the latest tag; I just pulled 46 updated versions of the docker image.
Anyway...
I have all the sensors.

Forecast:
Forecast temp_high_today and temp_low_today is in C (metric)(not correct for my case) Number convert correctly when I check them, just not converted here.
Daily temp hi/low under daily_forecast and hourly_forecast is F (imperial) (correct for my case).
Wind speed forecasts look good for units.
Pressure forecasts look good for units.
Wind bearing under daily_forecast might have a math issue (not an issue...yet). Reported are 191.041666666666666, 174.33333333333333334 and so on. Daily does not display cardinal direction.
Hourly wind direction displays cardinal and degrees is always an integer (good, reported as comparison to daily).

[briis]

Forecast temp_high_today and temp_low_today is in C (metric)(not correct for my case) Number convert correctly when I check them, just not converted here.

Correct, missing conversion. Has been fixed

Wind bearing under daily_forecast might have a math issue (not an issue...yet). Reported are 191.041666666666666, 174.33333333333333334 and so on

Missing rounding. Fixed

Daily does not display cardinal direction

Wind bearing is not part of the daily dataset received, so I calculate it from the hourly data, by taking an average value of all measurrements. I have now added cardinal direction also to daily

Now off to see if I can fix the Sea Level Pressure

[GlennGoddard]

Ok, I am missing something. The weather sensor with the wind_speed_template has round(2) but I'm occasionally getting 10+ decimal places. This is obviously a HA issue, but something else is not right.

I removed the math from the template and just used the given speed. I assume the calculation was for metric for m/s to km/hr??
Just curious then if those that need to convert have the non-rounding issue.

[briis]

I assume you use Imperial Units, so you don't need to a conversion. For some reason, the Weather Entity expects wind units as km/h, and as this integration delivers metric wind speeds in m/s, we need to do the conversion for metric users only. (Which I tried to write further down 😊)

So I am using it, and you are right, occasionally I see a lot of decimal points, but then they go away again. It is not the data delivered from this, but something HA does.

[briis]

This has become a long topic with many different things, and I believe all issues have been resolved. So I will close this now, and then please open new individual issues when something pops up.

For general discussions, we can use the Discussion area in this Github.

Thanks.