NMEA decoding, missing values treated as zero

[quantenschaum]

NMEA decoding, missing values treated as zero

[wellenvogel]

Most probably you are right - I have to go back to some older discussion about this stuff.
But we should not try to mix this with the undefined handling.

Where does the updating of the widget's props happen in the code? I'd like to understand, how that works.

hoc/Dynamic.jsx
Very generic handling that updates properties with values from the global store if they are changing.
Needs some thoughts...
Maybe undefined is not really the way to go .
Currently I'm a bit busy with some other tasks (porting to reactjs 18 with a couple of pitfalls) and some stuff outside AvNav - so will still need some time until digging in deeper.

[wellenvogel]

I agree. But this most probably requires some work as the consequences have to be checked carefully throughout the complete system.
As always there are at least 4 places:
(1) decoder (python)
(2) decoder (java)
(3) front end
(4) signalk handler

[quantenschaum]

But if you do not write missing values to the data store, they time out and get removed, then it is like as if they have never been there in the first place. So, this should be not be a problem for the frontend or SK.

[quantenschaum]

side question: When looking into this I stumbled upon this;

COG, course over ground, has the key track but in the GUI, in the widgets and int the data store it is available as gps.track but in the frontend it is named nav.gps.course, nav.gps.track does not exist. This is pretty confusing. Why?

I found the reason here. Why is it like that?

[quantenschaum]

Values in the data store should expire after a certain time if they are not updated. nav.gps.course does expire and gets removed from the store and becomes undefined in the in the frontend when access in widget code via storeKeys: { COG: 'nav.gps.course' }. Other values like nav.gps.sail_instrument.COG seem also to get removed from the store, but are still available in frontend code?

The COG widget displays --- if there is no course present, but the course vector is still shown with direction 000.

[wellenvogel]

yes, as I said - would need to go through all the places.
Maybe even needs some ideas how to show course vector/boat direction...

[quantenschaum]

Hmm, OK. - But a field becoming undefined (times out) can happen (sender goes offline or leaves the field blank).

Could you help me to understand why nav.gps.course becomes undefined in the frontend when removed from the store, but other fields like nav.gps.sail_instrument.COG do not? When I reload the frontend (F5) these values disappear.

and #347 (comment)

[quantenschaum]

Fields that have expired on the server are no longer present in the data returned by a request to /viewer/avnav_navi.php?request=gps. The field course is explicitly set to be undefined in gpsdata.js. When a widget requests data fields via the storeKeys dict, the expired fields are not removed from the widget's props and the old expired value is retained.

Where does the updating of the widget's props happen in the code? I'd like to understand, how that works.

[quantenschaum]

some ideas how to show course vector/boat direction...

If COG/HDT is undefined

• do not display a course vector and
• display the boat icon as circle/dot (no orientation)

[quantenschaum]

IMHO boat direction = HDM does not make sense. The map north is true north. If you don't have HDT, you cannot display the boat/vector correctly on the map.

[quantenschaum]

Sure, not in a hurry. - I came across this because I observed some unexpected values when my GPS was stationary (it omits COG when not moving) or when it lost its fix. COG was set to 0 in the NMEA parser. After changing this to not updating COG, it expires and is removed from the store. In gpsdata.js the course is explicitly set to undefined, so COG is displayed as --- as expected. Interestingly this also works for other values as well which are not explicitly set to undefined, like HDT. After changing the default value of the HDT widget it also shows --- when SOG expires.

This is the behaviour I prefer. If the value is not present (never was or expired), it should not be displayed as 0 but as --- indicating that there is no such value available. Especially for depth it would be very dangerous to display an old value when the sensor stopped working and does not send any data. Here 0 is displayed as default, which is obviously not correct.

But in the sail_instrument fields like nav.gps.sail_instrument.COG retain their last value when the value has expired and was removed from the store on the server, even though the field no longer contained in the response to /viewer/avnav_navi.php?request=gps. 🤔

The course=HDM issue was just a side note here.

[quantenschaum]

The problem with removing expired keys seems to be in gpsdata.js. Data is updated and written to the global store in writeToStore, the additionalKeys are computed from the received data itself, so if a key is no longer present in the received data, it is not contained in the additionalKeys keys and the old value in the store remains as is. Key that are listed in filteredStoreKeys get removed from the store when expired, which is what I would expect.

[wellenvogel]

Basically gpsdata.js tries to collect all keys in additionalKeys (if they have been received at least once).
This should give exactly what you want - i.e. set the value to undefined in the store if no longer received.
If it has never been received it's not in the store any way.
But maybe there is a bug somewhere...

[quantenschaum]

Ok, that makes sense, but it doesn't behave like this. - When a field expires on the server it is removed from the store and no longer sent to the frontend. The additionalKeys then does no longer contain this certain (nested) keys and it doesn't get removed from the store in the frontend.

[quantenschaum]

The key I am looking at is nav.gps.sail_instrument.COG. The problem seems to be that the data structure is not flat, but additionalKeys['sail_instrument']={...} contains a dict which is overwritten in handleGpsResponse. Merging existing dicts does solve this.