[BUG] SDK does not send Read DataReports with "suppressResponse=true"

[Apollon77]

[BUG] SDK does not send Read DataReports with "suppressResponse=true"

[bzbarsky-apple]

And for item 1, the "that initiates a Report transac­tion" wording is extremely important. It's not all empty data reports. It's specifically empty data reports that start a new Report transaction. That is, only things that are doing "8.6.2. Report Transaction Empty" from the main Matter spec. This is the "we got to max-interval for the subscription and have nothing to report" case.

There can be empty ReportData actions in other cases (wildcard read, subscribe that sent DataVersions that match current state) where this spec clause does not apply at all.

[bzbarsky-apple]

Given that clients do not send responses to either report

Clients absolutely send responses to some data reports. For example, non-last chunks during chunking, and all reports during subscription priming.

I created https://github.com/CHIP-Specifications/connectedhomeip-spec/pull/9996 to make the spec here clearer.

[bzbarsky-apple]

@Apollon77 When you say "first mandate", do you mean this bit:

MAY be set to TRUE for a Report Data action that initiates a Report transac­tion that conveys an empty list

or this bit:

SuppressResponse SHALL be set to TRUE for a Report Data action that is part of a Read transaction.

?

[Apollon77]

I mean the "SHALL" because the "MAY" is just valid for empty data reports how I read it. But in fact when device do not "opt in" to send suppressResponse=true for empty reports (what it does not do when I understood the code right) then the "SHALL" in the next line becomes effective required for the empty Datareport which would mean the "true" again ... so in fact I read this as

• Read DataReport-> SHALL always send "suppressResponse=true"
• Subscription DataReport -> empty DR MAY be "suppressResponse=true", else SHALL be "=false"

Maybe this part should be rephrased ? ;-)

[bzbarsky-apple]

What the current spec says (and I agree maybe it could be clearer in terms of phrasing, but it is clearly and unambiuously defined right now) is this:

1. Read DataReport -> SHALL always send "SuppressResponse=true"
2. Subscription DataReport that is not part of the priming sequence and is empty -> MAY send "SuppressResponse=true".

The SDK implements item 1, and never sets SuppressResponse=true for item 2, so the SDK is spec-compliant as far as I can tell here....

[bzbarsky-apple]

Or put another way, there are three transaction types that can have a ReportData action:

1. Report transaction (incremental subscription update): MAY have the value true if no data being reported.
2. Read transaction: SHALL have the value true.
3. Subscribe transaction (priming reports): SHALL have the value false (because the SubscribeResponse needs to follow the StatusReport).

and that is exactly the order the spec lists them in.

[Apollon77]

Ok, thats interesting ... I used a real matter device (Onevis) which I assume uses chip sdk. I can also verify with a chip device if you like ....

2024-06-26 15:58:10.056 DEBUG  InteractionClient    Sending read request to udp://fddc:3d12:3702:1:5297:c984:d888:279a:5540 on session secure/30654 for attributes */*/* and events undefined
2024-06-26 15:58:10.059 DEBUG  ExchangeManager      Message » id: 44676/36155/108400825 type: 1/2 acked: undefined reqAck: true duplicate: false payload: 153600171818290324ff0b18
2024-06-26 15:58:10.255 DEBUG  MessageExchange      Message « id: 30654/36155/235879994 type: 1/5 acked: 108400825 reqAck: true duplicate: false payload: 1536011535012600de56d41c370124020024031d2504fdff1824020118181535012600de56d41c370124020024031d24040018360215240016240101181818181535012600de56d41c370124020024031d240401183602041d041f0428043004310433043c043e043f0435042a1818181535012600de56d41c370124020024031d24040218360204291818181535012600de56d41c370124020024031d24040318360204011818181535012600de56d41c370124020024031d2504fcff1824020018181535012600de56d41c370124020024031d2504f8ff1836021818181535012600de56d41c370124020024031d2504f9ff1836021818181535012600de56d41c370124020024031d2504fbff18360205f8ff05f9ff05fbff05fdff040004010402040305fcff1818181535012600941b3b26370124020024031f2504fdff1824020118181535012600941b3b26370124020024031f24040018360215240105240202360307248f2869cd0dde7618340424fe01181818181535012600941b3b26370124020024031f2404021824020418181535012600941b3b26370124020024031f2404041824020418181535012600941b3b26370124020024031f2404031824020318181535012600941b3b26370124020024031f2504fcff1824020018181535012600941b3b26370124020024031f2504f8ff1836021818181535012600941b3b26370124020024031f2504f9ff1836021818181535012600941b3b26370124020024031f2504fbff18360205f8ff05f9ff05fbff05fdff040004020404040305fcff18181815350126004c48533237012402002403282504fcff18240200181815350126004c485332370124020024032824040018240201181815350126004c4853323701240200240328240406182c02025858181815350126004c4853323701240200240328240401182c02054f6e766973181815350126004c48533237012402002403282404021825023d14181815350126004c4853323701240200240328240403182c02025334181815350126004c48533237012402002403282404041825020110181815350126004c485332370124020024032824040718240201181815350126004c4853323701240200240328240408182c020131181815350126004c485332370124020024032824040918260290bd9800181815350126004c485332370124020024032824040a182c0209312e312e302d333135181815350126004c485332370124020024032824041318350224000324010318181815350126004c48533237012402002403282504fdff18240201181818290324ff0118
2024-06-26 15:58:10.272 DEBUG  InteractionMessenger Received DataReport chunk with 31 attributes and 0 events, suppressResponse: undefined, moreChunkedMessages: true
2024-06-26 15:58:10.273 DEBUG  ExchangeManager      Message » id: 44676/36155/108400826 type: 1/1 acked: 235879994 reqAck: true duplicate: false payload: 1524000024ff0b18
2024-06-26 15:58:10.486 DEBUG  MessageExchange      Message « id: 30654/36155/235879995 type: 1/5 acked: 108400826 reqAck: true duplicate: false payload: 15360115350126004c4853323701240200240328240405182c02200000000000000000000000000000000000000000000000000000000000000000181815350126004c4853323701240200240328240412182c021031394144373543314644353334304446181815350126004c48533237012402002403282504f8ff18360218181815350126004c48533237012402002403282504f9ff18360218181815350126004c48533237012402002403282504fbff18360205f8ff05f9ff05fbff05fcff040004060401040204030404040704080409040a041305fdff0405041218181815350126009971f45537012402002403302504fcff18240200181815350126009971f455370124020024033024040218240200181815350126009971f455370124020024033024040318240200181815350126009971f455370124020024033024040118350224003c2501840318181815350126009971f4553701240200240330240404182902181815350126009971f45537012402002403302504fdff18240201181815350126009971f455370124020024033024040018240200181815350126009971f45537012402002403302504f8ff18360204010403040518181815350126009971f45537012402002403302504f9ff18360204000402040418181815350126009971f45537012402002403302504fbff18360205f8ff05f9ff05fbff05fcff040204030401040405fdff04001818181535012600f35bb4c837012402002403312404001824020118181535012600f35bb4c837012402002403312404011836021530000820112014020519772901181818181535012600f35bb4c837012402002403312404021824020a18181535012600f35bb4c837012402002403312404031824021418181535012600f35bb4c8370124020024033124040418290218181535012600f35bb4c837012402002403312404051824020018181535012600f35bb4c8370124020024033124040618300208201120140205197718181535012600f35bb4c8370124020024033124040718340218181535012600f35bb4c837012402002403312504fcff1824020218181535012600f35bb4c837012402002403312504fdff1824020118181535012600f35bb4c837012402002403312504f8ff1836020401040504071818181535012600f35bb4c837012402002403312504f9ff183602040004030404040604081818181535012600f35bb4c837012402002403312504fbff1836020400040104020403040404050406040705f8ff05f9ff05fbff05fcff05fdff18181815350126007149efa5370124020024033324040018360218181818290324ff0118
2024-06-26 15:58:10.496 DEBUG  InteractionMessenger Received DataReport chunk with 29 attributes and 0 events, suppressResponse: undefined, moreChunkedMessages: true
2024-06-26 15:58:10.497 DEBUG  ExchangeManager      Message » id: 44676/36155/108400827 type: 1/1 acked: 235879995 reqAck: true duplicate: false payload: 1524000024ff0b18
2024-06-26 15:58:10.649 DEBUG  MessageExchange      Message « id: 30654/36155/235879996 type: 1/5 acked: 108400827 reqAck: true duplicate: false payload: 15360115350126007149efa5370124020024033324040034051835022c00064f545f44454629013402340330040600000000000036051004000000001836061010fe80000000000000dcaa98125f275f521010fdb824be22185de5b71bab90dc3310f91010fddc3d12370200015297c984d888279a1824070418181815350126007149efa5370124020024033324040118240206181815350126007149efa53701240200240333240408182802181815350126007149efa537012402002403332504fcff18240200181815350126007149efa537012402002403332504fdff18240201181815350126007149efa537012402002403332504f8ff18360218181815350126007149efa537012402002403332504f9ff183602040018181815350126007149efa537012402002403332504fbff18360204000401040805f8ff05f9ff05fbff05fcff05fdff18181815350126003815a741370124020024033c2504fcff18240200181815350126003815a741370124020024033c24040018240200181815350126003815a741370124020024033c240401183402181815350126003815a741370124020024033c240402183402181815350126003815a741370124020024033c2504fdff18240201181815350126003815a741370124020024033c2504f8ff18360218181815350126003815a741370124020024033c2504f9ff18360204000401040218181815350126003815a741370124020024033c2504fbff18360205f8ff05f9ff05fbff05fcff04000401040205fdff1818181535012600a4489236370124020024033e2504fcff1824020018181535012600a4489236370124020024033e240400183602153001f215300101012402013703241400182604ddc8552b26055dff0240370624150125114c0418240701240801300941041e77bbcb34f55eacd075a6fd10c68fba4150bb2cfcabc6211f40af8211d9699fd5a58f936ab80a2ff215919f2742e5ca2dd1edb11caecccd52c63896121e9601370a350128011824020136030402040118300414b571f06bc91ef0366d167f3f9d71644a4037b34e3005141394ab873fb3b593f679a186712b88708abcdc0418300b405e1cf218d30637cdfb72781bc39fd030cd93c7f16abdde386390ec2deace943af36c77266936be70e4835b1c5727f09b956406cd80616dd5a6ef0fe9eaf083c418340224fe01181818181535012600a4489236370124020024033e2404021824020518181535012600a4489236370124020024033e2404031824020118181535012600a4489236370124020024033e24040118360218181818290324ff0118
2024-06-26 15:58:10.659 DEBUG  InteractionMessenger Received DataReport chunk with 21 attributes and 0 events, suppressResponse: undefined, moreChunkedMessages: true
2024-06-26 15:58:10.660 DEBUG  ExchangeManager      Message » id: 44676/36155/108400828 type: 1/1 acked: 235879996 reqAck: true duplicate: false payload: 1524000024ff0b18
2024-06-26 15:58:10.855 DEBUG  MessageExchange      Message « id: 30654/36155/235879997 type: 1/5 acked: 108400828 reqAck: true duplicate: false payload: 1536011535012600a4489236370124020024033e240401340518350230014104586a50058658ad98b896e5555a4099e5cc1297ab017a420172b49e917aa68264a8cf16842e2d52b876061b11ec6f89df6638440e5f850fb31f0f33455b5129a82502f1ff24030125044c042c050024fe011818181535012600a4489236370124020024033e24040418360210e715300101002402013703241400182604659e552b2605e5d4024037062414001824070124080130094104586a50058658ad98b896e5555a4099e5cc1297ab017a420172b49e917aa68264a8cf16842e2d52b876061b11ec6f89df6638440e5f850fb31f0f33455b5129a8370a35012901182402603004141394ab873fb3b593f679a186712b88708abcdc043005141394ab873fb3b593f679a186712b88708abcdc0418300b4022564f26dcd08ab9fba02e525b60622dfafba26e1be10bc24e5cbe4102ef2dd4ad4af9ff2fe4a28cc69eb1505a6735b185578637b80f189f96d24cac368fda65181818181535012600a4489236370124020024033e2404051824020118181535012600a4489236370124020024033e2504fdff1824020118181535012600a4489236370124020024033e2504f8ff18360204010403040504081818181535012600a4489236370124020024033e2504f9ff183602040004020404040604070409040a040b1818181535012600a4489236370124020024033e2504fbff18360205f8ff05f9ff05fbff05fcff04000402040304010404040505fdff1818181535012600b31a353a370124020024033f2504fcff1824020018181535012600b31a353a370124020024033f2504fdff1824020118181535012600b31a353a370124020024033f2404001836021818181535012600b31a353a370124020024033f2404011836021818181535012600b31a353a370124020024033f2404021824020418181535012600b31a353a370124020024033f2404031824020318181535012600b31a353a370124020024033f2504f8ff183602040204051818181535012600b31a353a370124020024033f2504f9ff18360204000401040304041818181535012600b31a353a370124020024033f2504fbff18360205f8ff05f9ff05fbff05fcff05fdff04000401040204031818181535012600fa2519da37012402002403352504fcff1824020018181535012600fa2519da37012402002403352404001824020f18181535012600fa2519da37012402002403352404011824020518181535012600fa2519da3701240200240335240402182c020d41706f6c6c6f6e54687265616418181535012600fa2519da37012402002403352404031825020122181818290324ff0118
2024-06-26 15:58:10.866 DEBUG  InteractionMessenger Received DataReport chunk with 21 attributes and 0 events, suppressResponse: undefined, moreChunkedMessages: true
2024-06-26 15:58:10.868 DEBUG  ExchangeManager      Message » id: 44676/36155/108400829 type: 1/1 acked: 235879997 reqAck: true duplicate: false payload: 1524000024ff0b18
2024-06-26 15:58:11.114 DEBUG  MessageExchange      Message « id: 30654/36155/235879998 type: 1/5 acked: 108400829 reqAck: true duplicate: false payload: 1536011535012600fa2519da3701240200240335240404182702771905021420112018181535012600fa2519da37012402002403352404051830020940fdb824be22185de518181535012600fa2519da37012402002403352404071836021527001ffb490eb030455a24010c2502000c25034c5b2504f48f2405032006ae2007b124080b240900290a290b290c280d181527008689eb35dc1b0baa240111250200242503100725041f662405032006c12007c0240800240900290a290b290c280d18152700d3ed0e61637542722401192502003c2503853f2504b01e2405032006c92007c9240800240900290a290b290c280d181818181535012600fa2519da37012402002403352404081836021527001ffb490eb030455a2501000c24020324030f24040124050324060324070c29082909181527008689eb35dc1b0baa250100242402092403032404012405032406032407112908290918152700d3ed0e61637542722501003c24020f2403032404012405032406032407192908290918152700525f275f1298aade250100b424022d24033f24040024050024060024070029082809181818181535012600fa2519da3701240200240335240409182602095c393818181535012600fa2519da370124020024033524040a1824024018181535012600fa2519da370124020024033524040b182402f118181535012600fa2519da370124020024033524040c1824025518181535012600fa2519da370124020024033524040d1824020918181535012600fa2519da370124020024033524043b1835022500a00224018f1818181535012600fa2519da370124020024033524043c18300204001fffe018181535012600fa2519da370124020024033524043d1835022900280129022903290429052806290729082909290a290b1818181535012600fa2519da370124020024033524043e18360204000400040004001818181535012600fa2519da37012402002403352504fdff1824020118181535012600fa2519da37012402002403352504f8ff1836021818181535012600fa2519da37012402002403352504f9ff18360204001818181535012600fa2519da37012402002403352504fbff18360205f8ff05f9ff05fbff05fcff040004010402040304040405040704080409040a040b040c040d043b043c043d043e05fdff1818181535012600003db59d370124020024032a2504fcff1824020018181535012600003db59d370124020024032a2404001836021818181535012600003db59d370124020024032a2504fdff1824020118181535012600003db59d370124020024032a24040118290218181535012600003db59d370124020024032a24040218240201181818290324ff0118
2024-06-26 15:58:11.126 DEBUG  InteractionMessenger Received DataReport chunk with 22 attributes and 0 events, suppressResponse: undefined, moreChunkedMessages: true
2024-06-26 15:58:11.128 DEBUG  ExchangeManager      Message » id: 44676/36155/108400830 type: 1/1 acked: 235879998 reqAck: true duplicate: false payload: 1524000024ff0b18
2024-06-26 15:58:11.301 DEBUG  MessageExchange      Message « id: 30654/36155/235879999 type: 1/5 acked: 108400830 reqAck: true duplicate: false payload: 1536011535012600003db59d370124020024032a24040318340218181535012600003db59d370124020024032a2504f8ff1836021818181535012600003db59d370124020024032a2504f9ff18360204001818181535012600003db59d370124020024032a2504fbff18360205f8ff05f9ff05fbff05fcff040005fdff040104020403181818153501260042f0e828370124020124031d2504fdff182402011818153501260042f0e828370124020124031d2404001836021525000a0124010218181818153501260042f0e828370124020124031d240401183602041d0403040404050406181818153501260042f0e828370124020124031d240402183602181818153501260042f0e828370124020124031d240403183602181818153501260042f0e828370124020124031d2504fcff182402001818153501260042f0e828370124020124031d2504f8ff183602181818153501260042f0e828370124020124031d2504f9ff183602181818153501260042f0e828370124020124031d2504fbff18360205f8ff05f9ff05fbff05fdff040004010402040305fcff1818181535012600755f109f37012402012403032504fcff1824020018181535012600755f109f37012402012403032504fdff1824020418181535012600755f109f37012402012403032404001824020018181535012600755f109f37012402012403032404011824020018181535012600755f109f37012402012403032504f8ff1836021818181535012600755f109f37012402012403032504f9ff183602040004401818181535012600755f109f37012402012403032504fbff18360205f8ff05f9ff05fbff05fcff05fdff0400040118181815350126006c4ce02037012402012403042504fcff18240201181815350126006c4ce02037012402012403042504fdff18240204181815350126006c4ce020370124020124030424040018240280181815350126006c4ce02037012402012403042504f8ff183602040004010402040318181815350126006c4ce02037012402012403042504f9ff18360204000401040204030404040518181815350126006c4ce02037012402012403042504fbff18360205f8ff05f9ff05fbff05fcff05fdff04001818181535012600032e838037012402012403052504fcff1824020018181535012600032e838037012402012403052504fdff1824020418181535012600032e838037012402012403052404001824020018181535012600032e838037012402012403052404011824020018181535012600032e838037012402012403052404021824020018181535012600032e83803701240201240305240403182802181818290324ff0118
2024-06-26 15:58:11.310 DEBUG  InteractionMessenger Received DataReport chunk with 32 attributes and 0 events, suppressResponse: undefined, moreChunkedMessages: true
2024-06-26 15:58:11.311 DEBUG  ExchangeManager      Message » id: 44676/36155/108400831 type: 1/1 acked: 235879999 reqAck: true duplicate: false payload: 1524000024ff0b18
2024-06-26 15:58:11.397 DEBUG  MessageExchange      Message « id: 30654/36155/235880000 type: 1/5 acked: 108400831 reqAck: true duplicate: false payload: 1536011535012600032e838037012402012403052404041824020018181535012600032e838037012402012403052504f8ff1836020400040104020403040404061818181535012600032e838037012402012403052504f9ff18360204000401040204030404040504061818181535012600032e838037012402012403052504fbff18360205f8ff05f9ff05fbff05fcff05fdff040004010402040304041818181535012600c56ab73d37012402012403062504fcff1824020118181535012600c56ab73d37012402012403062504fdff1824020418181535012600c56ab73d370124020124030624040018280218181535012600c56ab73d37012402012403062504004018290218181535012600c56ab73d3701240201240306250401401824020018181535012600c56ab73d3701240201240306250402401824020018181535012600c56ab73d3701240201240306250403401824020018181535012600c56ab73d37012402012403062504f8ff1836021818181535012600c56ab73d37012402012403062504f9ff1836020400040104020440044104421818181535012600c56ab73d37012402012403062504fbff18360205f8ff05f9ff05fbff05fcff05fdff040005004005014005024005034018181818290424ff0118
2024-06-26 15:58:11.403 DEBUG  InteractionMessenger Received DataReport chunk with 14 attributes and 0 events, suppressResponse: true, moreChunkedMessages: undefined
2024-06-26 15:58:11.407 DEBUG  ExchangeManager      Message » id: 44676/36155/108400832 type: 0/16 acked: 235880000 reqAck: false duplicate: false payload: 

As you can see all DataReports to this Read request are send with "suppressResponse=undefined aka false" beside the last one. This is sent with suppressResponse=true .

An eve Energy responds the same

PS: on a subscription response the last DataReport also has suppressResponse=undefiend/false as all others, so that's correct.

[bzbarsky-apple]

In the IM action for a read, the logical SuppressResponse boolean is true.

How that gets encoded when chunking is involved is covered by "10.7.3. ReportDataMessage", but now we do have a spec bug: it needs to have some wording about how "this is set to the value in the action, unless MoreChunkedMessages is true, in which case SuppressResponse must be false". This is obviously the only thing that makes sense, because for chunking you need the response to know when to send the next chunk.

An example that handles this better is "10.7.6. WriteRequestMessage", which says:

SuppressResponse SHALL NOT be set to true if MoreChunkedMessages is true.

So the SDK behavior here looks correct and we need to fix the encoding spec a bit, right?

[cecille]

Given that clients do not send responses to either report, does this flag actually convey any actionable information? For interoperability, we want to make sure the reports are processed properly by clients, so maybe we should just clarify the spec to match what clients in the field expect?

[Apollon77]

Thank you for all your support and the clarification