sunspec/pysunspec2

SunSpec Python library for interfacing with SunSpec devices.

pysunspec2's Issues

aggregated devices could not be handled

The SunSpec definition for Common Models says:
Device aggregations. A map may contain multiple Common Blocks. Each Common
Block marks the start of a new device. This is useful when a map represents an
aggregation of devices.

Solaredge does use this feature. In one map they aggregate one inverter and up to 3 ac meters. The inverter and all meters have their seperated common model block. With the current implementation of pysunspec2 the relationship between the common model and the according ac meter model gets lost.
Current wrong behavior:
A scan with such a setup generates one device with 1 to 4 items for common in the model list and 0 to 3 ac_meter items and some more for the inverter.
Expected behavior:
scan should generate 1 to 4 devices; one for the inverter and up to 3 for the ac_meters

Error reading in missing model during scan

Pad point value is not being read in

Update pytests

Problems in connection TCP

Hi,
I get a lot of connection errors when I try to read\write fields.
I used tcpdump to investigate it and I see that the sunspec client is the one that initiates the disconnection.
Here some lines from the tcpdump logs:

08:49:03.109412 IP DEVICE_IP > SUNSPEC_CLIENT_IP: Flags [P.], seq 4040936281:4040936298, ack 2089094857, win 1901, options [nop,nop,TS val 1574374142 ecr 602893784], length 17: HTTP: HTTP/1.0 200 OK
08:49:03.109763 IP SUNSPEC_CLIENT_IP > DEVICE_IP: Flags [.], ack 17, win 22, options [nop,nop,TS val 602893846 ecr 1574374142], length 0
08:49:03.117102 IP DEVICE_IP > SUNSPEC_CLIENT_IP: Flags [P.], seq 17:139, ack 1, win 1901, options [nop,nop,TS val 1574374152 ecr 602893846], length 122: HTTP
08:49:03.117672 IP SUNSPEC_CLIENT_IP > DEVICE_IP: Flags [.], ack 139, win 22, options [nop,nop,TS val 602893846 ecr 1574374152], length 0
08:49:03.117913 IP DEVICE_IP > SUNSPEC_CLIENT_IP: Flags [P.], seq 139:144, ack 1, win 1901, options [nop,nop,TS val 1574374152 ecr 602893846], length 5: HTTP
08:49:03.118478 IP SUNSPEC_CLIENT_IP > DEVICE_IP: Flags [.], ack 144, win 22, options [nop,nop,TS val 602893847 ecr 1574374152], length 0
08:49:03.118594 IP SUNSPEC_CLIENT_IP > DEVICE_IP: Flags [F.], seq 1, ack 144, win 22, options [nop,nop,TS val 602893847 ecr 1574374152], length 0

I also used modbus poll to check that my system is fine, and I didn't have any problems.

This problem happened before? Any ideas how to solve it?
Is there something in the creation of the client that I'm missing and causing this problem?
I would appreciate any idea\help....

EDIT: I'm running this on Linux computer. Is there a plan to support and check Linux?
I must use pysunspec2 because I need to check maps 701-713.

Thanks

smdx.from_smdx() does not put `size` attribute in points

pysunspec2/sunspec2/smdx.py

Line 159 in 4405d92

def from_smdx(element):

When using smdx.from_smdx() to convert from smdx files to dictionary, the size attribute is not created.

As a workaround, I found that using smdx.from_smdx() -> spreadsheet.to_spreadsheet() -> spreadsheet.from_spreadsheet() seems to do the job.

However, it would be great if smdx.from_smdx() performed the proper conversion on its own.

Stored sf_value leads to wrong cvalue

If the Point.get_value is called the first time, the sf_value is determined and stored.

When the device data are read again and the device changes the scale factor, the old scale factor gets used again and wrong values are delivered.

The sf_value should be reset on every read.

Custom model directories not being fully removed

Scale factor of 0 is not handled correctly when setting with a float

A point with a scale factor of 0 does not get set correctly when the value is a float. Currently, a value is not converted to an integer when the scale factor is 0.

The line in set_value - if self.sf_value: should be if self.sf_value is not None:

Calling read() on a model gives unexpected results

Reading a model (Group object with self.offset=0) results in reading its ID and Length and the next Length-2 points.
For example: d.common[0].read() will read all the points of the common model except for DA and Pad.
Instead, it could read only the ID and the Length, or ID, Length and the fixed block points.

A simple solution is to change line 85 of client.py:

data = self.model.device.read(self.model.model_addr + self.offset, self.len)

to:

data = self.model.device.read(self.model.model_addr + self.offset, self.len + 2 if self.offset == 0 else 0)

Add is_impl() method to Point class

Add an is implemented (is_impl()) method to the point class that indicates if the point is implemented. A point value of None or the unimplemented value indicate the point is not implemented.

FileClient has problem reading models without a value for the 'L' point

FileClientDevice does not populate repeating block groups

When initializing a FileClientDevice using a JSON file, models converted from SunSpec v1 SMDX do not properly populate. For example, take the JSON below, which contains an instance of Model 129, LVRT. When a FileClientDevice is instantiated from this JSON, the resulting object's Model 129 does not contain any data in the repeating blocks; only the fixed block's data is populated.

Source JSON:

{
    "name": null,
    "did": "fa0a9f2d-d503-470e-8ce8-ec9c51428829",
    "models": [
        {
            "ID": 1,
            "L": 66,
            "Mn": "Device Manufacturer",
            "Md": "Inverter 123",
            "Opt": null,
            "Vr": "v0.0.1",
            "SN": "9999abcd",
            "DA": null,
            "Pad": 32768
        },
        {
            "ID": 129,
            "L": 210,
            "ActCrv": 1,
            "ModEna": 0,
            "WinTms": null,
            "RvrtTms": null,
            "RmpTms": null,
            "NCrv": 4,
            "NPt": 10,
            "Tms_SF": -2,
            "V_SF": -1,
            "Pad": 32768,
            "curve": [
                {
                    "ActPt": 4,
                    "Tms1": 200,
                    "V1": 880,
                    "Tms2": 71,
                    "V2": 650,
                    "Tms3": 20,
                    "V3": 450,
                    "Tms4": 0,
                    "V4": 300,
                    "Tms5": 0,
                    "V5": 0,
                    "Tms6": 0,
                    "V6": 0,
                    "Tms7": 0,
                    "V7": 0,
                    "Tms8": 0,
                    "V8": 0,
                    "Tms9": 0,
                    "V9": 0,
                    "Tms10": 0,
                    "V10": 0,
                    "Tms11": null,
                    "V11": null,
                    "Tms12": null,
                    "V12": null,
                    "Tms13": null,
                    "V13": null,
                    "Tms14": null,
                    "V14": null,
                    "Tms15": null,
                    "V15": null,
                    "Tms16": null,
                    "V16": null,
                    "Tms17": null,
                    "V17": null,
                    "Tms18": null,
                    "V18": null,
                    "Tms19": null,
                    "V19": null,
                    "Tms20": null,
                    "V20": null,
                    "CrvNam": null,
                    "ReadOnly": 1
                },
                {
                    "ActPt": 0,
                    "Tms1": 0,
                    "V1": 0,
                    "Tms2": 0,
                    "V2": 0,
                    "Tms3": 0,
                    "V3": 0,
                    "Tms4": 0,
                    "V4": 0,
                    "Tms5": 0,
                    "V5": 0,
                    "Tms6": 0,
                    "V6": 0,
                    "Tms7": 0,
                    "V7": 0,
                    "Tms8": 0,
                    "V8": 0,
                    "Tms9": 0,
                    "V9": 0,
                    "Tms10": 0,
                    "V10": 0,
                    "Tms11": null,
                    "V11": null,
                    "Tms12": null,
                    "V12": null,
                    "Tms13": null,
                    "V13": null,
                    "Tms14": null,
                    "V14": null,
                    "Tms15": null,
                    "V15": null,
                    "Tms16": null,
                    "V16": null,
                    "Tms17": null,
                    "V17": null,
                    "Tms18": null,
                    "V18": null,
                    "Tms19": null,
                    "V19": null,
                    "Tms20": null,
                    "V20": null,
                    "CrvNam": null,
                    "ReadOnly": 0
                },
                {
                    "ActPt": 0,
                    "Tms1": 0,
                    "V1": 0,
                    "Tms2": 0,
                    "V2": 0,
                    "Tms3": 0,
                    "V3": 0,
                    "Tms4": 0,
                    "V4": 0,
                    "Tms5": 0,
                    "V5": 0,
                    "Tms6": 0,
                    "V6": 0,
                    "Tms7": 0,
                    "V7": 0,
                    "Tms8": 0,
                    "V8": 0,
                    "Tms9": 0,
                    "V9": 0,
                    "Tms10": 0,
                    "V10": 0,
                    "Tms11": null,
                    "V11": null,
                    "Tms12": null,
                    "V12": null,
                    "Tms13": null,
                    "V13": null,
                    "Tms14": null,
                    "V14": null,
                    "Tms15": null,
                    "V15": null,
                    "Tms16": null,
                    "V16": null,
                    "Tms17": null,
                    "V17": null,
                    "Tms18": null,
                    "V18": null,
                    "Tms19": null,
                    "V19": null,
                    "Tms20": null,
                    "V20": null,
                    "CrvNam": null,
                    "ReadOnly": 0
                },
                {
                    "ActPt": 0,
                    "Tms1": 0,
                    "V1": 0,
                    "Tms2": 0,
                    "V2": 0,
                    "Tms3": 0,
                    "V3": 0,
                    "Tms4": 0,
                    "V4": 0,
                    "Tms5": 0,
                    "V5": 0,
                    "Tms6": 0,
                    "V6": 0,
                    "Tms7": 0,
                    "V7": 0,
                    "Tms8": 0,
                    "V8": 0,
                    "Tms9": 0,
                    "V9": 0,
                    "Tms10": 0,
                    "V10": 0,
                    "Tms11": null,
                    "V11": null,
                    "Tms12": null,
                    "V12": null,
                    "Tms13": null,
                    "V13": null,
                    "Tms14": null,
                    "V14": null,
                    "Tms15": null,
                    "V15": null,
                    "Tms16": null,
                    "V16": null,
                    "Tms17": null,
                    "V17": null,
                    "Tms18": null,
                    "V18": null,
                    "Tms19": null,
                    "V19": null,
                    "Tms20": null,
                    "V20": null,
                    "CrvNam": null,
                    "ReadOnly": 0
                }
            ]
        }
    ]
}

Here is the resulting JSON after creating a FileClientDevice based on the file above. Note that all of the repeating block points contain the value null even where there are other values in the source JSON:

{
  "name": null,
  "did": "5690cb49-ed9e-4117-a26d-7e6060a275b0",
  "models": [
    {
      "ID": 1,
      "L": 66,
      "Mn": "Device Manufacturer",
      "Md": "Inverter 123",
      "Opt": null,
      "Vr": "v0.0.1",
      "SN": "9999abcd",
      "DA": null,
      "Pad": 32768
    },
    {
      "ID": 129,
      "L": 210,
      "ActCrv": 1,
      "ModEna": 0,
      "WinTms": null,
      "RvrtTms": null,
      "RmpTms": null,
      "NCrv": 4,
      "NPt": 10,
      "Tms_SF": -2,
      "V_SF": -1,
      "Pad": 32768,
      "curve": [
        {
          "ActPt": null,
          "Tms1": null,
          "V1": null,
          "Tms2": null,
          "V2": null,
          "Tms3": null,
          "V3": null,
          "Tms4": null,
          "V4": null,
          "Tms5": null,
          "V5": null,
          "Tms6": null,
          "V6": null,
          "Tms7": null,
          "V7": null,
          "Tms8": null,
          "V8": null,
          "Tms9": null,
          "V9": null,
          "Tms10": null,
          "V10": null,
          "Tms11": null,
          "V11": null,
          "Tms12": null,
          "V12": null,
          "Tms13": null,
          "V13": null,
          "Tms14": null,
          "V14": null,
          "Tms15": null,
          "V15": null,
          "Tms16": null,
          "V16": null,
          "Tms17": null,
          "V17": null,
          "Tms18": null,
          "V18": null,
          "Tms19": null,
          "V19": null,
          "Tms20": null,
          "V20": null,
          "CrvNam": null,
          "ReadOnly": null
        },
        {
          "ActPt": null,
          "Tms1": null,
          "V1": null,
          "Tms2": null,
          "V2": null,
          "Tms3": null,
          "V3": null,
          "Tms4": null,
          "V4": null,
          "Tms5": null,
          "V5": null,
          "Tms6": null,
          "V6": null,
          "Tms7": null,
          "V7": null,
          "Tms8": null,
          "V8": null,
          "Tms9": null,
          "V9": null,
          "Tms10": null,
          "V10": null,
          "Tms11": null,
          "V11": null,
          "Tms12": null,
          "V12": null,
          "Tms13": null,
          "V13": null,
          "Tms14": null,
          "V14": null,
          "Tms15": null,
          "V15": null,
          "Tms16": null,
          "V16": null,
          "Tms17": null,
          "V17": null,
          "Tms18": null,
          "V18": null,
          "Tms19": null,
          "V19": null,
          "Tms20": null,
          "V20": null,
          "CrvNam": null,
          "ReadOnly": null
        },
        {
          "ActPt": null,
          "Tms1": null,
          "V1": null,
          "Tms2": null,
          "V2": null,
          "Tms3": null,
          "V3": null,
          "Tms4": null,
          "V4": null,
          "Tms5": null,
          "V5": null,
          "Tms6": null,
          "V6": null,
          "Tms7": null,
          "V7": null,
          "Tms8": null,
          "V8": null,
          "Tms9": null,
          "V9": null,
          "Tms10": null,
          "V10": null,
          "Tms11": null,
          "V11": null,
          "Tms12": null,
          "V12": null,
          "Tms13": null,
          "V13": null,
          "Tms14": null,
          "V14": null,
          "Tms15": null,
          "V15": null,
          "Tms16": null,
          "V16": null,
          "Tms17": null,
          "V17": null,
          "Tms18": null,
          "V18": null,
          "Tms19": null,
          "V19": null,
          "Tms20": null,
          "V20": null,
          "CrvNam": null,
          "ReadOnly": null
        },
        {
          "ActPt": null,
          "Tms1": null,
          "V1": null,
          "Tms2": null,
          "V2": null,
          "Tms3": null,
          "V3": null,
          "Tms4": null,
          "V4": null,
          "Tms5": null,
          "V5": null,
          "Tms6": null,
          "V6": null,
          "Tms7": null,
          "V7": null,
          "Tms8": null,
          "V8": null,
          "Tms9": null,
          "V9": null,
          "Tms10": null,
          "V10": null,
          "Tms11": null,
          "V11": null,
          "Tms12": null,
          "V12": null,
          "Tms13": null,
          "V13": null,
          "Tms14": null,
          "V14": null,
          "Tms15": null,
          "V15": null,
          "Tms16": null,
          "V16": null,
          "Tms17": null,
          "V17": null,
          "Tms18": null,
          "V18": null,
          "Tms19": null,
          "V19": null,
          "Tms20": null,
          "V20": null,
          "CrvNam": null,
          "ReadOnly": null
        }
      ]
    }
  ]
}

Modbus exceptions are not handled properly during scan

Modbus exceptions are not handled when trying different Modbus addresses causing the scan to abort early.

Catch Modbus exceptions during scan.

device.py - Point.set_mb() length check is incorrect

In Point.set_mb, the length check if len(data) < mb_len should be if len(data) < mb_len * 2.

Create setup.py file

Easier usage would be welcome.

Hi,

I like that you have added the pysunspec to PYPi and this comment isn't going to come across well, so please don't take it to badly.
From an outsiders perspective

1. The doco says this project has some differences from v1 and because of that, it is not backwards compatible.
   From my point of view, it looks like a total re-write by a different team of developers and has breaking changes at all level of the code and that If I want to output the data from my inverter, my reading code will be a complete rewrite.

2. I had a simple read code from v1 that was loops within loops. The model was generic and so to was converting the data from one format to another. The whole model -> Block -> Points was easy to visualise in ones head. Now the blocks are called groups but they are now dictionaries, with a group called "Curve", which isn't too intuitive

3. When installing from PYPi, the models still have to be downloaded from github and placed manually in sitepackages. - this makes it a real pain to install.

4. Before you had a nice example on one page of looping over the Model -> Block ->Points, but don't see any nice examples in v2. I was looking at the to_spreadsheet to get some inspiration, but it just looks overly complex. My brain is having a much harder time following the V2 code compared to the v1 code.

5. It looks like there was concentration on accessing the data like d.DERMeasureAC[0].LLV.value which is nice, but I am trying to dump all the values from my inverter and store the output in simpler json files and using marshmallow to read those for processing like uploading to PVOutput. So accessing in that way is not really helpful.

Do you have an code examples like before that give a way to iterate over all the data and dump it out?

how to write model point as 'symbol'

Dear all,
It is unclear to me ( if possible ) how I can read/use the model points as symbols .

For example on model 715 I want to set 'OpCtl' to 1 , writing something like
Model715.OpCtl = 'START'

from model_715.json :
""
{
"access": "RW",
"desc": "Commands to PCS. Enumerated value.",
"label": "Set Operation",
"name": "OpCtl",
"size": 1,
"symbols": [
{
"label": "Stop the DER",
"name": "STOP",
"value": 0
},
{
"label": "Start the DER",
"name": "START",
"value": 1
},
{
"label": "Enter Standby Mode",
"name": "ENTER_STANDBY",
"value": 2
},
{
"label": "Exit Standby Mode",
"name": "EXIT_STANDBY",
"value": 3
}
],
"type": "enum16"
}
""
Thanks Antonio

Using write function

Hi,
I'm trying to use the "write" function in SunSpecModbusClientDeviceTCP.
I am having a tough time to understand what is the format of the 'data' parameter in the function.
From the comment in the function:
Parameters:

        addr :
            Starting Modbus address.

        count :
            Byte string containing register contents.

This is what I tried to do:

 d = client.SunSpecModbusClientDeviceTCP(slave_id=1, ipaddr=ip, ipport=port) 

 try:

    d.scan()
    
except Exception as e:

    print("Exception occured during Sunspec scan: ", e)
    
lst = [1,1]

byte_obj = bytes(lst)

d.write(d.DERCtlAC[0].model_addr+d.DERCtlAC[0].PFWInjEna.offset,byte_obj)

What is wrong in my code?
Thanks!

Fronius integration data problem

As described and discussed on HA Forum. After updates still do not work correctly.

client.py - scan() routine may cause reads that are not on point boundaries

The scan() routine in modbus/client.py performs the scan incorrectly. While performing the scan, it reads the full contents of each model rather than just the model id and length. For models that are larger than the maximum Modbus read request (125 registers), this may cause a read to occur starting in the middle of a point due to the read request having to be broken into multiple Modbus requests. It is guaranteed this will happen with the current model 701 definition if maximum sized reads are performed.

SunSpec Modbus server implementations may reject read requests that are not on point boundaries causing the scan to fail. SunSpec Modbus clients must only perform reads on point boundaries.

The scan routine should be updated to only access the model id and length during the scan. After the scan, the model definition can then be used to perform reads on point boundaries.

Missing license

What is the license for this project? Whatever it is, it ought to be added as a license file and also listed in the setup.py file under both the classifiers and the license parameter. If specified in each file as well then that ought to be consistent. spreadsheet.py for example seems to have an MIT license and device.py has nothing.

Opening a File Client, closing, and opening a Device will throw an error: FileClient has no close()

Modbus exception with d.scan() on SMA inverter

Dear developers,
I get a Modbus exception with v1.0.4 (both via pip3 install as well as from master branch) when trying to scan() an SMA inverter (STP-5000TL-20 via Ubuntu 20.04.02 LTS), see below:
$ python3
Python 3.8.5 (default, Jan 27 2021, 15:41:15)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import sunspec2.modbus.client as client
d = client.SunSpecModbusClientDeviceTCP(slave_id=126, ipaddr='192.168.0.125', ipport=502)
d.scan()
Traceback (most recent call last):
File "", line 1, in
File "/home/dennis/.local/lib/python3.8/site-packages/sunspec2/modbus/client.py", line 262, in scan
model.read()
File "/home/dennis/.local/lib/python3.8/site-packages/sunspec2/modbus/client.py", line 85, in read
data = self.model.device.read(self.model.model_addr + self.offset, self.len)
File "/home/dennis/.local/lib/python3.8/site-packages/sunspec2/modbus/client.py", line 317, in read
return self.client.read(addr, count, op)
File "/home/dennis/.local/lib/python3.8/site-packages/sunspec2/modbus/modbus.py", line 584, in read
data = self._read(addr + read_offset, read_count, op=op)
File "/home/dennis/.local/lib/python3.8/site-packages/sunspec2/modbus/modbus.py", line 545, in _read
raise ModbusClientException('Modbus exception %d: addr: %s count: %s' % (except_code, addr, count))
sunspec2.modbus.modbus.ModbusClientException: Modbus exception 2: addr: 40085 count: 98

Any idea how to solve this?

I modified the exception, to also show len(resp)=9 and resp[TCP_HDR_LEN+1]=0x83, the output is below, and I hope this helps to analyse the issue:

  File "/home/dennis/compiling/SST/pysunspec2/sunspec2/modbus/modbus.py", line 545, in _read
    raise ModbusClientException('Modbus exception %d: addr: %s count: %s, len(resp)=%d, resp[TCP_HDR_LEN+1]=0x%x' % (except_code, addr, count, len(resp), resp[TCP_HDR_LEN + 1]))
sunspec2.modbus.modbus.ModbusClientException: Modbus exception 2: addr: 40085 count: 98, len(resp)=9, resp[TCP_HDR_LEN+1]=0x83

Kind regards,
Dennis

Don't store models in the device model dictionary under None, only under model ID and/or model name

When pulling in models on device instantiation, if the model definition is not found, when storing the model in the device model dictionary, only put under model ID and not under None

e.g.

{1: [<sunspec2.file.client.FileClientModel object at 0x000001C9AB02D710>], None: [<sunspec2.file.client.FileClientModel object at 0x000001C9AB02D710>, <sunspec2.file.client.FileClientModel object at 0x000001C9AB02D780>], 701: [<sunspec2.file.client.FileClientModel object at 0x000001C9AB02D780>]}

device.py incorrect reference to model id attribute in exception message

Line 449 has an old model id attribute reference (self.model._id) that should be updated to self.model.model_id.

Opening a File Client, closing, and opening a Device will throw an error: FileClient has no close()

Modbus RTU connections may not be observing the inter-frame gap of 3.5 character times when performing requests

after first successful call to modbus.client.scan(), subsequent calls to scan() fail

When calling scan() a second time, there is no modbus traffic and the following exception arises:

>>> import sunspec2.modbus.client as client
>>> c = client.SunSpecModbusClientDeviceTCP()  # assumes a sunspec/modbus device at localhost:502 having unit_id 1
>>> c.connect()
>>> c.scan()
>>> c.scan()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/sunspec2/modbus/client.py", line 239, in scan
    model_id = mb.data_to_u16(model_id_data)
  File "/usr/lib/python3.8/site-packages/sunspec2/mb.py", line 112, in data_to_u16
    u16 = struct.unpack('>H', data[:2])
TypeError: a bytes-like object is required, not 'str'

This appears to be related to the assumption that the the data variable has been populated during the base_addr search. However, if self.base_addr has been found on the previous call to scan(), the data variable contains the empty string on the subsequent call. This causes struct.unpack() to throw the exception. This might be resolved easily by simply performing the base_addr search regardless of the pre-existing value of self.base_addr. I.e., removing this optimization.

Modbus client performs full model read on scan when there is no model definition

The Modbus client performs full model read on scan when there is not model definition. This can cause issues if the model is larger than allowed by a single Modbus read. Without a model definition, there is not point boundary information to guide the reads.

VERSION has not been updated since 1.0.0

pysunspec2/sunspec2/__init__.py

Line 2 in e946f86

VERSION = '1.0.0'

Please update the version string with each release. This will help applications know whether they can call upon certain features of the library depending on the version.

FileClientDevice doesn't appear to be populating curves from test files

Hi,

I was trying to write some tests against data downloaded from my inverter but found the curves/groups were not being populated from my test json files.

If I look at a test in pysunspec2 that I based my test on - https://github.com/sunspec/pysunspec2/blob/master/sunspec2/tests/test_file_client.py#L2645

    d = file_client.FileClientDevice(filename='C:/<snip>/input/device_1547.json')
    d.scan()
    assert d.volt_watt[0].curve[0].V1.value != None
    assert d.lfrt[0].curve[0].Tms1.value != None

curve values are not populated - so I don't see "Hz1": 4700 in the curve - I see null. Not sure If I am missing a piece of setup.

The device_1547.json has
{
"ID": 135,
"L": 60,
"ActCrv": 1,
"ModEna": 0,
"WinTms": null,
"RvrtTms": null,
"RmpTms": null,
"NCrv": 1,
"NPt": 6,
"Tms_SF": -2,
"Hz_SF": -2,
"Pad": 65535,
"curve": [
{
"ActPt": 6,
"Tms1": 196,
"Hz1": 4700,
...
doing d.get_json() returns
{
"ID": 135,
"L": 60,
"ActCrv": 1,
"ModEna": 0,
"WinTms": null,
"RvrtTms": null,
"RmpTms": null,
"NCrv": 1,
"NPt": 6,
"Tms_SF": -2,
"Hz_SF": -2,
"Pad": 65535,
"curve": [
{
"ActPt": null,
"Tms1": null,
"Hz1": null,
...

Add point update callback function

Add a callback function that can be called when the point value is updated. Add the following two attributes to the Point class:

update_func and update_func_arg. If the the update_func attribute is set, the update_func is called with the model and update_func_arg when the point value is updated.

update_func(model, update_func_arg)

The callback function should be used for an indication to be provided to another module and not be used for detailed processing.

Data conversion errors do not indicate the source of the error

When a data conversion error occurs it is difficult to determine the cause especially when multiple points are being updated.

Add more information about the cause of the error in the error string.

FileClientModel initializes FileClientGroup without proper points_class

Because the FileClientModel initializes its FileClientGroup parent class without referencing the FileClientPoint class, all points in the FileClientModel have the generic device.Point class instead of the file.client.FileClientPoint class.

One benefit of using the FileClientPoint class is that it contains a read() method, unlike the device.Point class which does not.

pysunspec2/sunspec2/file/client.py

Lines 54 to 55 in 642f1af

	FileClientGroup.__init__(self, gdef=gdef, model=self, model_offset=0, group_len=self.model_len, data=data,
	data_offset=0, group_class=group_class)

Should be:

 FileClientGroup.__init__(self, gdef=gdef, model=self, model_offset=0, group_len=self.model_len, data=data, 
                          data_offset=0, group_class=group_class, point_class=point_class)

Full model read is incomplete

When a full model read is performed, it is two registers short as the beginning of the model group points at the ID point but the model length does not include the ID and L points.

Publish to PyPI

This project should be published to PyPI so proper version constraints can be used when referencing it and we don't have to deal with the Git URL.

Socket timeout not handled as an address error during model scan

If a TCP socket timeout occurs during a model scan, the scan is aborted rather than moving to the next address. Some devices may not generate an exception when accessing an invalid address.

Update to treat a timeout during scan as an address access error and move to the next address candidate. This will create a longer total timeout since three addresses will be tried. Since this will add a delay for each address tried, make 40000 the first address tried as it is the most common SunSpec base address.

Support python 3.7 to 3.11

PySunspec readme mentions python 3.5 to 3.8 are supported.

We've been using it at my company in production with python 3.9 for a couple of months now and it seems to work fine. Our usage is limited to interraction with one inverter over TCP.

I created a quick tox configuration for it and the unit tests are green when executed against python 3.7, 3.8, 3.9, 3.10 or 3.11.

Also, python 3.5 and 3.6 have been marked as end-of-life for a few years now, so maybe they should be removed from the list of supported python versions?

If you're interrested, I'm happy to send a quick PR with this tox config and a readme update telling how to use it.

Model 203 negative PP voltage?

I think is something os wrong with phase-phase voltages I have read from my meter. I don't know it is a model issue or pysunspec2 lib.
But them shouldn't be negative and should be nearly 400V

common:
  ID:  1
  L:  65
  Mn:  WattNode
  Md:  WND-3Y-400-MB
  Opt:  Export+Import
  Vr:  31
  SN:  None
  DA:  None

ac_meter:
  ID:  203
  L:  105
  A:  42
  AphA:  7
  AphB:  12
  AphC:  21
  A_SF:  -1
  PhV:  23781
  PhVphA:  23781
  PhVphB:  21741
  PhVphC:  23619
  PPV:  -25609
  PhVphAB:  -26043
  PhVphBC:  -26313
  PhVphCA:  -24472
  V_SF:  -2
  [...]

Computed values contain floating point error

When a point has a nonzero scale factor, its cvalue (computed, scaled value) can contain floating point error. This causes problems when an application wants to verify a write by comparing the scaled value it has written to the device with the scaled value it reads back from the device.

For example, take a data point having an sf_value of -1. When the value is 302, the cvalue will read back (on my machine) as 30.200000000000003. The cvalue for this point really should be rounded to 30.2

Suggested change from (device.py#L220):

            if self.sf_value:
                sfv = self.sf_value
                if sfv:
                    v = v * math.pow(10, sfv)

To this:

            if self.sf_value:
                sfv = self.sf_value
                if sfv:
                    v = round(v * math.pow(10, sfv), -1 * sfv)

Add support for model 1 with a length of 67 to support legacy implementations

Originally, model 1 did not have a pad register at the end so the length is 67. The library should support model 1 with a length of 67 even though it does not conform to the model definition length of 68. This is the one and only condition of this type and allows support for legacy devices.

suns.py tool

With pysunspec there was a nice CLI utility, suns.py. With pysunspec2 I didn't find it.

Can I adapt the suns.py making it work with pysunspec2?

Thanks a lot for this library.

Functionality request: possibility to write values

As described and discussed on HA Forum.
E.g. writing data from a smart-meter (other than Fronius) to the Inverter to be usable in analytics.

Modbus client does not comply with the standard limit of registers for read/write requests

The Modbus specification states that the maximum number of registers returned from a Function 3 ("Read Holding Registers") or Function 4 ("Read Input Registers") is 125.
And the number of registers for a Function 16 ("Write Multiple registers") is 123.

The pysunspec2 Modbus client only supports one value for the max register count. By default it is 125. Because there is only one value, this means that there is no way for the client to reach both limits of the standard at the same time.

Could the client be modified to comply with the standard? I think either of the following would be fine:

• The default max number of registers for a write is by 123 registers, and default max for a read is 125
• When instantiating the client, the max count for write can be configured separately from max count for read

scan() performs full read of Modbus map

Instead of reading all model contents during scan, could the scan() method be changed to only read model Ids and lengths? This optimization would significantly improve performance on low-bandwidth Modbus infrastructure.

Is there a limit on number of points in a group?

I have a vendor model that was converted from SMDX. It has a repeating block that contains 128 points of type uint16. When I scan the device that exposes this model (which, FWIW, loads just fine in pysunspec), I get the following error:

>>> c.scan()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/sunspec2/modbus/client.py", line 260, in scan
    model = self.model_class(model_id=model_id, model_addr=addr, model_len=model_len, data=model_data,
  File "/usr/lib/python3.8/site-packages/sunspec2/modbus/client.py", line 160, in __init__
    SunSpecModbusClientGroup.__init__(self, gdef=gdef, model=self.model, model_offset=0, group_len=self.model_len,
  File "/usr/lib/python3.8/site-packages/sunspec2/modbus/client.py", line 69, in __init__
    device.Group.__init__(self, gdef=gdef, model=model, model_offset=model_offset, group_len=group_len,
  File "/usr/lib/python3.8/site-packages/sunspec2/device.py", line 401, in __init__
    raise ModelError('Nested groups too big')
sunspec2.device.ModelError: Nested groups too big

This is apparently due to a comparison of the "group length" with the literal ACCESS_REGION_REGS, which has a value of 123 (remember, there are 128 points in my model's repeating block).

Can you explain the reason for this 123-register limitation? Would it be feasible to change the library to allow models having groups that contain more registers than this?

Timeout during discovery is not handled correctly

Timeout during the discovery process for an unsupported Modbus address space causes discovery to abort rather than proceeding to the next address space.

How to use numbered models?

Dear all,
It is unclear to me how I can read/use the model parts, since for SMA (I'm using v1.0.5) there are no names like "common" attached to the models like in the repository Readme examples. It only has numbered items like 1:, 11:, ..
How can I use such a model?
Kind regards,
Dennis

Python 3.8.10 (default, Sep 28 2021, 16:10:42) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sunspec2.modbus.client as client
>>> d = client.SunSpecModbusClientDeviceTCP(slave_id=126, ipaddr='192.168.0.125', ipport=502)
>>> d.scan()
>>> d.models
{1: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63dcafe50>], 
11: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63dcafc10>], 
12: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63dcafa60>], 
103: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63ddc50d0>], 
120: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63ddc5e80>], 
121: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63ddadb50>], 
122: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63da86490>], 
123: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63da86820>], 
124: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63da86880>], 
126: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63da86520>], 
127: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63da864f0>], 
128: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63da86b80>], 
131: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63da86460>], 
132: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63da86430>], 
160: [<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7fc63da342e0>]}
>>> d.common[0].read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dennis/compiling/SST/pysunspec2/sunspec2/device.py", line 713, in __getattr__
    raise AttributeError("'%s' object has no attribute '%s'" % (self.__class__.__name__, attr))
AttributeError: 'SunSpecModbusClientDeviceTCP' object has no attribute 'common'
>>> d.common[1].read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dennis/compiling/SST/pysunspec2/sunspec2/device.py", line 713, in __getattr__
    raise AttributeError("'%s' object has no attribute '%s'" % (self.__class__.__name__, attr))
AttributeError: 'SunSpecModbusClientDeviceTCP' object has no attribute 'common'

Originally posted by @bijwaard in #36 (comment)

Some further tinkering:

>>> d.models[1]
[<sunspec2.modbus.client.SunSpecModbusClientModel object at 0x7f4851ed7f10>]
>>> d.models[1].read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'list' object has no attribute 'read'
>>> d.models[1][0].read()
>>> d.models[1][1].read()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: list index out of range
>>> print(d.models[1][0])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/dennis/compiling/SST/pysunspec2/sunspec2/device.py", line 458, in __str__
    return self.disp()
  File "/home/dennis/compiling/SST/pysunspec2/sunspec2/device.py", line 467, in disp
    s = '%s%s%s:\n' % (indent, self.gdef[mdef.NAME], index)
TypeError: 'NoneType' object is not subscriptable

Issue Scanning Device over TCP implementing Model 803

I'm trying to communicate with a Nuvation Low-Voltage BMS using this library over TCP.
The device has the common model, and models 801, 802, and 803. I'm trying to reach information in model 803.
Inside Model 803 is a repeating block, which represents battery stacks.

Using the 'master' branch, I encountered issue #14 , so I figured I would switch to the development branch where this appears to be fixed.
When using the current 'development' branch, I encountered the following while scanning the device:

d.scan()
Traceback (most recent call last):

  File "<ipython-input-6-e84d88ff024c>", line 1, in <module>
    d.scan()

  File "C:\Users\paul\Anaconda3\lib\site-packages\sunspec2\modbus\client.py", line 237, in scan
    model_id = mb.data_to_u16(model_id_data)

  File "C:\Users\paul\Anaconda3\lib\site-packages\sunspec2\mb.py", line 112, in data_to_u16
    u16 = struct.unpack('>H', data[:2])

TypeError: a bytes-like object is required, not 'str'

After this error, I can see the library has recognized the device has the common model, models 801 and 802, but not 803.
I can query what points are accessible, query/print values, etc. Have not tried writing any values - would like to confirm first I can read all information available on the device first.

Could you advise next steps on this? Any advice on troubleshooting this or potential workarounds would be helpful.