tasmota/decode-config

Backup/restore and decode configuration tool for Tasmota

Archived OLD development firmware builds

decode-config's Issues

intermittent struct.error

Describe the bug

I'm using tasmota 9.2.0.3. I have a cron job that uses decode-config to backup configs from my devices. Occasionally decode-config blows up with struct.error.

To Reproduce

decode-config -d unit1.example.com -T json --json-indent 4

Expected behavior

This normally outputs json but once in awhile it crashes.

Version Information

decode-config v9.2.0.3 [aeca3c0] by Norbert Richter <[email protected]>

Script:   decode-config
Python:   3.11.3
Platform: FreeBSD-13.2-RELEASE-amd64-64bit-ELF - amd64
OS:       FreeBSD 13.2-RELEASE FreeBSD 13.2-RELEASE r13 LBL
Time:     2023-04-22 13:23:08

System' information of your Tasmota data

{"bootcount": 325, "bootcount_reset_time": 1581993339, "cfg_crc": "0x5a49", "cfg_crc32": "0xc4545b4e", "cfg_holder": 4617, "cfg_size": 4096, "cfg_timestamp": 1682178556, "header": {"data": {"crc": "0x2e1e", "crc32": "0xc4545b4e", "platform": "ESP82xx", "size": 4096, "template": {"crc": "0x5a49", "crc32": "0xc4545b4e", "size": 4096, "version": "0x9020003"}, "version": "0x9020003"}, "env": {"platform": "FreeBSD-13.2-RELEASE-amd64-64bit-ELF", "python": "3.11.3", "script": "decode-config v9.2.0.3 [aeca3c0]", "system": "FreeBSD amd64 13.2-RELEASE FreeBSD 13.2-RELEASE r13 LBL"}, "timestamp": "2023-04-22 15:49:16"}, "version": "0x9020003"}

Additional context

Here's an example stack trace:

# get_fieldvalue FAILED fieldname=config_version, fielddef=(15, 'B', 3894, (None, '0 <= $ < len(Platform.STR)', ('Internal', None)), (None, False)), format_=B
Traceback (most recent call last):
  File "/home/ice/u0/leres/scripts/decode-config", line 5020, in <module>
    CONFIG['info'] = get_config_info(CONFIG['decode'])
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ice/u0/leres/scripts/decode-config", line 2631, in get_config_info
    config_version = get_field(decode_cfg, Platform.ALL, 'config_version', fielddef, raw=True, ignoregroup=True)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ice/u0/leres/scripts/decode-config", line 3851, in get_field
    value = get_fieldvalue(fieldname, fielddef, dobj, baseaddr+addroffset)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ice/u0/leres/scripts/decode-config", line 3648, in get_fieldvalue
    raise e
  File "/home/ice/u0/leres/scripts/decode-config", line 3644, in get_fieldvalue
    unpackedvalue = struct.unpack_from(format_, dobj, addr)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
struct.error: unpack_from requires a buffer of at least 3895 bytes for unpacking 1 bytes at offset 3894 (actual buffer size is 2920)

Restore drops '0' in version number. Won't complete

Describe the bug
A clear and concise description of what the bug is.
Sorry! Last one I promise. Backup and restores going sweetly now but when I do this:
decode-config.py -d 10.0.1.164 --backup-file Config_@f_@v, it creates a file called Config_Office_RGB_Bulb_8.1.0 which is great. When I try to restore though, I get the following error.

c:\Python\Python37>decode-config.py -d 10.0.1.164 --restore-file Config_Office_RGB_Bulb_8.1.0
ERROR 3 (@3297): File '.\Config_Office_RGB_Bulb_8.1.json' not found

Seems the 0 is being ignored.

To Reproduce
Steps to reproduce the behavior:
1. Program parameter:
2. Result:
As above
Expected behaviour
A clear and concise description of what you expected to happen.
Want to be able to copy the filename for the restore command without having to rename as storing backups with version numbers intact is important.
Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

decode-config.py v8.1.0.10 [00085] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.7.7
Platform: Windows-10-10.0.18362-SP0 - AMD64
OS:       Windows 10 10.0.18362
Time:     2020-03-20 14:31:05

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

{"bootcount": 63, "cfg_crc": "0xcffc", "cfg_crc32": "0xb8910bf8", "cfg_holder": 4617, "cfg_size": 4096, "cfg_timestamp": 1584615899, "displays": "0xffffffff", "energy_kWhtotal_time": 1583301699, "flag2": {"axis_resolution": 0}, "flag4": "0x00000000", "header": {"data": {"crc": "0x1b44", "crc32": "0xb8910bf8", "size": 4096, "version": "0x8010000"}, "format": {"jsoncompact": false, "jsonhidepw": false, "jsonindent": null, "jsonsort": true}, "os": ["AMD64", "Windows", "10", "10.0.18362", "Windows-10-10.0.18362-SP0"], "python": "3.7.7", "script": {"name": "decode-config.py", "version": "8.1.0.10 [00085]"}, "template": {"crc": "0xcffc", "crc32": "0xb8910bf8", "size": 4096, "version": "0x8010000"}, "timestamp": "2020-03-19 11:04:59"}, "last_module": 255, "monitors": "0xffffffff", "rgbwwTable": [255, 255, 255, 255, 255], "save_flag": 616, "sps30_inuse_hours": 0, "timezone_minutes": 0, "version": "0x8010000"}

Additional context
Add any other context about the problem here.

tuple index error in exec_function() command output, Power group

Describe the bug
When trying to used the "cmnd" output for the Power group, I get a tuple index out of range error. It occurs in exec_function(). Executes successfully for all other groups (although there is a non-fatal warning, see context below)

To Reproduce

1. Program parameter: decode-config.py -s tasmota-ABC123.dmp -T cmnd --group Power
2. Result: Premature exit - Internal error #21 (slightly different warning message using different versions, see below)

Version Information/Output:
realeased binary 9.3.1:

(env) ha% ./decode-config_linux -VV
decode-config_linux v9.3.1 [bc85da1] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.6.9
Platform: Linux-5.10.19-1-lts-x86_64-with-arch - x86_64
OS:       Linux 5.10.19-1-lts #1 SMP Fri, 26 Feb 2021 11:11:40 +0000
Time:     2021-03-30 17:36:22

(env) ha% ./decode-config_linux -s tasmota-ABC123.dmp -T cmnd --group Power
WARNING 21 (@3398): tuple index out of range
Premature exit - Internal error #21

current development 9.3.1.2:

(env) ha% ./decode-config.py -V
decode-config.py v9.3.1.2 [a432691] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.9.2
Platform: Linux-5.10.19-1-lts-x86_64-with-glibc2.33 - x86_64
OS:       Linux 5.10.19-1-lts #1 SMP Fri, 26 Feb 2021 11:11:40 +0000
Time:     2021-03-30 17:16:07

(env) ha% ./decode-config.py -s tasmota-ABC123.dmp -T cmnd --group Power
WARNING 21 (@3500): Replacement index 1 out of range for positional args tuple
Premature exit - Internal error #21

• 'system' information:

(env) ha% ./decode-config.py -f tasmota-ABC123.dmp -g System
{"bootcount": 24, "bootcount_reset_time": 1606584925, "cfg_crc": "0xf27f", "cfg_crc32": "0xbf6044e6", "cfg_holder": 4617, "cfg_size": 4096, "cfg_timestamp": 1609683854, "header": {"data": {"crc": "0xed7", "crc32": "0xbf6044e6", "platform": "ESP82xx", "size": 4096, "template": {"crc": "0xf27f", "crc32": "0xbf6044e6", "size": 4096, "version": "0x8050100"}, "version": "0x8050100"}, "env": {"platform": "Linux-5.10.19-1-lts-x86_64-with-glibc2.33", "python": "3.9.2", "script": "decode-config.py v9.3.1.2 [a432691]", "system": "Linux x86_64 5.10.19-1-lts #1 SMP Fri, 26 Feb 2021 11:11:40 +0000"}, "timestamp": "2021-01-03 14:24:14"}, "version": "0x8050100"}

Additional context:
When executing for any other group, I get 3 warnings starting with <strindex> out of range [0, 87] in <fielddef>. These seem to be unrelated and non-fatal. For example:

(env) ha% ./decode-config.py -s tasmota-ABC123.dmp -T cmnd --group Serial
<strindex> out of range [0, 87] in <fielddef> (15, '699s', (23, 87), ([8], None, ('Management', '"SwitchText{} {}".format(#+1,"\\"" if len($) == 0 else $)')))
<strindex> out of range [0, 87] in <fielddef> (15, '699s', (23, 95), (None, None, ('Light', None)))
<strindex> out of range [0, 87] in <fielddef> (2, '699s', (23, 87), ([28], None, ('Management', '"SwitchText{} {}".format(#+1,"\\"" if len($) == 0 else $)')))

# Serial:
  Baudrate 460800
  SBaudrate 38400
  SerialConfig 8N1
  SerialDelimiter 255
  TCPBaudrate 0

Syntax Error for use of 'nonlocal'

Describe the bug

ubuntu@ubuntu:~/tasmota/configs/decode-config-development$ ./decode-config.py -s 192.168.1.221
  File "./decode-config.py", line 4071
    nonlocal ack_flag
                    ^
SyntaxError: invalid syntax

To Reproduce

Steps to reproduce the behavior:
1. Program parameter:
2. Result:

Expected behavior

A clear and concise description of what you expected to happen.

Version Information

Please complete the following information
use decode-config.py -V to retrieve the info

ubuntu@ubuntu:~/tasmota/configs/decode-config-development$ ./decode-config.py -V
  File "./decode-config.py", line 4071
    nonlocal ack_flag
                    ^
SyntaxError: invalid syntax
ubuntu@ubuntu:~/tasmota/configs/decode-config-development$ ./decode-config.py -v
  File "./decode-config.py", line 4071
    nonlocal ack_flag
                    ^
SyntaxError: invalid syntax

System' information of your Tasmota data

Append -g System to your decode-config parameter, e. g. decode-config.py -s myconfig.dmp -g System

>>> paste output here <<<

Additional context

ubuntu@ubuntu:~/tasmota/configs/decode-config-development$ pip3 install -r requirements.txt
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: requests in /home/ubuntu/.local/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (2.25.1)
Requirement already satisfied: configargparse in /home/ubuntu/.local/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (1.5.3)
Collecting paho-mqtt
  Downloading paho-mqtt-1.6.1.tar.gz (99 kB)
     |████████████████████████████████| 99 kB 157 kB/s
Requirement already satisfied: idna<3,>=2.5 in /usr/lib/python3/dist-packages (from requests->-r requirements.txt (line 1)) (2.8)
Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.local/lib/python3.8/site-packages (from requests->-r requirements.txt (line 1)) (2021.5.30)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3/dist-packages (from requests->-r requirements.txt (line 1)) (1.25.8)
Requirement already satisfied: chardet<5,>=3.0.2 in /usr/lib/python3/dist-packages (from requests->-r requirements.txt (line 1)) (3.0.4)
Building wheels for collected packages: paho-mqtt
  Building wheel for paho-mqtt (setup.py) ... done
  Created wheel for paho-mqtt: filename=paho_mqtt-1.6.1-py3-none-any.whl size=62132 sha256=edd30e7c52ee7a5d638b646f416384488224e6a56913d18b66cdb228b75b0406
  Stored in directory: /home/ubuntu/.cache/pip/wheels/6a/48/01/c895c027e9b9367ec5470fbf371ee56e795a49ac6a19aa4c9f
Successfully built paho-mqtt
Installing collected packages: paho-mqtt
Successfully installed paho-mqtt-1.6.1
WARNING: You are using pip version 21.1.3; however, version 22.3.1 is available.
You should consider upgrading via the '/usr/bin/python3 -m pip install --upgrade pip' command.

Note:
I wanted to use the latest development branch because I cannot use the latest release version due to limitation of reading Tasmota 12.3.1.3.

This is the version that I have using pip3 install -r requirements.txt

ubuntu@ubuntu:~/tasmota/configs/$ decode-config.py -V
decode-config.py v12.3.1.0 [728642d] by Norbert Richter [email protected]

Script:   decode-config.py
Version:  12.3.1.0 [728642d]
Python:   3.8.10
Platform: Linux-5.4.0-1078-raspi-aarch64-with-glibc2.29 - aarch64
OS:       Linux 5.4.0-1078-raspi #89-Ubuntu SMP PREEMPT Mon Dec 5 08:38:35 UTC 2022
Time:     2023-01-16 19:39:00

This version works with Tasmota 12.1.1.2 but not with 12.3.1.3

WARNING 6: Tasmota configuration data v12.3.1.3 currently unsupported! The read
           configuration data is newer than the last supported v12.3.1 by this
           program. Newer Tasmota versions may contain changed data structures
           so that the data with older versions may become incompatible. You
           can force proceeding at your own risk by appending the parameter '--
           ignore-warnings'. Be warned: Forcing can lead to unpredictable
           results for your Tasmota device. In the worst case, your Tasmota
           device  will not respond and you will have to flash it again using
           the serial interface. If you are unsure and do not know the  changes
           in the configuration structure, you may able to use the developer
           version of this program from https://github.com/tasmota/decode-
           config/tree/development.
Premature exit - #6 Unsupported version

calls using get_grouplist(SETTINGS[0][2]) use latest device version, not configured device version

Describe the bug
In output_tasmotasubcmnds():

decode-config/decode-config.py

Line 4828 in 7584caf

groups = get_grouplist(SETTINGS[0][2])

creates groups based on the latest device version, which may create invalid commands for a device/file using an earlier version. I believe that the appropriate template has already been determined and is in CONFIG['info']['template'] and should be used instead.

SETTINGS[0][2] is also in the argparser:

decode-config/decode-config.py

Line 5067 in fad988d

groups = get_grouplist(SETTINGS[0][2])

but can't be replaced as CONFIG hasn't been defined yet. This could mean that invalid commands are suggested by the argparser help. This seems non-trivial to fix and a minor corner-case. I'd suggest ignoring the issue here (or maybe just including a note such as "some commands may not apply to devices/file with earlier versions of Tasmota")

To Reproduce
Using device with version 8.5.1:

% ./decode-config.py -s tasmota-ABC123.dmp --output-format cmnd --group Serial -v
INFO: Load data from file 'tasmota-ABC123.dmp'
INFO: File 'tasmota-ABC123.dmp' is using Tasmota v8.5.1 on ESP82xx
<strindex> out of range [0, 87] in <fielddef> (15, '699s', (23, 87), ([8], None, ('Management', '"SwitchText{} {}".format(#+1,"\\"" if len($) == 0 else $)')))
<strindex> out of range [0, 87] in <fielddef> (15, '699s', (23, 95), (None, None, ('Light', None)))
<strindex> out of range [0, 87] in <fielddef> (2, '699s', (23, 87), ([28], None, ('Management', '"SwitchText{} {}".format(#+1,"\\"" if len($) == 0 else $)')))

# Serial:
  Baudrate 460800
  SBaudrate 38400
  SerialConfig 8N1
  SerialDelimiter 255
  TCPBaudrate 0

Expected behavior
replacing line 4828 with: groups = get_grouplist(CONFIG['info']['template']), avoids the <strindex> out of range errors.

Version Information (please complete the following information):

• decode-config:

decode-config.py v9.3.1.2 [6081560] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.9.2
Platform: Linux-5.10.19-1-lts-x86_64-with-glibc2.33 - x86_64
OS:       Linux 5.10.19-1-lts #1 SMP Fri, 26 Feb 2021 11:11:40 +0000
Time:     2021-03-31 12:31:16

• 'system' information of your Tasmota data:

{"bootcount": 24, "bootcount_reset_time": 1606584925, "cfg_crc": "0xf27f", "cfg_crc32": "0xbf6044e6", "cfg_holder": 4617, "cfg_size": 4096, "cfg_timestamp": 1609683854, "header": {"data": {"crc": "0xed7", "crc32": "0xbf6044e6", "platform": "ESP82xx", "size": 4096, "template": {"crc": "0xf27f", "crc32": "0xbf6044e6", "size": 4096, "version": "0x8050100"}, "version": "0x8050100"}, "env": {"platform": "Linux-5.10.19-1-lts-x86_64-with-glibc2.33", "python": "3.9.2", "script": "decode-config.py v9.3.1.2 [6081560]", "system": "Linux x86_64 5.10.19-1-lts #1 SMP Fri, 26 Feb 2021 11:11:40 +0000"}, "timestamp": "2021-01-03 14:24:14"}, "version": "0x8050100"}

Additional context
In my specific case, the templates in SETTINGS that caused issue came from:

decode-config/decode-config.py

Lines 1922 to 1926 in fad988d

	SETTING_9_0_0_3.update ({
	'switchtext': (Platform.ALL, '699s',(0x017,SETTING_9_0_0_3[SETTINGVAR]['TEXTINDEX'].index('SET_SWITCH_TXT1')),
	([8], None, ('Management', '"SwitchText{} {}".format(#+1,"\\"" if len($) == 0 else $)')) ),
	'shelly_dimmer': (Platform.ALL, '699s',(0x017,SETTING_9_0_0_3[SETTINGVAR]['TEXTINDEX'].index('SET_SHD_PARAM')),
	(None, None, ('Light', None)) ),

decode-config/decode-config.py

Lines 1988 to 1990 in fad988d

	SETTING_9_2_0_6.update ({
	'switchtext_esp32': (Platform.ESP32, '699s',(0x017,SETTING_9_2_0_6[SETTINGVAR]['TEXTINDEX'].index('SET_SWITCH_TXT1')),
	([28], None, ('Management', '"SwitchText{} {}".format(#+1,"\\"" if len($) == 0 else $)')) ),

decode-config crashes with the Sonoff Zbridge

Describe the bug
decode-config crashes with the Sonoff Zbridge

To Reproduce
Steps to reproduce the behavior:
1. Program parameter: decode-config_win64 -T command -f Config_Zbridge_04C9EB_2539_9.2.0.2.dmp
2. Result:

'stty' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
File "decode-config.py", line 4940, in
IndexError: list index out of range
[12600] Failed to execute script decode-config

Expected behavior
A clear and concise description of what you expected to happen.

Normal output as per any other kind of sonoff/tasmota device.

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

Script:   decode-config.py
Python:   3.6.8
Platform: Windows-10-10.0.18362-SP0 - AMD64
OS:       Windows 10 10.0.18362
Time:     2021-01-05 16:58:15

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

(shouldn't that say -s, not -f?)

Same crash dump as before. 

However, since this data generates info about my PC, I have created something similar.
NB This is from a DIFFERENT .dmp file (of an RF Bridge)

{"bootcount": 36, "cfg_crc": "0x5814", "cfg_holder": 4617, "cfg_size": 3584, "header": {"data": {"crc": "0x5814", "size": 3584, "template": {"crc": "0x5814", "size": 3584, "version": "0x6040112"}, "version": "0x6050000"}, "env": {"platform": "Windows-10-10.0.18362-SP0", "python": "3.6.8", "script": "decode-config.py v9.2.0 [f54a77a]", "system": "Windows AMD64 10 10.0.18362"}, "timestamp": "2021-01-05 17:02:22"}, "version": "0x6050000"}

Additional context

I realise this is a very new device, so you may not yet have had a chance to develop for it yet.

--output-format cmnd is crashing with no group specified.

Describe the bug

Output as JSON works fine but when I try to specify --output-format cmnd it crashes.

I looked at the list of groups:
(choose from 'Control', 'Display', 'Domoticz', 'Internal', 'Knx', 'Light', 'Management', 'Mqtt', 'Power', 'Rf', 'Rules', 'Sensor', 'Serial', 'Setoption', 'Shutter', 'System', 'Telegram', 'Timer', 'Wifi', 'Zigbee')

It seems to be specific to Management as it works for most groups I've tried, but Management comes back with the same error if I add --group Management.

To Reproduce

_1. Program parameter: decode-config.py --source 192.168.2.86 --output-format cmnd
_2. Result:

WARNING 21 (@4692): 'gpio'
Premature exit - #21 Internal error

Expected behavior

Expected a full list with no filter

Version Information

decode-config.py v12.2.0.0 [ec10988] by Norbert Richter [email protected]

Script:   decode-config.py
Python:   3.9.2
Platform: Linux-5.15.19-1-pve-x86_64-with-glibc2.31 - x86_64
OS:       Linux 5.15.19-1-pve #1 SMP PVE 5.15.19-1 (Fri, 04 Feb 2022 06:09:14 +0100)
Time:     2022-10-19 03:37:11

System' information of your Tasmota data

{
	"bootcount": 27,
	"bootcount_reset_time": 1648700436,
	"cfg_crc": "0x99ae",
	"cfg_crc32": "0x93577ccd",
	"cfg_holder": 4617,
	"cfg_size": 4096,
	"cfg_timestamp": 1665854767,
	"header": {
		"data": {
			"crc": "0xcb1",
			"crc32": "0x93577ccd",
			"hardware": "ESP82/32",
			"size": 4096,
			"version": {
				"id": "0xc010104",
				"name": "12.1.1.4"
			}
		},
		"env": {
			"platform": "Linux-5.15.19-1-pve-x86_64-with-glibc2.31",
			"python": "3.9.2",
			"script": "decode-config.py v12.2.0.0 [ec10988]",
			"system": "Linux x86_64 5.15.19-1-pve #1 SMP PVE 5.15.19-1 (Fri, 04 Feb 2022 06:09:14 +0100)"
		},
		"template": {
			"crc": "0x99ae",
			"crc32": "0x93577ccd",
			"size": 4096,
			"version": {
				"id": "0xc010103",
				"name": "12.1.1.3"
			}
		},
		"timestamp": "2022-10-15 17:26:07"
	},
	"version": "0xc010104"
}

Additional context

It's working on different device that's running 12.2.0 so I tried to upgrade this problem device but it's having the same issue. This is a vont bulb if that matters. I get the same error across all of my vont bulbs (which are otherwise working fine)

Support version Tasmota 2022.01.3

Is your feature request related to a problem? Please describe

I have a couple tasmotas with version Tasmota 2022.01.3 by Theo Arends.

Describe the solution you'd like

Support this version if possible.

Describe alternatives you've considered

A binary cfg backup

I can provide a binary cfg backup from this version

Failed restoring a JSON from v9.3.1 onto a 9.5.0.8 device

Describe the bug

Restoring a JSON from v9.3.1 onto a 9.5.0.8 device failed.

To Reproduce

1. Have a device running 9.5.0
2. Backup the configuration as JSON decode-config -s 192.168.y.z -o device_9.5.0.json
3. Upgrade the device to 9.5.0.8
4. Clear the configuration with reset 1
5. Re-upload the backed up configuration with decode-config -s 192.168.y.z -i device_9.5.0.json
6. Fails with:

Load data from device '192.168.x.y'
WARNING 9 (@4247): file 'R:\tasmota\conf\skt_xxxxx_9.3.1.json' array 'sensors[3]' exceeds max number of elements [2]
Premature exit - Restore data error #9
WARNING 9 (@4261): file 'R:\tasmota\conf\skt_xxxxx_9.3.1.json' array 'sensors' couldn't restore, format has changed! Restore value contains <class 'list'> but an array of size [2] is expected
Premature exit - Restore data error #9

Expected behavior

Device should be reconfigured with a compatible configuration

Version Information (please complete the following information):

decode-config.py v9.5.0.8 [aa8ab02] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.8.6
Platform: Windows-10-10.0.19041-SP0 - AMD64
OS:       Windows 10 10.0.19041
Time:     2021-09-11 15:48:22

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

Didn't found a proper syntax to do it from a JSON file

Additional context

sensors field in v9.3.1:

  "sensors": [
    "0xffffffff",
    "0xffffffff",
    "0xffffffff"
  ],

became in 9.5.0.8

  "sensors": [
    [
      "0xffffffff", 
      "0xffffffff", 
      "0xffffffff", 
      "0xffffffff"
    ], 
    [
      "0xffffffff", 
      "0xffffffff", 
      "0xffffffff", 
      "0xffffffff"
    ]
  ], 

Friendly name gets stripped of national chars

Describe the bug
Dumping config for a Sonoff Pow R2, I see

  "friendlyname": [
    "Kkken fryser", 

Not affiliated to KKK :-) the actual friendly name is "Køkken fryser" (Kitchen freezer). Looks like chars outside of 7-bit ascii gets stripped. Same with other similar chars. This happens both when dumping to bash terminal and a file. Same when using @f to put friendly name into output file name (not translated to _).

To Reproduce
Configure Tasmota with such a name, and run decode-config.

Expected behavior
Name preserved. More generally, I'd expect UTF-8 encoding as the "natural" text file encoding.

Version Information (please complete the following information):

./decode-config.py -VV
decode-config.py v8.1.0.6 [00076] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.6.9
Platform: Linux-4.15.0-76-generic-x86_64-with-Ubuntu-18.04-bionic - x86_64
OS:       Linux 4.15.0-76-generic #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020
Time:     2020-02-20 10:34:17

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

{
  "bootcount": 11, 
  "cfg_crc": "0xdfda", 
  "cfg_crc32": "0xbdc51bb5", 
  "cfg_holder": 4617, 
  "cfg_size": 4096, 
  "cfg_timestamp": 1582153201, 
  "displays": "0xffffffff", 
  "energy_kWhtotal_time": 1579286020, 
  "flag2": {
    "axis_resolution": 0
  }, 
  "flag4": "0x00000000", 
  "header": {
    "data": {
      "crc": "0xde3a", 
      "crc32": "0xbdc51bb5", 
      "size": 4096, 
      "version": "0x8010000"
    }, 
    "format": {
      "jsoncompact": false, 
      "jsonhidepw": false, 
      "jsonindent": 2, 
      "jsonsort": true
    }, 
    "os": [
      "x86_64", 
      "Linux", 
      "4.15.0-76-generic", 
      "#86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020", 
      "Linux-4.15.0-76-generic-x86_64-with-Ubuntu-18.04-bionic"
    ], 
    "python": "3.6.9", 
    "script": {
      "name": "decode-config.py", 
      "version": "8.1.0.6 [00076]"
    }, 
    "template": {
      "crc": "0xdfda", 
      "crc32": "0xbdc51bb5", 
      "size": 4096, 
      "version": "0x8010000"
    }, 
    "timestamp": "2020-02-19 23:00:01"
  }, 
  "last_module": 0, 
  "monitors": "0xffffffff", 
  "rgbwwTable": [
    255, 
    255, 
    255, 
    255, 
    255
  ], 
  "save_flag": 109, 
  "sps30_inuse_hours": 0, 
  "timezone_minutes": 0, 
  "version": "0x8010000"
}

Additional context

[FR] Update for v.12.0.1

Describe the solution you'd like

Support for latest tasmota version (v12.0.1)

Describe alternatives you've considered

Use '--ignore-warnings flag (config exported from v11.1.0) but I am not sure if this will break anything

MqttFingerprint truncated after dumping to json

Describe the bug
After dumping configuration to json, mqttfingerprint seems to be somehow broken. After restore it is shorter than expected and seems to be missing some values. I'm suspecting that any character which is not valid UTF-8 gets skipped. Please see reproduction session for details. Same issue occurs when dumping json from .dmp file.

When using .dmp files to restore, fingerprints are fine.
To Reproduce
Steps to reproduce the behavior:

$ curl 'http://<device-ip>/cm?&user=<user>&password=<password>&cmnd=mqttfingerprint1'
{"MqttFingerprint1":"D8 75 36 AF 9D 90 13 5F 9C 46 64 BD 1F A3 D8 15 35 CC 5D F8"}%                                                                                                                                 
$ curl 'http://<device-ip>/cm?&user=<user>&password=<password>&cmnd=mqttfingerprint2'
{"MqttFingerprint2":"D8 75 36 AF 9D 90 13 5F 9C 46 64 BD 1F A3 D8 15 35 CC 5D F8"}%
$ decode-config.py -d <device-ip>-u <user> -p <password> -g mqtt --json-indent 2 -o dump.json
$ grep mqtt_fingerprint -A 2 dump.json
  "mqtt_fingerprint": [
    "u6_Fd5]", 
    "u6_Fd5]"
$ decode-config.py -d <device-ip> -u <user> -p <password> -g mqtt -i dump.json -vv
INFO: Load data from device '<device-ip>'
INFO: Device '<device-ip>' is using Tasmota 8.2.0
INFO: Reading restore file './dump.json' (JSON format)
INFO: Config file contains data of Tasmota 8.2.0
INFO: Push new data to '<device-ip>' using restore file './dump.json'
INFO: Restore successful to device '<device-ip>' using restore file './dump.json'
$ curl 'http://<device-ip>/cm?&user=<user>&password=<password>&cmnd=mqttfingerprint2'                                             
{"MqttFingerprint2":"75 36 5F 46 64 35 5D 00 00 00 00 00 00 00 00 00 00 00 00 00"}%                                                                                                                                 
$ curl 'http://<device-ip>/cm?&user=<user>&password=<password>&cmnd=mqttfingerprint1'
{"MqttFingerprint1":"75 36 5F 46 64 35 5D 00 00 00 00 00 00 00 00 00 00 00 00 00"}% 

Expected behavior
Mqttfingerprint1 and Mqttfingerprint2 should not change after restoring from backup.

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

decode-config.py v8.2.0.0 [00064] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.8.2
Platform: Linux-5.6.2-arch1-2-x86_64-with-glibc2.2.5 - x86_64
OS:       Linux 5.6.2-arch1-2 #1 SMP PREEMPT Sun, 05 Apr 2020 05:13:14 +0000
Time:     2020-04-15 20:49:24

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

{
  "bootcount": 46, 
  "bootcount_reset_time": 1586780660, 
  "cfg_crc": "0x905c", 
  "cfg_crc32": "0x5f9d84bc", 
  "cfg_holder": 4617, 
  "cfg_size": 4096, 
  "cfg_timestamp": 1586977104, 
  "displays": "0xffffffff", 
  "energy_kWhtotal_time": 1548000555, 
  "flag2": {
    "axis_resolution": 0
  }, 
  "header": {
    "data": {
      "crc": "0x60fa", 
      "crc32": "0x5f9d84bc", 
      "size": 4096, 
      "version": "0x8020000"
    }, 
    "format": {
      "jsoncompact": false, 
      "jsonhidepw": false, 
      "jsonindent": 2, 
      "jsonsort": true
    }, 
    "os": [
      "x86_64", 
      "Linux", 
      "5.6.2-arch1-2", 
      "#1 SMP PREEMPT Sun, 05 Apr 2020 05:13:14 +0000", 
      "Linux-5.6.2-arch1-2-x86_64-with-glibc2.2.5"
    ], 
    "python": "3.8.2", 
    "script": {
      "name": "decode-config.py", 
      "version": "8.2.0.0 [00064]"
    }, 
    "template": {
      "crc": "0x905c", 
      "crc32": "0x5f9d84bc", 
      "size": 4096, 
      "version": "0x8020000"
    }, 
    "timestamp": "2020-04-15 18:58:24"
  }, 
  "last_module": 0, 
  "monitors": "0xffffffff", 
  "rgbwwTable": [
    255, 
    255, 
    255, 
    255, 
    255
  ], 
  "save_flag": 79, 
  "sps30_inuse_hours": 0, 
  "timezone_minutes": 0, 
  "version": "0x8020000"
}

Additional context
When storing dump directly from Tasmota (.dmp) and uploading it back, fingerprint is preserved.

Ideas for new functions

1. it will be great to have a command/rule like "if ping [1.2.3.4 and/or 4.5.6.7] true/false do ...".
   Example: So it will be possible, when you leave your home with your smartphone, to detect if you at home or not. So you can automate turn off all you have to turn off. Or turn on/off your homealarm or some other stuff.

2.expand the command websend in this way, that i can define what will be send.
(Like as unix wget)
So it will be possible for example: to control my ipcam turn on or off ftp upload ect. Or turn around PTZ to different positions. Or some other stuff.

No module named 'configargparse', try "python -m pip install 'configargparse'" on MacOs

Installed python3 with brew.

python3 ./decode-config.py -f Config_wifi_light12_9.1.0.dmp
No module named 'ConfigArgparse', try "python -m pip install 'ConfigArgparse'"

% pip3 list
Package Version

---

certifi 2020.12.5
chardet 4.0.0
ConfigArgParse 1.2.3
idna 2.10
pip 20.3.1
requests 2.25.1
setuptools 51.0.0
urllib3 1.26.2
wheel 0.36.1

I tried changing:
import ConfigArgParse as configargparse

but no difference.

UnicodeEncodeError: 'latin-1' codec can't encode character '\u25a0' in position 583: ordinal not in range(256)

Describe the bug

Command "./decode-config.py -s 192.168.11.138" raises the following error:
Traceback (most recent call last):
File "./decode-config.py", line 6470, in
print(get_jsonstr(CONFIG['groupmapping'], ARGS.jsonsort, ARGS.jsonindent, ARGS.jsoncompact))
UnicodeEncodeError: 'latin-1' codec can't encode character '\u25a0' in position 583: ordinal not in range(256)

A clear and concise description of what the bug is.

To Reproduce

Steps to reproduce the behavior:
1. Program parameter:
./decode-config.py -s 192.168.11.138
2. Result:
File "./decode-config.py", line 6470, in
print(get_jsonstr(CONFIG['groupmapping'], ARGS.jsonsort, ARGS.jsonindent, ARGS.jsoncompact))
UnicodeEncodeError: 'latin-1' codec can't encode character '\u25a0' in position 583: ordinal not in range(256)

Expected behavior

A clear and concise description of what you expected to happen.

Version Information

Please complete the following information
use decode-config.py -V to retrieve the info

>>> paste output here <<<
Script:   decode-config.py
Python:   3.7.3
Platform: Linux-5.10.63-v7+-armv7l-with-debian-10.12 - armv7l
OS:       Linux 5.10.63-v7+ #1496 SMP Wed Dec 1 15:58:11 GMT 2021
Time:     2022-08-31 15:02:52

System' information of your Tasmota data

Append -g System to your decode-config parameter, e. g. decode-config.py -s myconfig.dmp -g System

>>> paste output here <<<
{"bootcount": 24, "bootcount_reset_time": 1604220613, "cfg_crc": "0x132e", "cfg_crc32": "0x98db5fc2", "cfg_holder": 4617, "cfg_size": 4096, "cfg_timestamp": 1660993002, "header": {"data": {"crc": "0xb2ff", "crc32": "0x98db5fc2", "hardware": "ESP82", "size": 4096, "version": {"id": "0xc010000", "name": "12.1.0"}}, "env": {"platform": "Linux-5.10.63-v7+-armv7l-with-debian-10.12", "python": "3.7.3", "script": "decode-config.py v12.1.1.1 [87254b6]", "system": "Linux armv7l 5.10.63-v7+ #1496 SMP Wed Dec 1 15:58:11 GMT 2021"}, "template": {"crc": "0x132e", "crc32": "0x98db5fc2", "size": 4096, "version": {"id": "0xc000204", "name": "12.0.2.4"}}, "timestamp": "2022-08-20 10:56:42"}, "version": "0xc010000"}

Additional context

That happens only on a tasmota device with shutter configuration

Merge decode-config.exe...

in your repo. Is this okay for you?
Greets Jason

Github Release Action for adding binary executables to Release Assets

Is your feature request related to a problem? Please describe

Some (many?) users cannot run python scripts directly on their computers. Python may not be installed by default (Windows) or they are running Docker containers where python is not available (example: openHAB).
Even though it is not hard to run pyinstaller --onefile decode-config.py to create an executable, it would be nice if executables for some of the most popular platforms (Windows, Ubuntu 20/22, MacOS 10/11-) were available for direct download from the Github Releases page.

Describe the solution you'd like

A GitHub action which builds executables for the platforms listed above and adds them to "Assets" on the Github Releases page, in addition to the already present source code archives.

Describe alternatives you've considered

Building locally each time I install a new (major) version of Tasmota. And forgetting how to do that exactly each time I try (Yes, I know that this is a me-problem)

Additional context

Since this is a me-problem with probably low priority for most people I will try to create a PR containing a Github Action to implement this feature request.
It has been mentioned before here (and not by me), so I assume there is at least some interest in the user-base. The PR should be fairly low-impact on the project, except for polluting the "Assets" panel with a larger number of choices for people to select from.

Release action for packaging and publishing reference outdated version on release update

when updating an existing release action for deploying references outdated version of source (#39)

Parameters are not used / forwarded

OS: Win7 64bit
Python: 3.8 (latest compatible with Win7)
Problem with: Decode-config.py +cmd / powershell / powershell 32 bit with / without admin privileges

Command -m pip install ... Was executed fine

Describe the bug
All parameters forwarded like -s / --source are not recognized, and only the short info about the commands to be used is plotted (like only "decode-config.py" is executed.

Therefore also no json file etc. Is created or plotted.

To Reproduce
_Steps to reproduce the behavior: Win7? Phyton 3.8?
_1. Program parameter: any. E.g. "decode-config.py -s 192.168.0.123"
_2. Result: no config file is created

Expected behavior
Dmp file / config from device can be decoded.

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:
  Latest from git-master

paste decode-config.py output here <<<
Like the output if no parameter is added

- 'system' information of your Tasmota data (append `-g System` to your decode-config parameter, e. g. `decode-config -f myconfig.dmp -g System`):  

doesn't work

paste decode-config.py output here <<<

**Additional context**
_Add any other context about the problem here._

conversion between versions 6.5.0.16 -> to 8.5.1 not possible

Describe the bug
By executing command py decode-config.py -s tasmota_8.5.1.dmp -i config-tasmota_6.5.0.16.json the execution fails with:

WARNING 9 (@3734): file 'config-tasmota_6.5.0.16.json' array 'energy_power_delta' couldn't restore, format has changed! Restore value contains (class 'int') but an array of size [3] is expected
Premature exit - Restore data error #9"

To Reproduce
_backup file with 6.5.0.16, update to 8.5.1 and restore the backup
1. Program parameter: -s [TARGET] -i [SOURCE]
2. Result: WARNING 9 (@3734): file 'config-tasmota_6.5.0.16.json' array 'energy_power_delta' couldn't restore, format has changed! Restore value contains (class 'int') but an array of size [3] is expected
Premature exit - Restore data error #9"

Expected behavior
conversion between formats is executed (can convert data from older Tasmota versions (from version v5.10.0) to a newer one and vice versa)

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

v8.5.1 [00069]

Can't re-upload a config just after download

Describe the bug

Can't re-upload the downloaded config

To Reproduce

Gosund SP111 running Tasmota 9.2.0.2 as cloned from github on 4 jan at ~ 20h CET

Download the conf

$ decode-config.py -c ~/iot-tools/tasmota/conf-backup/my.conf -s 192.168.168.111 -o @[email protected]
Load data from device '192.168.168.111'
Backup successful to '.\skt_minipc_9.2.0.2.json' (JSON format)

Trying to reupload

$ decode-config.py -c ~/iot-tools/tasmota/conf-backup/my.conf -s 192.168.168.111 -i skt_minipc_9.2.0.2.json
Load data from device '192.168.168.111'
WARNING 9: file '.\skt_minipc_9.2.0.2.json' - {'energy_kWhyesterday': 22222200} (valid range exceeding)
Premature exit - Restore data error #9

Removing the parameter energy_kWhyesterday from the JSON an tying to re-upload again

$ decode-config.py -c ~/iot-tools/tasmota/conf-backup/my.conf -s 192.168.168.111 -i skt_minipc_9.2.0.2.json
Load data from device '192.168.168.111'
WARNING 21 (@3316): index out of range
Premature exit - Internal error #21
WARNING 9 (@3906): file '.\skt_minipc_9.2.0.2.json' array 'rules' couldn't restore, format has changed! Restore value contains <class 'list'> but an array of size [3] is expected
Premature exit - Restore data error #9

Expected behavior

Both cases should have reloaded the configuration correctly.
The 2nd case is more problematic

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

decode-config.py v9.2.0.2 [adb6a8a] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.8.6
Platform: Windows-10-10.0.19041-SP0 - AMD64
OS:       Windows 10 10.0.19041
Time:     2021-01-04 21:28:19

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

{
  "bootcount": 53, 
  "bootcount_reset_time": 1584792271, 
  "cfg_crc": "0x5d71", 
  "cfg_crc32": "0x97a0667f", 
  "cfg_holder": 4617, 
  "cfg_size": 4096, 
  "cfg_timestamp": 1609793459, 
  "header": {
    "data": {
      "crc": "0x49b6", 
      "crc32": "0x97a0667f", 
      "platform": "ESP82xx", 
      "size": 4096, 
      "template": {
        "crc": "0x5d71", 
        "crc32": "0x97a0667f", 
        "size": 4096, 
        "version": "0x9020002"
      }, 
      "version": "0x9020002"
    }, 
    "env": {
      "platform": "Windows-10-10.0.19041-SP0", 
      "python": "3.8.6", 
      "script": "decode-config.py v9.2.0.2 [adb6a8a]", 
      "system": "Windows AMD64 10 10.0.19041"
    }, 
    "timestamp": "2021-01-04 20:50:59"
  }, 
  "version": "0x9020002"
}

Additional context

RuntimeError: dictionary keys changed during iteration

Describe the bug
./decode-config.py -v --device 10.10.80.224
INFO: Load data from device '10.10.80.224'
INFO: Device '10.10.80.224' is using Tasmota v8.2.0.4 on ESP32
Traceback (most recent call last):
File "./decode-config.py", line 3943, in
print(get_jsonstr(CONFIG['mapping'], ARGS.jsonsort, ARGS.jsonindent, ARGS.jsoncompact))
File "./decode-config.py", line 1741, in get_jsonstr
for key in configmapping:
RuntimeError: dictionary keys changed during iteration

The same error on tasmota 7.x devices.

To Reproduce
The Problem is comming with commit:
SHA-1: 3ed9f48

• Change JSON output key sort case independently
  Earlier versions are working.

Expected behavior
Output of json :)

Version Information (please complete the following information):
decode-config.py v8.2.0.5 [00127] by Norbert Richter [email protected]

Script: decode-config.py
Python: 3.8.2
Platform: Linux-5.4.0-26-generic-x86_64-with-glibc2.29 - x86_64
OS: Linux 5.4.0-26-generic #30-Ubuntu SMP Mon Apr 20 16:58:30 UTC 2020
Time: 2020-04-27 08:30:51

[Question] Is friendly name skipped on error?

When generating a backup using @f to set friendly name in backup filename it's skipped.

This is the only device that happend and it's a multiple socket. I don't know if it's due an error reading the first or it has a limit on length. All 5 input boxes of friendly name has been set into the web config.

How to keep track of Zigbee ad-hoc configuration commands?

Is your feature request related to a problem? Please describe.

I did not realise until too late that the ZBbridge commands that configure the Zigbee appliances are "ad-hoc" and are not saved in the configuration dump.

Describe the solution you'd like
Presumably these changes are stored in zigbee-related flash memory somewhere. If so, could there be a tool to save those potentially complicated configuration items? E.g. group memberships, device friendly names, etc.

Describe alternatives you've considered

An article written telling us how to save these ad-hoc configurations via a series of interrogatory commands that could be cut and paste somewhere would be good.

Thanks!

cmnd format: wrong index for ShutterButton

Describe the bug
Using the "cmnd" format, the output of the ShutterButton commands is wrong (index starts at 0 instead of 1)

To Reproduce

1. Configure a Tasmota device as Shutter/Blind
2. Do a backup of the configuration (decode-config_linux --source x.y.z.t --backup-type dmp --backup-file /tmp/shutter)
3. Export the Shutter configuration from the file (decode-config_linux --source /tmp/shutter.dmp --output-format cmnd --group Shutter)
4. Check the " ShutterButton" lines, the index starts at 0 instead of 1

Expected behavior
The index starts at 1

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

decode-config_linux v8.4.0 [00067] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.8.2
Platform: Linux-5.4.0-42-generic-x86_64-with-glibc2.4 - x86_64
OS:       Linux 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020
Time:     2020-08-24 11:04:34

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

{
  "bootcount": 19, 
  "bootcount_reset_time": 1592817296, 
  "cfg_crc": "0x1d70", 
  "cfg_crc32": "0x05b04d4b", 
  "cfg_holder": 4617, 
  "cfg_size": 4096, 
  "cfg_timestamp": 1598254567, 
  "header": {
    "data": {
      "crc": "0x3753", 
      "crc32": "0x5b04d4b", 
      "platform": "ESP82xx", 
      "size": 4096, 
      "template": {
        "crc": "0x1d70", 
        "crc32": "0x5b04d4b", 
        "size": 4096, 
        "version": "0x8030100"
      }, 
      "version": "0x8030100"
    }, 
    "env": {
      "platform": "Linux-5.4.0-42-generic-x86_64-with-glibc2.4", 
      "python": "3.8.2", 
      "script": "decode-config.py v8.4.0 [00067]", 
      "system": "Linux x86_64 5.4.0-42-generic #46-Ubuntu SMP Fri Jul 10 00:24:02 UTC 2020"
    }, 
    "timestamp": "2020-08-24 07:36:07"
  }, 
  "version": "0x8030100"
}

Additional context
Add any other context about the problem here.

decode-config_linux --source /tmp/shutter.dmp --output-format cmnd --group Shutter

# Shutter:
  KeeloqSet 0 0 0 0
  ShutterButton0 1 51 26 38 51 0 0 0 1 1
  ShutterButton0 2 1 26 13 1 0 0 0 1 1
  ShutterButton0 3 0 0 0 0 0 0 0 0 0
  ShutterButton0 4 0 0 0 0 0 0 0 0 0
  ShutterCalibration1 300,500,700,900,1000
  ShutterCalibration2 0,0,0,0,0
  ShutterCalibration3 0,0,0,0,0
  ShutterCalibration4 0,0,0,0,0
  ShutterCloseDuration1 15.0
  ShutterCloseDuration2 0.0
  ShutterCloseDuration3 0.0
  ShutterCloseDuration4 0.0
  ShutterEnableEndStopTime1 0
  ShutterEnableEndStopTime2 0
  ShutterEnableEndStopTime3 0
  ShutterEnableEndStopTime4 0
  ShutterInvert1 0
  ShutterInvert2 0
  ShutterInvert3 0
  ShutterInvert4 0
  ShutterInvertWebButtons1 0
  ShutterInvertWebButtons2 0
  ShutterInvertWebButtons3 0
  ShutterInvertWebButtons4 0
  ShutterLock1 0
  ShutterLock2 0
  ShutterLock3 0
  ShutterLock4 0
  ShutterMotorDelay1 0.0
  ShutterMotorDelay2 0.0
  ShutterMotorDelay3 0.0
  ShutterMotorDelay4 0.0
  ShutterOpenDuration1 15.0
  ShutterOpenDuration2 0.0
  ShutterOpenDuration3 0.0
  ShutterOpenDuration4 0.0
  ShutterPosition1 0
  ShutterPosition2 0
  ShutterPosition3 0
  ShutterPosition4 0
  ShutterRelay1 1
  ShutterRelay2 0
  ShutterRelay3 0
  ShutterRelay4 0
  ShutterSetHalfway1 50
  ShutterSetHalfway2 0
  ShutterSetHalfway3 0
  ShutterSetHalfway4 0

decode-config behind a proxy ERROR 10 (@3892): Device did not respond properly

Hi,
i try to convert to json may tasmota device..
url is
http://login:[email protected]/my_device/

this url works fine in a browser...
but when i use decode-config like this :
python3 decode-config.py --source http://login:[email protected]/my_device/ --json-indent 2 > conf_my_device.json
i have this error :
ERROR 10 (@3892): Device did not respond properly
i use a proxy and login passwd for proxy nginx and this is not tasmota webserver login...
is it possible to use decode-config with a proxy ?
regards

Error in Decode-Config on ESP 32

Describe the bug

I try to send a json File to an ShellyPlusI4 witch i first load from it. Ist dosn´t work

To Reproduce

Steps to reproduce the behavior:
_1. Program parameter:

1. python decode-config.py -s 192.168.1.37 -t json --json-show-pw -o settings.json
2. python decode-config.py -s 192.168.1.37 --restore-file settings.json

2. Result:
Load data by http from device '192.168.1.37'
WARNING 9: Data incompatibility: Device '192.168.1.37' hardware is 'ESP32 (excl S3/S2/C3)', restore file '.\settings.json' is for hardware 'ESP82'
Premature exit - #9 Restore data error

Expected behavior

A clear and concise description of what you expected to happen.

Version Information

Please complete the following information
use decode-config.py -V to retrieve the info

decode-config.py v11.0.0.1 [adb1f66] by Norbert Richter [email protected]

Script:   decode-config.py
Python:   3.10.3
Platform: Windows-10-10.0.22000-SP0 - AMD64
OS:       Windows 10 10.0.22000
Time:     2022-03-22 18:39:54

System' information of your Tasmota data

Append -g System to your decode-config parameter, e. g. decode-config.py -s myconfig.dmp -g System

{"bootcount": 9, "bootcount_reset_time": 1647636654, "cfg_crc": "0x1ad9", "cfg_crc32": "0x9a20fa5f", "cfg_holder": 4617, "cfg_size": 4096, "cfg_timestamp": 1647967362, "header": {"data": {"crc": "0x9e22", "crc32": "0x9a20fa5f", "hardware": "ESP32 (excl S3/S2/C3)", "size": 4096, "version": {"id": "0xb000000", "name": "11.0.0"}}, "env": {"platform": "Windows-10-10.0.22000-SP0", "python": "3.10.3", "script": "decode-config.py v11.0.0.1 [adb1f66]", "system": "Windows AMD64 10 10.0.22000"}, "template": {"crc": "0x1ad9", "crc32": "0x9a20fa5f", "size": 4096, "version": {"id": "0xa010006", "name": "10.1.0.6"}}, "timestamp": "2022-03-22 16:42:42"}, "version": "0xb000000"}

convert json to dmp doesn't find json file

Describe the bug

Trying to convert a json file to dmp file does not work.

Having this files/directories:

[myuser@myhost tasmota]$ pwd
/home/myuser/tasmota

[myuser@myhost tasmota]$ ls -la
insgesamt 308
drwxr-xr-x   3 myuser myuser   4096 19. Jun 22:26 .
drwx------ 155 myuser myuser  12288 19. Jun 21:33 ..
-rw-r--r--   1 myuser myuser   4096 19. Jun 21:34 Config_kaffee_12.5.0.dmp
-rw-r--r--   1 myuser myuser  31379 19. Jun 22:11 Config_kaffee_12.5.0.json
-rw-r--r--   1 myuser myuser  31381 19. Jun 22:11 Config_newdev_12.5.0.json
drwxr-xr-x   5 myuser myuser   4096 18. Apr 09:07 decode-config-12.5.0
-rw-r--r--   1 myuser myuser 223607 19. Jun 21:33 decode-config_v12.5.0.tar.gz

Config_kaffee_12.5.0.dmp is a configuration backup made via Tasmota WebUI.
Config_kaffee_12.5.0.json is the converted/decoded dmp file

Config_newdev_12.5.0.json is a copy of Config_kaffee_12.5.0.json, but with other hostname.

Now I try to convert the Config_newdev_12.5.0.json to a dmp file that I could import/restore to a new Tasmota device.
But none of the tested methods works:

[myuser@myhost tasmota]$ decode-config-12.5.0/decode-config.py -s Config_newdev_12.5.0.json -t dmp -o Config_newdev_12.5.0.dmp
Load data by http from device 'Config_newdev_12.5.0.json'
ERROR 22: Failed to establish HTTP connection to 'Config_newdev_12.5.0.json:80'
Premature exit - #22 HTTP connection error

[myuser@myhost tasmota]$ decode-config-12.5.0/decode-config.py -s ./Config_newdev_12.5.0.json -t dmp -o Config_newdev_12.5.0.dmp
Load data by http from device './Config_newdev_12.5.0.json'
ERROR 22: Failed to establish HTTP connection to './Config_newdev_12.5.0.json:80'
Premature exit - #22 HTTP connection error

[myuser@myhost tasmota]$ decode-config-12.5.0/decode-config.py -s file://./Config_newdev_12.5.0.json -t dmp -o Config_newdev_12.5.0.dmp
ERROR 3 (@3881): File 'file://./Config_newdev_12.5.0.json' not found
Premature exit - #3 File not found

[myuser@myhost tasmota]$ decode-config-12.5.0/decode-config.py -s file://Config_newdev_12.5.0.json -t dmp -o Config_newdev_12.5.0.dmp
ERROR 3 (@3881): File 'file://Config_newdev_12.5.0.json' not found
Premature exit - #3 File not found

[myuser@myhost tasmota]$ decode-config-12.5.0/decode-config.py -s file://$(pwd)/Config_newdev_12.5.0.json -t dmp -o Config_newdev_12.5.0.dmp
ERROR 3 (@3881): File 'file:///home/myuser/tasmota/Config_newdev_12.5.0.json' not found
Premature exit - #3 File not found

To Reproduce

1. download https://github.com/tasmota/decode-config/archive/refs/tags/v12.5.0.tar.gz and extract
2. create dmp file from Tasmota 12.5.0 WebUI and convert it to json
3. try to create a dmp file from the json file (examples see above)

Expected behavior

json file should be converted to a dmp file

Version Information

Please complete the following information
use decode-config.py -V to retrieve the info

[myuser@myhost tasmota]$ decode-config-12.5.0/decode-config.py -V
decode-config.py v12.5.0.0 [f670155] by Norbert Richter [email protected]

Script:   decode-config.py
Version:  12.5.0.0 [f670155]
Python:   3.11.3
Platform: Linux-6.3.8-arch1-1-x86_64-with-glibc2.37 - x86_64
OS:       Linux 6.3.8-arch1-1 #1 SMP PREEMPT_DYNAMIC Wed, 14 Jun 2023 20:10:31 +0000
Time:     2023-06-19 22:53:45

System' information of your Tasmota data

Append -g System to your decode-config parameter, e. g. decode-config.py -s myconfig.dmp -g System

[myuser@myhost tasmota]$ decode-config-12.5.0/decode-config.py -s Config_kaffee_12.5.0.dmp -g System
{"bootcount": 23, "bootcount_reset_time": 1662488095, "cfg_crc": "0x7c9c", "cfg_crc32": "0x355e8eed", "cfg_holder": 4617, "cfg_size": 4096, "cfg_timestamp": 1687185041, "header": {"data": {"crc": "0xdd0f", "crc32": "0x355e8eed", "hardware": "ESP82", "size": 4096, "version": {"id": "0xc050000", "name": "12.5.0"}}, "env": {"platform": "Linux-6.3.8-arch1-1-x86_64-with-glibc2.37", "python": "3.11.3", "script": "decode-config.py v12.5.0.0 [f670155]", "system": "Linux x86_64 6.3.8-arch1-1 #1 SMP PREEMPT_DYNAMIC Wed, 14 Jun 2023 20:10:31 +0000"}, "template": {"crc": "0x7c9c", "crc32": "0x355e8eed", "size": 4096, "version": {"id": "0xc050000", "name": "12.5.0"}}, "timestamp": "2023-06-19 14:30:41"}, "version": "0xc050000"}

Additional context

N/A

Apparent off-by-one issue with GPIO listings in "Command" mode. [Low Priority]

Describe the bug

Listing the information from a 6.3.0.12 saved configuration file gave the GPIOs starting from GPIO1, rather than GPIO0.

To Reproduce

decode-config.py -T Command -f /path/to/saved/configs/Config_Sonoff_6.3.0.dmp

...produces (excerpt):-

Management:

Emulation 0
FriendlyName1 TV-Switch
FriendlyName2 TV-Switch2
FriendlyName3 TV-Switch3
FriendlyName4 TV-Switch4
Gpio1 0
Gpio2 0
Gpio3 4
Gpio4 0
Gpio5 0
Gpio6 0
Gpio7 0
Gpio8 0
Gpio9 0
Gpio10 0
Gpio11 0
Gpio12 0
Gpio13 17
Gpio14 37
Gpio15 38
Gpio16 39
Gpio17 21
Gpio18 0

Expected behavior

I would have expected the first GPIO to have been labelled as "GPIO0", as that's how they are labelled in the actual TASMOTA application.

In this particular instance, the DS1820 sensor (ID #4) is actually on GPIO2, not GPIO3.

Version Information (please complete the following information):

decode-config.py v8.1.0.1 [00063] by Norbert Richter [email protected]

Script: decode-config.py
Python: 3.6.9
Platform: Linux-4.15.0-50-generic-x86_64-with-Ubuntu-18.04-bionic - x86_64
OS: Linux 4.15.0-50-generic #54-Ubuntu SMP Mon May 6 18:46:08 UTC 2019
Time: 2019-12-30 16:42:41

decode-config/decode-config.py -g system --json-indent 2 -f ~/Sonoff/Config_TV_Switch_6.3.0.12" (6)".dmp
{
"bootcount": 37,
"cfg_crc": "0x28ca",
"cfg_holder": 4619,
"cfg_size": 3584,
"displays": "0xffffffff",
"drivers": [
"0xffffffff",
"0xffffffff",
"0xffffffff"
],
"energy_kWhtotal_time": 1543697777,
"flag": {},
"flag2": {
"axis_resolution": 0
},
"flag3": {},
"header": {
"data": {
"crc": "0x28ca",
"size": 3584,
"version": "0x603000c"
},
"format": {
"jsoncompact": false,
"jsonhidepw": false,
"jsonindent": 2,
"jsonsort": true
},
"os": [
"x86_64",
"Linux",
"4.15.0-50-generic",
"#54-Ubuntu SMP Mon May 6 18:46:08 UTC 2019",
"Linux-4.15.0-50-generic-x86_64-with-Ubuntu-18.04-bionic"
],
"python": "3.6.9",
"script": {
"name": "decode-config.py",
"version": "8.1.0.1 [00063]"
},
"template": {
"crc": "0x28ca",
"size": 3584,
"version": "0x603000b"
},
"timestamp": "2019-12-30 07:47:18"
},
"last_module": 17,
"monitors": "0xffffffff",
"rgbwwTable": [
255,
255,
255,
255,
255
],
"save_flag": 1957,
"sensors": [
"0xffffffff",
"0xffffffff",
"0xffffffff"
],
"timezone_minutes": 0,
"version": "0x603000c"
}

Additional context
Add any other context about the problem here.

Very low priority (more than ten GPIOs, so I had to take my socks off to keep count). :-)

Incorrect decoding (it seems) of Sensor29 configuration

Describe the bug
When reading back a device configured to use Sensor29(https://tasmota.github.io/docs/MCP230xx/) the config data displayed is wrong for ports 8-15.

Using using tasmota 8.2.0 compiled as tasmota-sensors with only this setting: #define USE_MCP230xx

The output is shown as follows:

decode-config_win64.exe -d 192.168.1.216 -T cmnd
  <snip>
  Sensor29 0,2,1,1
  Sensor29 1,2,1,1
  Sensor29 2,2,1,1
  Sensor29 3,2,1,1
  Sensor29 4,2,1,1
  Sensor29 5,2,1,1
  Sensor29 6,2,1,1
  Sensor29 7,2,1,1
  Sensor29 8,0,0,0
  Sensor29 9,7,1,3
  Sensor29 10,7,1,3
  Sensor29 11,6,1,3
  Sensor29 12,3,0,3
  Sensor29 13,0,0,0
  Sensor29 14,0,0,0
  Sensor29 15,7,1,3

While all the lines should be identical (which is what I configured on the device).

To Reproduce
Steps to reproduce the behavior:

• Configure tasmota with support for MCP230xx
• Configure Sensors as follows Sensor29 0,2,1,1
• Repeat for all 16 inputs of Sensor29 Sensor29 1,2,1,1 .. etc.
• Re-read the configuration using decode-config_win64.exe -d 192.168.1.216 -T cmnd
• See that ports 8-15 are wrong - The ones for port 0-7 are correct.

Expected behavior
The proper config is shown for all 16 ports and not just the lower 8

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

C:\Users\garci\Downloads\tasmota>decode-config_win64.exe -VV
decode-config_win64.exe v8.2.0.0 [00063] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.6.8
Platform: Windows-10-10.0.18362-SP0 - AMD64
OS:       Windows 10 10.0.18362
Time:     2020-05-03 20:02:36

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):
  decode-config_win64.exe -d 192.168.1.216 -g System

{"bootcount": 9, "bootcount_reset_time": 1587951307, "cfg_crc": "0x2607", "cfg_crc32": "0x83687e57", "cfg_holder": 4617, "cfg_size": 4096, "cfg_timestamp": 1588457925, "displays": "0xffffffff", "energy_kWhtotal_time": 1587951307, "flag2": {"axis_resolution": 0}, "header": {"data": {"crc": "0xbd8b", "crc32": "0x83687e57", "size": 4096, "version": "0x8020000"}, "format": {"jsoncompact": false, "jsonhidepw": false, "jsonindent": null, "jsonsort": true}, "os": ["AMD64", "Windows", "10", "10.0.18362", "Windows-10-10.0.18362-SP0"], "python": "3.6.8", "script": {"name": "decode-config.py", "version": "8.2.0.0 [00063]"}, "template": {"crc": "0x2607", "crc32": "0x83687e57", "size": 4096, "version": "0x8020000"}, "timestamp": "2020-05-02 22:18:45"}, "last_module": 255, "monitors": "0xffffffff", "rgbwwTable": [255, 255, 255, 255, 255], "save_flag": 38, "sps30_inuse_hours": 0, "timezone_minutes": 0, "version": "0x8020000"}

JSON output has extra characters

    "ntp_server": [
        "openhabianpi",
        "pool.ntp.orgrg",
        "us.pool.ntp.orgg"
    ],
    "ota_url": "http://thehackbox.org/tasmota/tasmota.binff.bin",

Should be .org on 2 & 3 and just tasmota.bin

A clue could be that the device originally was 6.x and had the default (pre-compiled binaries and OTAURL). For example, the OTAURL was http://thehackbox.org/tasmota/release/sonoff.bin. So the configuration backup dmp must still have the remainders (ff.bin) of that string. May be the same for the NTP servers.

This could be the case for other strings but I just spotted these in my backup.

mqtt discovery, compare & restore

Is your feature request related to a problem? Please describe.
I briefly thought I'd accidentally reset ALL of my devices - turns out the issue was that my router needed a restart phew, but this made me think it'd be a good time to backup all of my device configs, which is how (after being disappointed to learn that the config export is binary) I found this project. I'm also a big fan of version control which seems to be very lacking in the home automation space in general, but it looks like most of the parts are already here!

Describe the solution you'd like

1. autodiscovery and backup of all tasmota devices using specified MQTT server/credentials
2. save all decoded configs in specified folder, ideally with option for mac address for filename, overwrite if exists
3. option to restore all from a folder, using mqtt server to get IPs for mac addresses, ideally with a check to see if config has changed to avoid writing unchanged configs
4. profit

Describe alternatives you've considered
none

Additional context
intention for this is making this work with git, allowing you to commit changes when you do something useful, checkout/roll back to a known good state when you're experimenting and break your house (or are part way though doing something but have to stop and would like everything to keep working until you get back to working on it)

Command option - TimeZone command is not proper

Describe the bug

When the timezone has a minute componet:
JSON dump shows two parameters but Timezone command omits the minute component and shows only the hours.

To Reproduce

Steps to reproduce the behavior:

ubuntu@ubuntu:~/tasmota/configs$ decode-config.py -s $deviceip --ignore-warnings | jq

  "timezone": 5,
  "timezone_minutes": 30,
  "toffset": [
    60,
    120
  ],

ubuntu@ubuntu:~/tasmota/configs$ decode-config.py -s $deviceip --ignore-warnings -T command | grep -i time

  Time 1
  TimeDST 0,0,3,1,2,120
  TimeSTD 0,0,10,1,3,60
  Timezone 5

Expected behavior

The timezone command should be:

Timezone +05.30

Version Information

Please complete the following information
use decode-config.py -V to retrieve the info

decode-config.py v12.3.1.0 [728642d] by Norbert Richter [email protected]

Script:   decode-config.py
Version:  12.3.1.0 [728642d]
Python:   3.8.10
Platform: Linux-5.4.0-1078-raspi-aarch64-with-glibc2.29 - aarch64
OS:       Linux 5.4.0-1078-raspi #89-Ubuntu SMP PREEMPT Mon Dec 5 08:38:35 UTC 2022
Time:     2023-01-16 22:01:59

System' information of your Tasmota data

Append -g System to your decode-config parameter, e. g. decode-config.py -s myconfig.dmp -g System

>>> paste output here <<<

Additional context

Add any other context about the problem here.

Managing multiple devices

Hi. This is a semi-philosophical speculative rambling to see what others are doing / what others think.

I have just started using decode-config to configure 130 Shelly Plug S devices that are part of a solution to reduce energy spending in the heating of building. The plugs are glued to electrical heaters and limit their schedules.

Any way, speaking in generic terms and thinking of every decode-config user out there, how can we use decode-config in a practical way for a number of devices, not just one?

The Unix way: each tool does only its own thing

So, this is a nice school of thought. We leave decode-config to do its job of communicating with a single device, and then use other tools like bash script to discover and iterate devices and call decode-config over and over.

Advantages:

• no bloating of the software with too many options

• easier to maintain

Disadvantages:

• requires further scripting knowledge, which is not the best thing to require in the IoT space which is meant for a wider audience.

• multiple users will solve the same problems over and over again, in many different projects. What a waste of time and effort.

The user-friendly way, enhancing functionality

Some options can be added to decode-config, or a separate tool config-manager is devised, allowing:

• defining groups of devices (not to be confused with Tasmota command groups of the current --group option), similar to "Rooms" in some IoT apps. It's simple, just a name given to a list of devices, which can be specified in JSON format, or in a config file, for example.

• incorporate some iteration feature so that decode-config can go over such device groups and apply commands to each one in sequence (or even in parallel, why not?)

• we'll probably want some discovery features also. Asking the software to scan the network and generate a list of devices that can be used in the features suggested above.

Thoughts?

I'd love to hear what @curzon01 and others think of this. And also possibly other ideas you have, how have you been solving this sort of problems which are probably common to many of us?

I guess I'm just trying to stay with the simplicity of the command-line and avoiding home automation servers...

Thanks!

Error when trying to restore saved config in JSON

Describe the bug
When I try to restore the previously saved config (JSON), I get these errors:

"zb_topic_fname" is deprecated
"zigbee_distinct_topics" is deprecated

To Reproduce

1. command "SetOption100 1"
2. command "SetOption112 1"
3. decode-config_win64.exe -s admin:[email protected] -o zig.json
4. decode-config_win64.exe -s admin:[email protected] -i zig.json

Expected behavior
Should work.

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

decode-config_win64.exe v9.1.0 [a93dad9] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.6.8
Platform: Windows-10-10.0.19041-SP0 - AMD64
OS:       Windows 10 10.0.19041
Time:     2020-11-30 21:36:32

Package up and publish to pypi

Is your feature request related to a problem? Please describe.

Would be great if this could be packaged up and distributed on pypi.

This would enable users to easily install via tools like pip or pipx.

Describe the solution you'd like

Leveraging a tool such as poetry could help with this or you could do it manually with setup.py.

Could be easily plugged in via a GitHub action such as https://github.com/inverse/python-iolite-client/blob/master/.github/workflows/publish.yml

Describe alternatives you've considered

Distributing via Docker could also be a solution.

Additional context
Add any other context or screenshots about the feature request here.

AttributeError when restoring

Describe the bug
When trying to restore a JSON config to a device, the following error message appears:

$ python3 decode-config.py -S --dry-run -e -s http://admin:[email protected] -i Config_sonoff_315698_5784_8.1.0_ori.conf
Traceback (most recent call last):
  File "decode-config.py", line 4768, in <module>
    restore(ARGS.restorefile, ARGS.backupfileformat, CONFIG)
  File "decode-config.py", line 4235, in restore
    new_decode_cfg = mapping2bin(config, jsonconfig, restorefilename)
  File "decode-config.py", line 4053, in mapping2bin
    set_field(_buffer, 1<<config['info']['platform'], name, setting_fielddef, data, addroffset=0, filename=filename)
  File "decode-config.py", line 3653, in set_field
    restoremap = restoremapping.get(name, None)
AttributeError: 'str' object has no attribute 'get'

The same happens also without the -S --dry-run -e parameters, these were just added one after the other for testing.

To Reproduce

1. Download a Config dump
2. decode it using python3 decode-config.py -s Config_sonoff_315698_5784_8.1.0.dmp -o Config_sonoff_315698_5784_8.1.0_ori.conf
3. Try to restore the config back to the device using command stated above

Expected behavior
Config restored to device

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

decode-config.py v8.4.0 [00067] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.6.9
Platform: Linux-4.15.0-109-generic-x86_64-with-Ubuntu-18.04-bionic - x86_64
OS:       Linux 4.15.0-109-generic #110-Ubuntu SMP Tue Jun 23 02:39:32 UTC 2020
Time:     2020-08-12 14:29:50

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

$ python3 decode-config.py -f Config_sonoff_315698_5784_8.1.0.dmp -g System
{"bootcount": 9, "cfg_crc": "0x9453", "cfg_crc32": "0xad5ab1e8", "cfg_holder": 4618, "cfg_size": 4096, "cfg_timestamp": 1597226542, "header": {"data": {"crc": "0xdaa8", "crc32": "0xad5ab1e8", "size": 4096, "template": {"crc": "0x9453", "crc32": "0xad5ab1e8", "size": 4096, "version": "0x8010000"}, "version": "0x8010000"}, "env": {"platform": "Linux-4.15.0-109-generic-x86_64-with-Ubuntu-18.04-bionic", "python": "3.6.9", "script": "decode-config.py v8.4.0 [00067]", "system": "Linux x86_64 4.15.0-109-generic #110-Ubuntu SMP Tue Jun 23 02:39:32 UTC 2020"}, "timestamp": "2020-08-12 10:02:22"}, "version": "0x8010000"}

Additional context
Add any other context about the problem here.

Windows 64-Bit exe fails on dump command with stack trace

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:
1. Program parameter:. \decode-config_win64.exe -d < device ip > --backup-file Config-@f --backup-type json
2. Result:

Expected behavior
A clear and concise description of what you expected to happen.

According to https://tasmota.github.io/docs/#/Upgrading?id=decode-config-tool the command should create a new backup straight from the device.

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

decode-config_win64.exe v8.2.0.0 [00063] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.6.8
Platform: Windows-10-10.0.18362-SP0 - AMD64
OS:       Windows 10 10.0.18362
Time:     2020-03-26 00:26:18

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

{
   "bootcount":12,
   "cfg_crc":"0x4c46",
   "cfg_crc32":"0x543b80aa",
   "cfg_holder":4617,
   "cfg_size":4096,
   "cfg_timestamp":1584881422,
   "displays":"0xffffffff",
   "energy_kWhtotal_time":1579366855,
   "flag2":{
      "axis_resolution":0
   },
   "flag4":"0x00000000",
   "header":{
      "data":{
         "crc":"0x2c6a",
         "crc32":"0x543b80aa",
         "size":4096,
         "version":"0x8010000"
      },
      "format":{
         "jsoncompact":false,
         "jsonhidepw":false,
         "jsonindent":null,
         "jsonsort":true
      },
      "os":[
         "AMD64",
         "Windows",
         "10",
         "10.0.18362",
         "Windows-10-10.0.18362-SP0"
      ],
      "python":"3.6.8",
      "script":{
         "name":"decode-config.py",
         "version":"8.2.0.0 [00063]"
      },
      "template":{
         "crc":"0x4c46",
         "crc32":"0x543b80aa",
         "size":4096,
         "version":"0x8010000"
      },
      "timestamp":"2020-03-22 12:50:22"
   },
   "last_module":0,
   "monitors":"0xffffffff",
   "rgbwwTable":[
      255,
      255,
      255,
      255,
      255
   ],
   "save_flag":1045,
   "sps30_inuse_hours":0,
   "timezone_minutes":0,
   "version":"0x8010000"
}

Additional context
Add any other context about the problem here.

TypeError: '<' not supported

Decoding of a saved configuration from Tasmota 6.7.1 produces an error:

./decode-config.py -f Config_6.7.1.dmp

Traceback (most recent call last):
  File "./decode-config.py", line 3342, in <module>
    print(json.dumps(configmapping, sort_keys=args.jsonsort, indent=None if args.jsonindent<0 else args.jsonindent, separators=(',', ':') if args.jsoncompact else (', ', ': ') ))
TypeError: '<' not supported between instances of 'NoneType' and 'int'

--backup-file does not save entire name so restore fails

Describe the bug
A clear and concise description of what the bug is.
Using decode-config_win64 -d 10.0.1.67 --backup-file Config_@f_@v produces a file called Config_Sonoff_S20_6.6.0.json. Subsequently using decode-config_win64 -d 10.0.1.67 --restore-file Config_@f_@v produces and error that states:

ERROR 3 (@3125): File '.\Config_Sonoff_6.6.0.json' not found

To Reproduce
Steps to reproduce the behavior:
1. Program parameter:
2. Result:
As above

Expected behavior
A clear and concise description of what you expected to happen.
Restore would locate correct file name

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

decode-config_win64 v8.1.0.0 [00062] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.6.8
Platform: Windows-10-10.0.18362-SP0 - AMD64
OS:       Windows 10 10.0.18362
Time:     2020-03-18 19:46:21

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

{"SensorBits1": {}, "bootcount": 14, "cfg_crc": "0xefa0", "cfg_holder": 4617, "cfg_size": 3584, "displays": "0xffffffff", "energy_kWhtotal_time": 1567749179, "flag": {}, "flag2": {"axis_resolution": 0}, "flag3": {}, "header": {"data": {"crc": "0xefa0", "size": 3584, "version": "0x6060000"}, "format": {"jsoncompact": false, "jsonhidepw": false, "jsonindent": null, "jsonsort": true}, "os": ["AMD64", "Windows", "10", "10.0.18362", "Windows-10-10.0.18362-SP0"], "python": "3.6.8", "script": {"name": "decode-config.py", "version": "8.1.0.0 [00062]"}, "template": {"crc": "0xefa0", "size": 3584, "version": "0x605000f"}, "timestamp": "2020-03-18 06:48:43"}, "last_module": 255, "monitors": "0xffffffff", "rgbwwTable": [255, 255, 255, 255, 255], "save_flag": 44, "sensors": ["0xffffffff", "0xffffffff", "0xffffffff"], "sps30_inuse_hours": 0, "timezone_minutes": 0, "version": "0x6060000"}

Additional context
Add any other context about the problem here.

Error output when restoring a backup.

Describe the bug
A clear and concise description of what the bug is.

I just did another testing.
Tried to restore with decode-config.
Firstly I restored a script with a restore-Subset!
Command:
decode-config-development>decode-config.py -d 192.168.148.213 --restore-file Config_Rules_Gosund_4_213.json
The script was restored correctly, but on console I get an error Output!
ERROR 11 (@4287): Config data upload failed - <div style='text-align:center;'>Gerät wird jetzt neu gestartet</div><br></div> Premature exit - Upload error #11
When restoring a complete Backup the script was also restored and activated correctly, but I got the same error output!

May be you have an idea to solve this issue.

Tested with decode-config developer v8.3.1.7 [00179] commit aff1983

Thanks again for implementing the scripting flag.

When converting a dmp file to json, file name does not follow dmp file

Describe the bug
A clear and concise description of what the bug is.
When using decode-config.py -f Config_washer_7021_8.1.0.dmp --backup-file Config-@f --backup-type json, the created file is name Config-Washing_Machine.json instead of Config_Washing_Machine.json (replaces underscore with hyphen).

To Reproduce
Steps to reproduce the behavior:
1. Program parameter:
2. Result:

Expected behavior
A clear and concise description of what you expected to happen.
json file to match source filename.

Version Information (please complete the following information):

• decode-config: [e.g. use decode-config -VV to retrieve the info]:

decode-config.py v8.1.0.10 [00085] by Norbert Richter <[email protected]>

Script:   decode-config.py
Python:   3.7.7
Platform: Windows-10-10.0.18362-SP0 - AMD64
OS:       Windows 10 10.0.18362
Time:     2020-03-19 22:18:33

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

{"bootcount": 174, "cfg_crc": "0x1d7d", "cfg_crc32": "0xe700f372", "cfg_holder": 4617, "cfg_size": 4096, "cfg_timestamp": 1584500617, "displays": "0xffffffff", "energy_kWhtotal_time": 1543840427, "flag2": {"axis_resolution": 0}, "flag4": "0x00000000", "header": {"data": {"crc": "0x433d", "crc32": "0xe700f372", "size": 4096, "version": "0x8010000"}, "format": {"jsoncompact": false, "jsonhidepw": false, "jsonindent": null, "jsonsort": true}, "os": ["AMD64", "Windows", "10", "10.0.18362", "Windows-10-10.0.18362-SP0"], "python": "3.7.7", "script": {"name": "decode-config.py", "version": "8.1.0.10 [00085]"}, "template": {"crc": "0x1d7d", "crc32": "0xe700f372", "size": 4096, "version": "0x8010000"}, "timestamp": "2020-03-18 03:03:37"}, "last_module": 0, "monitors": "0xffffffff", "rgbwwTable": [255, 255, 255, 255, 255], "save_flag": 1312, "sps30_inuse_hours": 0, "timezone_minutes": 0, "version": "0x8010000"}

Additional context
Add any other context about the problem here.

WARNING 9: Restore file '.\send.json' contains obsolete name 'fast_power_cycle_disable'

I'm reading json from a Shelly 2.5 with version 12.0.2.3.
The result is
"flag3": {..."fast_power_cycle_disable": 1,...}. When I try to restore the same thing I get the following message: WARNING 9: Restore file '.\send.json' contains obsolete name 'fast_power_cycle_disable'.

Release executables for the latest version

Is your feature request related to a problem? Please describe

Hello, I run in a Windows x64 environment and I used a script to back up my Tasmota devices with decode-config.
When a new version was released, I would simply replace the binary file with the newer version, but this time you didn't release the executable builds of the decode-config tool.

Describe the solution you'd like

I'd prefer if you kept releasing decode-config with the binary files included in the package.

Describe alternatives you've considered

Additional context

decode-config and Tasmota scripting

Is your feature request related to a problem? Please describe.

I am using the current developement version of tasmota with rules disabled and
scripting enabled.
The scripts are compressed!

Error output is:
File "C:\Users\vultu\AppData\Local\Programs\Python\Python37\lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u0484' in position 17424: character maps to

There is no error with version 8.3.1 (Fred) of Tasmota
Scripts in this version are not compressed.

I asked gemu2015 regarding this issue. I got this answer.

i never used this tool. the only difference between rules and scripting is that rules is using 3 separate sections while scripting uses 1 section of script buffer (containing the 3 rules buffers). i am not very familiar with python so you should better ask the author of decode-config (Norbert Richter) to look at this problem.

It would be nice if decode-config could decode Tasmota scripting versions
with compressed scripts.
regards Billy

Allow to source non-dmp files for format conversions in one fell swoop

Is your feature request related to a problem? Please describe

I want to edit a config.dmp to publish it.

• works: decode-config.py -s config.dmp --backup-file config.json
• works: nano config.json
• doesn't work: decode-config.py -s config.json --backup-file edited.dmp

Describe the solution you'd like

decode-config.py -s config.json --backup-file.dmp

Describe alternatives you've considered

• works: decode-config.py -s config.dmp --backup-file config.json
• works: nano config.json
• probably works: decode-config.py -s dummydevice --restore-file config.json
• probably works: decode-config.py -s dummydevice --backup-file edited.dmp

Additional context

File conversion is the first marketed program feature.

Feature request: Command output diff showing only non-default settings

Is your feature request related to a problem? Please describe.

• It is difficult to determine what was changed on a Tasmota device.
• decode-config -T command is helpful but it shows every single command supported by the device, which makes it difficult to find portions that are relevant.

Describe the solution you'd like

• An option which retrieves the current configuration in command format using a minimal set of commands and arguments that would be required to restore the device to the current configuration from the default configuration.
• May require a method to account for differences in default settings or command syntax between Tasmota release versions.

Describe alternatives you've considered

• dumping a default config and using a diff tool to locate differences
  • this seems like it would work as long as the Tasmota command syntax does not exceed a certain complexity

Additional context

• Enterprise network devices often have a console command which prints all non-default configuration in a correct and minimized format. Any grouped or nested clauses are cleaned up, and the minimum set of optional arguments is used rather than every optional argument. (Hopefully.)
• This can be used to help visually locate configuration items that were changed. I would argue that this is an essential quality-of-life feature in those environments.
• It's not uncommon for certain items to be excluded if they are delegated to another component of the system, or if they have a security implication. Additionally some commands pertaining to persistent state or filesystem access might not be included. Perhaps any security credentials e.g. WiFi or MQTT passwords should be blinded by default, requiring positive action from the user to request those details be included.

Invalid URL: file path occasionally converted to URL

Describe the bug
Occasionally the absolute file path passed to decode-config is converted to a URL.
/tmp/tasmota01.local.dmp => http:///tmp/tasmota01.local.dmp/dl

To Reproduce
Steps to reproduce the behavior:
Run decode-config many times on a file, e.g.:

decode-config -s /tmp/tasmota01.local.dmp

> 95% of the time this works fine
however occasionally it yields this error:

Traceback (most recent call last):
File "/usr/local/bin/decode-config", line 5207, in <module>
  CONFIG['encode'] = pull_tasmotaconfig(ARGS.device, ARGS.port, username=ARGS.username, password=ARGS.password)
File "/usr/local/bin/decode-config", line 3198, in pull_tasmotaconfig
  _, body = get_tasmotaconfig('dl', host, port, username, password, contenttype='application/octet-stream')
File "/usr/local/bin/decode-config", line 3133, in get_tasmotaconfig
  res = requests.get(url, auth=auth)
File "/home/backuper/.local/lib/python3.9/site-packages/requests/api.py", line 75, in get
  return request('get', url, params=params, **kwargs)
File "/home/backuper/.local/lib/python3.9/site-packages/requests/api.py", line 61, in request
  return session.request(method=method, url=url, **kwargs)
File "/home/backuper/.local/lib/python3.9/site-packages/requests/sessions.py", line 528, in request
  prep = self.prepare_request(req)
File "/home/backuper/.local/lib/python3.9/site-packages/requests/sessions.py", line 456, in prepare_request
  p.prepare(
File "/home/backuper/.local/lib/python3.9/site-packages/requests/models.py", line 316, in prepare
  self.prepare_url(url, params)
File "/home/backuper/.local/lib/python3.9/site-packages/requests/models.py", line 393, in prepare_url
  raise InvalidURL("Invalid URL %r: No host supplied" % url)
requests.exceptions.InvalidURL: Invalid URL 'http:///tmp/tasmota01.local.dmp/dl': No host supplied

Expected behavior
decode-config should behave always the same

Version Information (please complete the following information):

$ /usr/local/bin/decode-config -VV
decode-config v9.5.0 [af097d6] by Norbert Richter <[email protected]>

Script:   decode-config
Python:   3.9.5
Platform: Linux-5.11.0-37-generic-x86_64-with - x86_64
OS:       Linux 5.11.0-37-generic #41~20.04.2-Ubuntu SMP Fri Sep 24 09:06:38 UTC 2021
Time:     2021-10-08 19:11:46

• 'system' information of your Tasmota data (append -g System to your decode-config parameter, e. g. decode-config -f myconfig.dmp -g System):

{
  "bootcount": 691,
  "bootcount_reset_time": 1593870950,
  "cfg_crc": "0x36c3",
  "cfg_crc32": "0xe496b08b",
  "cfg_holder": 4617,
  "cfg_size": 4096,
  "cfg_timestamp": 1633647601,
  "header": {
    "data": {
      "crc": "0x7c2b",
      "crc32": "0xe496b08b",
      "platform": "ESP82xx",
      "size": 4096,
      "template": {
        "crc": "0x36c3",
        "crc32": "0xe496b08b",
        "size": 4096,
        "version": "0x8030100"
      },
      "version": "0x8030100"
    },
    "env": {
      "platform": "Linux-5.11.0-37-generic-x86_64-with-Ubuntu-20.04-focal",
      "python": "3.6.9",
      "script": "decode-config.py v9.5.0",
      "system": "Linux x86_64 5.11.0-37-generic #41~20.04.2-Ubuntu SMP Fri Sep 24 09:06:38 UTC 2021"
    },
    "timestamp": "2021-10-07 23:00:01"
  },
  "version": "0x8030100"
}

Additional context
Add any other context about the problem here.

Convert ESP8266 configuration file to ESP32 configuration file

Is your feature request related to a problem? Please describe

I am in the process of swapping a lot of ESP8266 devices into ESP32 devices. e.g. Shelly 1/2.... to Shelly Plus.
It would be great if the configuration could be converted here, e.g. shutter settings, setoptions, MQTT, wifi, etc.
Currently this does not seem possible.

Describe the solution you'd like

Maybe you could allow it after all and leave out the GPIO configurations and set other things that are ESP8266 "only" and ESP32 default settings. This could save a lot of time when "upgrading" and only have to worry about the "specialties".

Describe alternatives you've considered

I converted a file "manually" in the editor (I forgot what I configured, long time ago) and it was possible in many places. On the whole, many settings are identical.

Additional context

An alternative would be if the possible parts could be read in right away during the import into Tasmote and you get a short hint what could not be taken over. Then you can take care of it.

The whole thing could perhaps work similar to the Fritzbox, when you migrate from an older hardware to a new one. (if known).

Python error (Windows Executable)

When executing decode-config_win64.exe I get the error
Error loading Python DLL C:\Users\ADMIN~1\AppDataLocal\Temp\....\python36.dll
I have python 3.8. installed, but that should not matter for an executable.