Startup hangs with pyserial-asyncio-fast

[acseven]

Startup hangs with pyserial-asyncio-fast

[zxdavb]

I am sorry - I have run out of ideas.

IMO, something is happening at the (virtualised) hardware layer.

The next step would be to take ProxMox completely out of the equation.

[acseven]

@zxdavb Just to let you know that this all ended up originating in the USB not being properly seated in the connectors, even though I tried a multiple of them; . I got a few verifications requests from Peter and ended up reaching that conclusion, the dongle needs to be somewhat forcibly plugged in such a position so that it connects properly. Apologies for such an underwhelming conclusion, but it is what it is. On to configuring now.

[zxdavb]

The behaviour you describe, being quite erratic, lead me to suspect a hardware problem.

I see no way that ramses_rf can cause a CPU leakage - it does not write direct to any hardware - it goes through the venerable pyserial library. There is no evidence in the screen captures you provide to justify blaming ramses_cc.

I do not do any testing on Proxmox, so that's a complicating factor. Perhaps you could describe your proxmox settings w.r.t. the serial port?

I have asked the community for their input: https://community.home-assistant.io/t/honeywell-ch-dhw-via-rf-evohome-sundial-hometronics-chronotherm/151584/4401 (this is the best thread, as it is much more active).

But you imply that when it does work, it continues to work fine? By 'fine', I mean packets are received ongoing, until you next restart HA? Please confirm.

2024-04-30T19:35:21.496069 # ramses_tx 0.31.19
2024-04-30T19:35:21.555956 000  I --- 18:003599 63:262142 --:------ 7FFF 016 0010018F304A381B76302E33312E3139

Above, the 7FFF packet tells us that the USB dongle received this command from ramses_rf (i.e. from ramses_cc), and the echo from the dongle was subsequently received by ramses_rf.

Thus, communication between the two is established & there is no evidence of any issue in this instance...

During startup, ramses_rf will send up to 24 of these commands, up to 20 a second, and give up after that time (or 3 seconds, whichever comes first):

_SIGNATURE_GAP_SECS = 0.05
_SIGNATURE_MAX_TRYS = 24
_SIGNATURE_MAX_SECS = 3

My experience is that one (or two) such echo is enough; in your case, it sent two of these only because the 2nd command was sent before the 1st echo was received - a matter of timing, and quite OK.

You didn't mention seeing a message in your HA log file like so:

Failed to initialise Transport within 3 secs

... or::

Transport did not initialise successfully

Please have a look for them & let me know.

So, I wonder if - with a VM - these values are not generous enough, maybe:

_SIGNATURE_GAP_SECS = 0.05
_SIGNATURE_MAX_TRYS = 100
_SIGNATURE_MAX_SECS = 5

I will make it so, for the next update.

[zxdavb]

You could try running in read-only mode & seeing what that does:

ramses_cc:
  ramses_rf:
    disable_sending: true

[acseven]

Many thanks on the extensive reply. I'll try to chip in on all your comments/questions:

I see no way that ramses_rf can cause a CPU leakage - it does not write direct to any hardware - it goes through the venerable pyserial library. There is no evidence in the screen captures you provide to justify blaming ramses_cc.

The thing is, it's very clear that it is this specific integration that causes it. I have this HA setup for quite a long time (over a year), using other USB dongles with passthrough (z-wave and SkyConnect) which work as expected, and had never anything like this. By killing the integration, HA just spins up as expected. I'm struggling to find ways of showing logs or anything that will reflect the behavior experienced, but at the moment is less erratic (which was in the beginning) and is now more consistent: if the integration is somehow enabled HA Core will just stop responding to any request after a few seconds of booting up.

Update: I just bumped into this post that explains a bit more options for debugging (and that grabbed my attention for instability reports with the latest version of HA Core):
https://community.home-assistant.io/t/2024-5-tracking-down-instability-issues-caused-by-integrations/724441

Update 1: I upgraded HA Core to 2025.5.1

I do not do any testing on Proxmox, so that's a complicating factor. Perhaps you could describe your proxmox settings w.r.t. the serial port?

I've test all the options for the USB passthrough, but all return the same behavior.

I have asked the community for their input: https://community.home-assistant.io/t/honeywell-ch-dhw-via-rf-evohome-sundial-hometronics-chronotherm/151584/4401 (this is the best thread, as it is much more active).

Thanks, checked the one reply there and they're using the same USB stick as me.

But you imply that when it does work, it continues to work fine? By 'fine', I mean packets are received ongoing, until you next restart HA? Please confirm.

By 'fine' I mean I see packet logs in the file, as the configuration documentation makes to expect. But I wasn't able to go any further than the log where you saw the packets flowing. Just that one time.

You didn't mention seeing a message in your HA log file like so:

At this moment the only thing HA logs mentions of ramses is what is in the second screenshot I pasted in the OP. It just stops working right after.

You could try running in read-only mode & seeing what that does:

That works! at least it did right now:

2024-05-03 17:02:01.570 WARNING (SyncWorker_5) [homeassistant.loader] We found a custom integration ramses_cc which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-05-03 17:02:07.559 WARNING (MainThread) [ramses_tx.schemas] Best practice is exactly one gateway (HGI) in the known_list: but, len([]) = 0
2024-05-03 17:02:07.559 WARNING (MainThread) [ramses_tx.schemas] It is strongly recommended to provide a known_list, and use it as a whitelist (device_id filter), configure: enforce_known_list = True

However, there's no ramses CC integration added to HA in read only mode.

Some additional logs after adding a bunch of logger lines to the confgiuration.yaml file; it mentions it's using a cached schema. Is there a way of resetting that?

This one a lot of times:

2024-05-03 17:40:48.367 DEBUG (MainThread) [homeassistant.bootstrap] Waiting on tasks: {<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.12/asyncio/futures.py:387, <1 more>, Task.task_wakeup()]>, <Task pending name='setup component ramses_cc' coro=<async_setup_component() running at /usr/src/homeassistant/homeassistant/setup.py:166> wait_for=<Future pending cb=[_chain_future.<locals>._call_check_cancel() at /usr/local/lib/python3.12/asyncio/futures.py:387, <1 more>, Task.task_wakeup()]> cb=[set.remove(), gather.<locals>._done_callback() at /usr/local/lib/python3.12/asyncio/tasks.py:767]>}
2024-05-03 17:40:48.368 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'ramses_cc': <ramses_cc: 0 / 1>}

2024-05-03 17:40:57.407 DEBUG (MainThread) [homeassistant.loader] Component ramses_cc import took 0.079 seconds (loaded_executor=True)
2024-05-03 17:40:57.407 INFO (MainThread) [homeassistant.setup] Setting up ramses_cc
2024-05-03 17:40:57.408 INFO (MainThread) [custom_components.ramses_cc.schemas] Using the cached schema
2024-05-03 17:40:57.408 WARNING (MainThread) [ramses_tx.schemas] Best practice is exactly one gateway (HGI) in the known_list: but, len([]) = 0
2024-05-03 17:40:57.408 WARNING (MainThread) [ramses_tx.schemas] It is strongly recommended to provide a known_list, and use it as a whitelist (device_id filter), configure: enforce_known_list = True
2024-05-03 17:45:57.413 INFO (MainThread) [custom_components.ramses_cc.broker] Saving the client state cache (packets, schema)

I also tested without the read only mode and on 2025.5.1 again, but the behavior is the same (Core stops responding). I can't get logs easily right now due to being outside the LAN.

Many thanks for your support.

[acseven]

I had some time to delve into this again, with weird findings.

So, before HA stops responding, ramses logs appear to be as expected:

2024-05-04 15:31:15.834 WARNING (SyncWorker_3) [homeassistant.loader] We found a custom integration ramses_cc which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-05-04 15:31:17.058 INFO (MainThread) [homeassistant.bootstrap] Setting up stage 2: {'filesize', 'sonoff', 'nut', 'oralb', 'homeassistant_hardware', 'owntracks', 'thread', 'scene', 'backup', 'sensor', 'monitor_docker', 'webrtc', 'file_upload', 'logbook', 'ffmpeg', 'lovelace', 'automation', 'plex', 'openweathermap', 'search', 'config', 'input_number', 'mqtt', 'ble_monitor', 'ipp', 'proxmoxve', 'zone', 'scheduler', 'nmap_tracker', 'input_button', 'assist_pipeline', 'mobile_app', 'moon', 'default_config', 'input_select', 'application_credentials', 'homekit', 'ios', 'device_automation', 'input_text', 'pi_hole', 'notify', 'ramses_cc', 'apple_tv', 'workday', 'proximity', 'bodymiscale', 'blueprint', 'group', 'device_tracker', 'ipma', 'binary_sensor', 'tasmota', 'image_upload', 'persistent_notification', 'integration', 'homeassistant_sky_connect', 'holidays', 'otbr', 'google_translate', 'thermal_comfort', 'person', 'switch_as_x', 'script', 'system_health', 'input_datetime', 'conversation', 'portainer', 'map', 'wake_on_lan', 'home_connect', 'toshiba_ac', 'battery_notes', 'diagnostics', 'tts', 'xiaomi_ble', 'shell_command', 'trace', 'template', 'zwave_js', 'tag', 'climate', 'anniversaries', 'hacs', 'homeassistant_alerts', 'media_source', 'uptimerobot', 'intent', 'androidtv', 'sun', 'measureit', 'history', 'generic', 'input_boolean', 'unifi', 'timer', 'hardware', 'synology_dsm', 'energy', 'wake_word', 'analytics', 'utility_meter', 'onboarding', 'counter', 'schedule', 'my', 'bluetooth_adapters', 'stream', 'rest', 'stt'}
2024-05-04 15:31:17.082 DEBUG (MainThread) [homeassistant.setup] Dependency ramses_cc will wait for dependencies dict_keys(['mqtt'])
2024-05-04 15:31:17.884 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'oralb': <oralb: 0 / 1>, 'owntracks': <owntracks: 0 / 1>, 'mobile_app': <mobile_app: 0 / 1>, 'default_config': <default_config: 0 / 1>, 'ramses_cc': <ramses_cc: 0 / 1>, 'tasmota': <tasmota: 0 / 1>, 'homeassistant_sky_connect': <homeassistant_sky_connect: 0 / 1>, 'otbr': <otbr: 0 / 1>, 'home_connect': <home_connect: 0 / 1>, 'xiaomi_ble': <xiaomi_ble: 0 / 1>, 'analytics': <analytics: 0 / 1>}
2024-05-04 15:31:18.890 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'owntracks': <owntracks: 0 / 1>, 'mobile_app': <mobile_app: 0 / 1>, 'default_config': <default_config: 0 / 1>, 'ramses_cc': <ramses_cc: 0 / 1>, 'tasmota': <tasmota: 0 / 1>, 'homeassistant_sky_connect': <homeassistant_sky_connect: 0 / 1>, 'home_connect': <home_connect: 0 / 1>, 'analytics': <analytics: 0 / 1>, 'switch': <switch: 0 / 1>}
2024-05-04 15:31:22.652 DEBUG (MainThread) [homeassistant.loader] Component ramses_cc import took 3.453 seconds (loaded_executor=True)
2024-05-04 15:31:22.653 INFO (MainThread) [homeassistant.setup] Setting up ramses_cc
2024-05-04 15:31:22.653 INFO (MainThread) [custom_components.ramses_cc.schemas] Using the cached schema
2024-05-04 15:31:22.661 WARNING (MainThread) [ramses_tx.schemas] Best practice is exactly one gateway (HGI) in the known_list: but, len([]) = 0
2024-05-04 15:31:22.662 WARNING (MainThread) [ramses_tx.schemas] It is strongly recommended to provide a known_list, and use it as a whitelist (device_id filter), configure: enforce_known_list = True
2024-05-04 15:31:22.662 INFO (MainThread) [ramses_tx.protocol] The known_list should include exactly one gateway (HGI), but does not (make sure you specify class: HGI)
2024-05-04 15:31:22.744 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:22.849 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:22.990 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:23.174 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:23.230 DEBUG (MainThread) [homeassistant.bootstrap] Integration remaining: defaultdict(<class 'float'>, {'plex': 5.057080840999333, 'pi_hole': 3.6258307239986607, 'apple_tv': 3.2426433540003927, 'workday': 3.220161987999745, 'bodymiscale': 6.412747069998659, 'ipma': 3.1584538009992684, 'google_translate': 2.881641148998824, 'portainer': 2.8279264489992784, 'toshiba_ac': 2.747958326999651, 'battery_notes': 34.448957873994004, 'hacs': 1.656660839000324, 'uptimerobot': 1.6551393150002696, 'androidtv': 1.6542452619996766, 'measureit': 3.3070310950006387, 'synology_dsm': 1.6026008499993623, 'utility_meter': 1.5967709870001272, 'unifi': 1.5965252949990827, 'proximity': 1.5364360999992641, 'ios': 1.518246038000143, 'xiaomi_ble': 1.4957137490000605, 'oralb': 1.4000832149995404, 'otbr': 1.0317468569992343, 'ramses_cc': 0.5770354569995106, 'tasmota': 0.3996083499987435, 'owntracks': 0.3916138900003716})
2024-05-04 15:31:23.230 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'sensor': <sensor: 1 / 0>, 'ramses_cc': <ramses_cc: 1 / 0>}
2024-05-04 15:31:23.233 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:23.295 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:23.375 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:23.458 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:23.564 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:23.743 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:23.837 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:23.907 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:23.998 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:24.143 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:24.204 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:24.233 DEBUG (MainThread) [homeassistant.bootstrap] Integration remaining: defaultdict(<class 'float'>, {'workday': 4.223105728000519, 'ipma': 4.161397541000042, 'google_translate': 3.8845848889995978, 'portainer': 3.830870189000052, 'toshiba_ac': 3.7509020670004247, 'battery_notes': 50.49605771400638, 'hacs': 2.6596045790010976, 'uptimerobot': 2.6580830550010432, 'androidtv': 2.6571890020004503, 'measureit': 5.312918575002186, 'synology_dsm': 2.605544590000136, 'utility_meter': 2.599714727000901, 'unifi': 2.5994690349998564, 'proximity': 2.539379840000038, 'ios': 2.5211897780009167, 'xiaomi_ble': 2.498657489000834, 'oralb': 2.403026955000314, 'otbr': 2.034690597000008, 'ramses_cc': 1.5799791970002843, 'tasmota': 1.4025520899995172, 'owntracks': 1.3945576300011453, 'home_connect': 0.9736377280005399})
2024-05-04 15:31:24.233 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'ramses_cc': <ramses_cc: 1 / 0>}
2024-05-04 15:31:24.270 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:24.326 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'
2024-05-04 15:31:24.402 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F4404497776302E33312E3139\r\n'

As far as I can tell, the last log entry before it stops responding is relative to ´zigpy´:

2024-05-04 15:31:24.875 INFO (ImportExecutor_0) [zigpy.serial] Using pyserial-asyncio-fast in place of pyserial-asyncio

I can't understand how this is related. I use zigbee2mqtt for zigbee devices.

[zxdavb]

The thing is, it's very clear that it is this specific integration that causes it.

So now I'm thinking maybe an infinite loop in my code? This makes sense, as it would be in the main thread (it wouldn't be a recursive loop, as that would break out with an exception).

Note to self: change ramses_cc so that setup() returns True if the packet source simply exists, rather than waiting to confirm that it is spewing out packets.

However, there's no ramses CC integration added to HA in read only mode.

There should be.

Using pyserial-asyncio-fast in place of pyserial-asyncio...

I can't understand how this is related. I use zigbee2mqtt for zigbee devices.

And Boom!

This makes sense to me.

The obvious thing for your to try is restarting HA with zigbee disabled. That information would be useful to me.

I guess the solution is for ramses_cc to use pyserial-asyncio-fast and not pyserial-asyncio...

It is not clear to me if HA obligates that I must be using pyserial-asyncio-fast, or not. Ihave looked around, but no information have been make available to me.

[zxdavb]

So we have this in zigpy:

try:
    import serial_asyncio_fast as pyserial_asyncio

    LOGGER.info("Using pyserial-asyncio-fast in place of pyserial-asyncio")
except ImportError:
    import serial_asyncio as pyserial_asyncio

I wonder what would happen if I do the same?

[acseven]

This makes sense to me.

Good enough for me then! :-)

The obvious thing for your to try is restarting HA with zigbee disabled. That information would be useful to me.

I actually had done already, given the zigpy being there, and it didn't make a difference. In fact, given I'm using zigbee2mqtt wouldn't it make sense that no actual zigbee is beign used by HA Core? I did also disable all addons and possible integrations that would make any use of the zigbee coordinator (SkyConnect, also with Thread support), but that zigpy line still appears.

So we have this in zigpy:

try:
    import serial_asyncio_fast as pyserial_asyncio

    LOGGER.info("Using pyserial-asyncio-fast in place of pyserial-asyncio")
except ImportError:
    import serial_asyncio as pyserial_asyncio

I wonder what would happen if I do the same?

Is this something I can test?

[zxdavb]

It all depends on how technical you are.

• can you edit the manifest.json file of the ramses_cc integration
• are you comfortable to edit the python files of the ramses_cc integration directly?

Or maybe wait until I have some spare time to do some testing...

[acseven]

If it's editing files in the config/custom_components integration folder, that should be fine!

[zxdavb]

In 0.x.20 (i.e. 0.31.20 and 0.41.20), which uses ramses_rf version 0.31.20, ramses_rf now includes:

try:
    import serial_asyncio_fast as pyserial_asyncio

    LOGGER.info("Using pyserial-asyncio-fast in place of pyserial-asyncio")
except ImportError:
    import serial_asyncio as pyserial_asyncio

Hopefully released today - YMMV.

[acseven]

Thank you the the update.

I've ran this with 0.31.20, first with read only - it runs though still doesn't show an integration.

However, without the read only flag, HA Core behaves the same way, eventually it just stops responding to any requests and only a hard shutdown is able to solve it.

Current logs (filtered by "ramses" occurences):

2024-05-08 11:30:14.021 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration ramses_cc which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-05-08 11:30:15.475 INFO (MainThread) [homeassistant.bootstrap] Setting up stage 2: {'logbook', 'stt', 'schedule', 'map', 'sun', 'assist_pipeline', 'search', 'input_datetime', 'trace', 'history', 'lovelace', 'filesize', 'tts', 'moon', 'input_text', 'home_connect', 'rest', 'onboarding', 'device_tracker', 'ios', 'bluetooth_adapters', 'measureit', 'plex', 'scheduler', 'nmap_tracker', 'nut', 'owntracks', 'monitor_docker', 'uptimerobot', 'tasmota', 'device_automation', 'mobile_app', 'homeassistant_sky_connect', 'proxmoxve', 'input_number', 'group', 'integration', 'persistent_notification', 'battery_notes', 'shell_command', 'synology_dsm', 'xiaomi_ble', 'diagnostics', 'thread', 'input_boolean', 'zone', 'switch_as_x', 'proximity', 'mqtt', 'person', 'media_source', 'application_credentials', 'homekit', 'blueprint', 'system_health', 'ipp', 'sensor', 'ble_monitor', 'input_button', 'image_upload', 'hardware', 'default_config', 'sonoff', 'ramses_cc', 'wake_on_lan', 'binary_sensor', 'timer', 'thermal_comfort', 'wake_word', 'stream', 'google_translate', 'generic', 'portainer', 'backup', 'pi_hole', 'toshiba_ac', 'androidtv', 'utility_meter', 'homeassistant_alerts', 'scene', 'template', 'ffmpeg', 'hacs', 'tag', 'my', 'notify', 'unifi', 'holidays', 'openweathermap', 'climate', 'input_select', 'conversation', 'automation', 'zwave_js', 'config', 'script', 'analytics', 'counter', 'file_upload', 'apple_tv', 'anniversaries', 'webrtc', 'ipma', 'bodymiscale', 'otbr', 'energy', 'workday', 'homeassistant_hardware', 'intent', 'oralb'}
2024-05-08 11:30:15.531 DEBUG (MainThread) [homeassistant.setup] Dependency ramses_cc will wait for dependencies dict_keys(['mqtt'])
2024-05-08 11:30:16.109 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'zone': <zone: 1 / 0>, 'device_tracker': <device_tracker: 0 / 1>, 'conversation': <conversation: 0 / 1>, 'assist_pipeline': <assist_pipeline: 0 / 1>, 'home_connect': <home_connect: 0 / 1>, 'ios': <ios: 0 / 1>, 'owntracks': <owntracks: 0 / 1>, 'tasmota': <tasmota: 0 / 1>, 'mobile_app': <mobile_app: 0 / 1>, 'homeassistant_sky_connect': <homeassistant_sky_connect: 0 / 1>, 'input_number': <input_number: 1 / 0>, 'xiaomi_ble': <xiaomi_ble: 0 / 1>, 'input_boolean': <input_boolean: 1 / 0>, 'proximity': <proximity: 0 / 1>, 'stream': <stream: 1 / 0>, 'default_config': <default_config: 0 / 1>, 'ramses_cc': <ramses_cc: 0 / 1>, 'analytics': <analytics: 0 / 1>, 'counter': <counter: 1 / 0>, 'otbr': <otbr: 0 / 1>, 'oralb': <oralb: 0 / 1>}
2024-05-08 11:30:22.725 WARNING (ImportExecutor_0) [ramses_tx.transport] EXPERIMENTAL: Using pyserial-asyncio-fast in place of pyserial-asyncio
2024-05-08 11:30:22.780 DEBUG (MainThread) [homeassistant.loader] Component ramses_cc import took 5.677 seconds (loaded_executor=True)
2024-05-08 11:30:22.782 INFO (MainThread) [homeassistant.setup] Setting up ramses_cc
2024-05-08 11:30:22.783 INFO (MainThread) [custom_components.ramses_cc.schemas] Using the cached schema
2024-05-08 11:30:22.783 WARNING (MainThread) [ramses_tx.schemas] Best practice is to provide a known_list and enforce it, configure: enforce_known_list = True
2024-05-08 11:30:22.783 WARNING (MainThread) [ramses_tx.protocol] The known_list SHOULD include exactly one gateway (HGI), but does not (it should specify 'class: HGI')
2024-05-08 11:30:22.789 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:22.842 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:23.025 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:23.149 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:23.196 DEBUG (MainThread) [homeassistant.bootstrap] Integration remaining: defaultdict(<class 'float'>, {'synology_dsm': 6.141085041977931, 'google_translate': 5.299921925005037, 'portainer': 5.2770999890053645, 'pi_hole': 5.257073431974277, 'toshiba_ac': 5.1811121460050344, 'androidtv': 5.108338592981454, 'utility_meter': 5.102612694026902, 'hacs': 4.8384813540033065, 'unifi': 4.309434861002956, 'openweathermap': 3.28592973400373, 'apple_tv': 2.702721412992105, 'ipma': 2.40619098598836, 'bodymiscale': 4.7844407230149955, 'workday': 2.3841578389983624, 'ios': 2.374978377018124, 'proximity': 2.3687572620110586, 'home_connect': 2.3155373519985005, 'xiaomi_ble': 2.1732950640143827, 'oralb': 2.1648131809779443, 'otbr': 0.5688369349809363, 'ramses_cc': 0.412768984970171, 'owntracks': 0.40289926301920786, 'tasmota': 0.3511867880006321})
2024-05-08 11:30:23.196 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'ramses_cc': <ramses_cc: 1 / 0>}
2024-05-08 11:30:23.246 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:23.375 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:23.436 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:23.513 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:23.587 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:23.657 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:23.738 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:23.804 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:24.211 DEBUG (MainThread) [homeassistant.bootstrap] Integration remaining: defaultdict(<class 'float'>, {'toshiba_ac': 6.196296857029665, 'androidtv': 6.123523304006085, 'unifi': 5.324619572027586, 'ipma': 3.4213756970129907, 'workday': 3.3993425500229932, 'home_connect': 3.3307220630231313, 'xiaomi_ble': 3.1884797750390135, 'oralb': 3.179997892002575, 'otbr': 1.5840216460055672, 'ramses_cc': 1.4279536959948018, 'owntracks': 1.4180839740438387, 'tasmota': 1.366371499025263, 'ios': 0.41263198404340073})
2024-05-08 11:30:24.211 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'ramses_cc': <ramses_cc: 1 / 0>}
2024-05-08 11:30:24.311 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:24.508 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:25.180 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:25.241 DEBUG (MainThread) [homeassistant.bootstrap] Integration remaining: defaultdict(<class 'float'>, {'toshiba_ac': 7.226852822001092, 'ipma': 4.451931661984418, 'home_connect': 4.361278027994558, 'ramses_cc': 2.4585096609662287})
2024-05-08 11:30:25.242 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'ramses_cc': <ramses_cc: 1 / 0>, 'sensor': <sensor: 1 / 0>, 'weather': <weather: 1 / 0>, 'climate': <climate: 1 / 0>, 'select': <select: 1 / 0>, 'switch': <switch: 1 / 0>}
2024-05-08 11:30:25.246 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:25.318 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:25.398 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'

Here is an excerpt of the middle of the log, when ramses loads:

2024-05-08 11:30:21.235 DEBUG (MainThread) [homeassistant.setup] Phase platform_setup for mobile_app (139824207502400) took 0.08s (elapsed=0.08s) (wait_time=0.00s)
2024-05-08 11:30:21.333 DEBUG (MainThread) [homeassistant.setup] Phase platform_setup for monitor_docker (139824327554176) took 0.11s (elapsed=0.11s) (wait_time=0.00s)
2024-05-08 11:30:21.867 DEBUG (MainThread) [homeassistant.setup] Phase config_entry_setup for plex (5d3165b8f3a87da9638f878fe3aa0789) took 0.87s (elapsed=5.44s) (wait_time=4.57s)
2024-05-08 11:30:21.928 INFO (ImportExecutor_0) [zigpy.serial] Using pyserial-asyncio-fast in place of pyserial-asyncio
2024-05-08 11:30:22.082 DEBUG (MainThread) [homeassistant.setup] Phase config_entry_setup for sonoff (cc5c94efb530cc3cdb3c65cb10711358) took 4.22s (elapsed=4.22s) (wait_time=0.00s)
2024-05-08 11:30:22.169 DEBUG (MainThread) [homeassistant.bootstrap] Integration remaining: defaultdict(<class 'float'>, {'uptimerobot': 5.6258740210323595, 'synology_dsm': 5.114531430997886, 'battery_notes': 76.77472195925657, 'google_translate': 4.273368314024992, 'portainer': 4.250546378025319, 'pi_hole': 4.230519820994232, 'toshiba_ac': 4.154558535024989, 'androidtv': 4.081784982001409, 'utility_meter': 4.076059083046857, 'hacs': 3.8119277430232614, 'unifi': 3.2828812500229105, 'openweathermap': 2.259376123023685, 'apple_tv': 1.6761678020120598, 'ipma': 1.3796373750083148, 'bodymiscale': 2.7313335010549054, 'workday': 1.3576042280183174, 'ios': 1.3484247660380788, 'proximity': 1.3422036510310136, 'home_connect': 1.2889837410184555, 'xiaomi_ble': 1.1467414530343376, 'oralb': 1.1382595699978992})
2024-05-08 11:30:22.169 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {}
2024-05-08 11:30:22.615 DEBUG (MainThread) [homeassistant.loader] Component homeassistant_sky_connect import took 5.537 seconds (loaded_executor=True)
2024-05-08 11:30:22.615 INFO (MainThread) [homeassistant.setup] Setting up homeassistant_sky_connect
2024-05-08 11:30:22.615 INFO (MainThread) [homeassistant.setup] Setup of domain homeassistant_sky_connect took 0.00 seconds
2024-05-08 11:30:22.616 DEBUG (MainThread) [homeassistant.setup] Phase config_entry_setup for homeassistant_sky_connect (b637404a3d9100a3d19b541cb3140aca) took 0.00s (elapsed=0.00s) (wait_time=0.00s)
2024-05-08 11:30:22.626 DEBUG (MainThread) [homeassistant.loader] Component otbr import took 5.548 seconds (loaded_executor=True)
2024-05-08 11:30:22.626 INFO (MainThread) [homeassistant.setup] Setting up otbr
2024-05-08 11:30:22.627 INFO (MainThread) [homeassistant.setup] Setup of domain otbr took 0.00 seconds
2024-05-08 11:30:22.633 DEBUG (MainThread) [homeassistant.loader] Importing platforms for scheduler executor=['switch'] loop=[] took 5.55s
2024-05-08 11:30:22.633 INFO (MainThread) [homeassistant.components.switch] Setting up scheduler.switch
2024-05-08 11:30:22.639 DEBUG (MainThread) [homeassistant.setup] Phase config_entry_platform_setup for scheduler (3646aa123c2a4876a11518bbb72921db) took -0.60s (elapsed=0.01s) (wait_time=0.61s)
2024-05-08 11:30:22.654 WARNING (MainThread) [homeassistant.helpers.frame] Detected that custom integration 'scheduler' calls `async_track_state_change` instead of `async_track_state_change_event` which is deprecated and will be removed in Home Assistant 2025.5 at custom_components/scheduler/timer.py, line 209: self._sun_tracker = async_track_state_change(, please create a bug report at https://github.com/nielsfaber/scheduler-component/issues
2024-05-08 11:30:22.665 DEBUG (MainThread) [homeassistant.loader] Importing platforms for monitor_docker executor=['switch'] loop=[] took 5.58s
2024-05-08 11:30:22.665 INFO (MainThread) [homeassistant.components.switch] Setting up monitor_docker.switch
2024-05-08 11:30:22.667 DEBUG (MainThread) [homeassistant.setup] Phase platform_setup for monitor_docker (139824325532800) took 0.00s (elapsed=0.00s) (wait_time=0.00s)
2024-05-08 11:30:22.725 WARNING (ImportExecutor_0) [ramses_tx.transport] EXPERIMENTAL: Using pyserial-asyncio-fast in place of pyserial-asyncio
2024-05-08 11:30:22.780 DEBUG (MainThread) [homeassistant.loader] Component ramses_cc import took 5.677 seconds (loaded_executor=True)
2024-05-08 11:30:22.782 INFO (MainThread) [homeassistant.setup] Setting up ramses_cc
2024-05-08 11:30:22.783 INFO (MainThread) [custom_components.ramses_cc.schemas] Using the cached schema
2024-05-08 11:30:22.783 WARNING (MainThread) [ramses_tx.schemas] Best practice is to provide a known_list and enforce it, configure: enforce_known_list = True
2024-05-08 11:30:22.783 WARNING (MainThread) [ramses_tx.protocol] The known_list SHOULD include exactly one gateway (HGI), but does not (it should specify 'class: HGI')
2024-05-08 11:30:22.789 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:22.792 DEBUG (MainThread) [homeassistant.loader] Component owntracks import took 5.689 seconds (loaded_executor=True)
2024-05-08 11:30:22.792 INFO (MainThread) [homeassistant.setup] Setting up owntracks
2024-05-08 11:30:22.793 INFO (MainThread) [homeassistant.setup] Setup of domain owntracks took 0.00 seconds
2024-05-08 11:30:22.842 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:22.844 DEBUG (MainThread) [homeassistant.loader] Component tasmota import took 5.740 seconds (loaded_executor=True)
2024-05-08 11:30:22.844 INFO (MainThread) [homeassistant.setup] Setting up tasmota

And the final lines of the full log:

2024-05-08 11:30:25.361 DEBUG (MainThread) [homeassistant.loader] Importing platforms for light executor=['device_action'] loop=[] took 0.02s
2024-05-08 11:30:25.368 INFO (MainThread) [homeassistant.setup] Setting up automation
2024-05-08 11:30:25.380 INFO (MainThread) [homeassistant.setup] Setup of domain automation took 0.01 seconds
2024-05-08 11:30:25.392 DEBUG (MainThread) [homeassistant.loader] Importing platforms for automation executor=['logbook'] loop=[] took 0.01s
2024-05-08 11:30:25.398 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F57C1154476302E33312E3230\r\n'
2024-05-08 11:30:25.428 INFO (SyncWorker_14) [homeconnect.sseclient] Connecting ...
2024-05-08 11:30:25.507 INFO (Thread-10 (_listen)) [homeconnect] Listening to event stream for all devices
2024-05-08 11:30:37.683 INFO (Thread-8 (_thread_main)) [azure.iot.device.common.mqtt_transport] message received on $iothub/methods/POST/smmobile/?$rid=1

[acseven]

Also, don't know if using 0.41.20 would help, but to try that one is there any way of setting read_only mode in the config_flow?

[zxdavb]

Yes, you can do this.

• Settings > Devices & Services > RAMSES RF > CONFIGURE > Gateway configuration

Enter:

disable_sending: true

... and click on SUBMIT.

I believe you then have to:

• Settings (three dot menu) > Reload

[zxdavb]

I am not sure about this:

[homeassistant.setup] Dependency ramses_cc will wait for dependencies dict_keys(['mqtt'])

Can you try installing MQTT (even if you don't use it). Or you can edit the manifest.yaml so that it removes MQTT (only edit the dependencies key):

{
    "domain": "ramses_cc",
    "name": "RAMSES RF",
    "codeowners": ["@zxdavb"],
    "dependencies": [],
    "documentation": "https://github.com/zxdavb/ramses_cc",
    "issue_tracker": "https://github.com/zxdavb/ramses_cc/issues",
...

[zxdavb]

I am not sure about this:

11:30:25.241 ... bootstrap] Integration remaining: defaultdict(... 'ramses_cc': 2.4585096609662287})
11:30:25.242 ... bootstrap] Running timeout Zones: {'ramses_cc': <ramses_cc: 1 / 0>, ...

... never really seen it before.

[acseven]

Yes, you can do this.

• Settings > Devices & Services > RAMSES RF > CONFIGURE > Gateway configuration

Enter:

disable_sending: true

... and click on SUBMIT.

I believe you then have to:

• Settings (three dot menu) > Reload

I remembered that I have to edit the configuration.yaml to get HA core access back, so it's better to keep using 0.3x.xx while getting this isn't working properly. I don't know where to edit the configuration which will be set by the flow, but even if it's a file as well it's likely to be a lot more difficult to find ramses there than on configuration.yaml.

[homeassistant.setup] Dependency ramses_cc will wait for dependencies dict_keys(['mqtt'])

I have MQTT installed and running (many entities, including from Zigbee2MQTT), I guess that means it's just waiting MQTT to load (which it does).

I am not sure about this:

11:30:25.241 ... bootstrap] Integration remaining: defaultdict(... 'ramses_cc': 2.4585096609662287})
11:30:25.242 ... bootstrap] Running timeout Zones: {'ramses_cc': <ramses_cc: 1 / 0>, ...

... never really seen it before.

That I can't comment on as well.

THanks for your help, once again.

[acseven]

Is there any configuration flag that allows clearing ramses_cc's cache?

[zxdavb]

Please see: https://github.com/zxdavb/ramses_cc/wiki/2.-Configuration

... and: https://github.com/zxdavb/ramses_cc/wiki/3.-Troubleshooting-&-Logging

[acseven]

Thank you for the pointers, I had been through the docs, but forgot about that.

Right now, I'm using:

ramses_cc:
  restore_cache:
    restore_schema: false # for initial debug
    restore_state: false # for initial debug
  ramses_rf:
    disable_sending: true # for initial debug
#    disable_sending: false  # production
  serial_port: /dev/serial/by-id/usb-SparkFun_evofw3_atmega32u4-if00
  packet_log:
    file_name: packet.log
    rotate_backups: 7

and

logger:
  #  default: warning
  filters:
    py.warnings:
      - "InsecureRequestWarning"
  default: info
  logs:
    homeassistant.bootstrap: debug
    homeassistant.setup: debug
    homeassistant.loader: debug
    homeassistant.config_entries: debug
    custom_components.ramses_cc: info
    ramses_rf.dispatcher: info

In any case, I've ran this a bunch of times and even though sometimes it doesn't log zigpy, most of the times the log ends with the zigpy.serial line:

2024-05-08 21:49:39.275 INFO (MainThread) [homeassistant.components.light] Setting up switch_as_x.light
2024-05-08 21:49:39.278 DEBUG (MainThread) [homeassistant.setup] Phase config_entry_setup for switch_as_x (8e26b3824990aab0e4e0daccfab05191) took 0.00s (elapsed=0.00s) (wait_time=0.00s)
2024-05-08 21:49:39.279 INFO (MainThread) [homeassistant.components.light] Setting up switch_as_x.light
2024-05-08 21:49:39.282 DEBUG (MainThread) [homeassistant.setup] Phase config_entry_setup for switch_as_x (604c3dcb7683256aea3d15eb320a2d70) took 0.00s (elapsed=0.00s) (wait_time=0.00s)
2024-05-08 21:49:39.282 INFO (MainThread) [homeassistant.components.websocket_api.http.connection] [140290425987520] Received unknown command: mobile_app/push_notification_channel
2024-05-08 21:49:39.282 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F59F807F976302E33312E3230\r\n'
2024-05-08 21:49:39.289 DEBUG (MainThread) [homeassistant.loader] Component switch import took 0.000 seconds (loaded_executor=False)
2024-05-08 21:49:39.289 INFO (MainThread) [homeassistant.setup] Setting up switch
2024-05-08 21:49:39.290 INFO (MainThread) [homeassistant.setup] Setup of domain switch took 0.00 seconds
2024-05-08 21:49:39.301 DEBUG (MainThread) [homeassistant.setup] Adding wait for wait_base_component for scheduler (3646aa123c2a4876a11518bbb72921db) of 2.16
2024-05-08 21:49:39.302 DEBUG (MainThread) [homeassistant.setup] Adding wait for wait_base_component for sonoff (cc5c94efb530cc3cdb3c65cb10711358) of 1.03
2024-05-08 21:49:39.302 INFO (MainThread) [homeassistant.components.switch] Setting up sonoff.switch
2024-05-08 21:49:39.305 DEBUG (MainThread) [homeassistant.loader] Component portainer import took 5.949 seconds (loaded_executor=True)
2024-05-08 21:49:39.306 INFO (MainThread) [homeassistant.setup] Setting up portainer
2024-05-08 21:49:39.306 INFO (MainThread) [homeassistant.setup] Setup of domain portainer took 0.00 seconds
2024-05-08 21:49:39.307 DEBUG (MainThread) [homeassistant.loader] Component google_translate import took 5.951 seconds (loaded_executor=True)
2024-05-08 21:49:39.307 INFO (MainThread) [homeassistant.setup] Setting up google_translate
2024-05-08 21:49:39.307 INFO (MainThread) [homeassistant.setup] Setup of domain google_translate took 0.00 seconds
2024-05-08 21:49:39.308 INFO (MainThread) [homeassistant.components.websocket_api.http.connection] [140290425987520] Received unknown command: mobile_app/push_notification_channel
2024-05-08 21:49:39.339 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F59F807F976302E33312E3230\r\n'
2024-05-08 21:49:41.066 INFO (ImportExecutor_0) [zigpy.serial] Using pyserial-asyncio-fast in place of pyserial-asyncio

I can add the following, which makes me think that this is somehow triggering some host hardware issue: as I mentioned previously I can access HA via the proxmox console just fine, and running top returns high CPU occupancy but no process with actual high occupancy. On the other hand, running top confirms that the HA VM is using up the CPU. Nonetheless, proxmox still runs very fluid. The only actual issue is that HA Core stops replying to any requests.

[acseven]

Updated from proxmox 7.4 to 8.2.2, and still the same. But this could be related to proxmox.

[zxdavb]

Did you try this: https://community.home-assistant.io/t/honeywell-ch-dhw-via-rf-evohome-sundial-hometronics-chronotherm/151584/4418?u=zxdavb

[acseven]

If you mean the USB passthrough, yes, I have tried both options.

At the moment, with the following read only config I get some very bare HA logs and no packet logs:

ramses_cc:
  restore_cache:
    restore_schema: false # for initial debug
    restore_state: false # for initial debug
  ramses_rf:
    disable_sending: true # for initial debug
#    disable_sending: false  # production
  serial_port: /dev/serial/by-id/usb-SparkFun_evofw3_atmega32u4-if00
  packet_log:
    file_name: packet.log
    rotate_backups: 7

2024-05-19 11:32:16.040 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration ramses_cc which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-05-19 11:33:31.043 WARNING (ImportExecutor_0) [ramses_tx.transport] EXPERIMENTAL: Using pyserial-asyncio-fast in place of pyserial-asyncio
2024-05-19 11:33:31.141 DEBUG (MainThread) [homeassistant.loader] Component ramses_cc import took 0.186 seconds (loaded_executor=True)
2024-05-19 11:33:31.141 INFO (MainThread) [homeassistant.setup] Setting up ramses_cc
2024-05-19 11:33:31.143 WARNING (MainThread) [ramses_tx.schemas] Best practice is to provide a known_list and enforce it, configure: enforce_known_list = True
2024-05-19 11:33:31.154 INFO (MainThread) [homeassistant.setup] Setup of domain ramses_cc took 0.01 seconds

Setting disable_sending to false still breaks Home Assistant Core shortly after ramses initiates (grep ramses log below):

31.20 HA log, with `disable_sending` set to `false`

2024-05-19 12:00:43.829 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration ramses_cc which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-05-19 12:00:47.457 WARNING (ImportExecutor_0) [ramses_tx.transport] EXPERIMENTAL: Using pyserial-asyncio-fast in place of pyserial-asyncio
2024-05-19 12:00:47.506 DEBUG (MainThread) [homeassistant.loader] Component ramses_cc import took 1.811 seconds (loaded_executor=True)
2024-05-19 12:00:47.507 INFO (MainThread) [homeassistant.setup] Setting up ramses_cc
2024-05-19 12:00:47.508 WARNING (MainThread) [ramses_tx.schemas] Best practice is to provide a known_list and enforce it, configure: enforce_known_list = True
2024-05-19 12:00:47.508 WARNING (MainThread) [ramses_tx.protocol] The known_list SHOULD include exactly one gateway (HGI), but does not (it should specify 'class: HGI')
2024-05-19 12:00:47.528 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:47.583 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:47.691 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:47.981 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:47.991 DEBUG (MainThread) [homeassistant.bootstrap] Integration remaining: defaultdict(<class 'float'>, {'uptimerobot': 2.029499216005206, 'measureit': 3.9247204689309, 'ipma': 1.8964051899965852, 'plex': 1.835440317983739, 'sonoff': 1.4513579990016297, 'hacs': 0.883039818960242, 'ipp': 0.7880118349567056, 'nmap_tracker': 0.7819328799378127, 'utility_meter': 1.7988148420117795, 'ramses_cc': 0.4839294699486345, 'filesize': 0.2664719599997625, 'battery_notes': 0.5289429936092347, 'nut': 0.00603698892518878})
2024-05-19 12:00:47.991 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'owntracks': <owntracks: 0 / 1>, 'analytics': <analytics: 0 / 1>, 'otbr': <otbr: 0 / 1>, 'mobile_app': <mobile_app: 0 / 1>, 'default_config': <default_config: 0 / 1>, 'tasmota': <tasmota: 0 / 1>, 'erse': <erse: 0 / 1>, 'ramses_cc': <ramses_cc: 1 / 0>}
2024-05-19 12:00:48.035 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.086 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.145 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.197 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.254 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.307 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.364 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.450 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.512 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.569 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.626 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.684 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:48.739 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:49.241 DEBUG (MainThread) [homeassistant.bootstrap] Integration remaining: defaultdict(<class 'float'>, {'uptimerobot': 3.2796064120484516, 'measureit': 6.424934861017391, 'ipma': 3.1465123860398307, 'plex': 3.0855475140269846, 'sonoff': 2.701465195044875, 'hacs': 2.1331470150034875, 'ipp': 2.038119030999951, 'nmap_tracker': 2.032040075981058, 'utility_meter': 5.549136430141516, 'ramses_cc': 1.73403666599188, 'filesize': 1.516579156043008, 'battery_notes': 20.530658130301163, 'nut': 1.2561441849684343, 'moon': 1.2420857379911467, 'androidtv': 1.1210205419920385, 'pi_hole': 1.0863359400536865, 'synology_dsm': 1.0232218829914927, 'toshiba_ac': 0.8741012860555202})
2024-05-19 12:00:49.241 DEBUG (MainThread) [homeassistant.bootstrap] Running timeout Zones: {'default_config': <default_config: 0 / 1>, 'erse': <erse: 0 / 1>, 'ramses_cc': <ramses_cc: 1 / 0>}
2024-05-19 12:00:49.289 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'

Home Assistant logs below end (almost every time) with the zigpy.serial entry I mentioned before. When it isn't there the last line is a ramses_tx.transport entry.

2024-05-19 12:00:49.289 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F9082E12776302E33312E3230\r\n'
2024-05-19 12:00:51.112 INFO (ImportExecutor_0) [zigpy.serial] Using pyserial-asyncio-fast in place of pyserial-asyncio

I'm not using versions 41.x because I don't know of an easy way of setting disable_sending back to false via SSH, which I do in 31.x by editing the configuration.yaml file.

I can also add that with both runs above, no integration entities are created, whereas at sometime in initial tries it did (although with issues).

I've just now tried out 31.16 and although it doesn't prevent HA Core from running fine, ramses_cc fails to load:

31.16 log, with `disable_sending` set to `false`

2024-05-19 12:24:07.943 WARNING (SyncWorker_4) [homeassistant.loader] We found a custom integration ramses_cc which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-05-19 12:24:11.700 DEBUG (MainThread) [homeassistant.loader] Component ramses_cc import took 2.676 seconds (loaded_executor=True)
2024-05-19 12:24:11.701 INFO (MainThread) [homeassistant.setup] Setting up ramses_cc
2024-05-19 12:24:11.706 WARNING (MainThread) [homeassistant.helpers.frame] Detected that custom integration 'ramses_cc' accesses hass.helpers.storage. This is deprecated and will stop working in Home Assistant 2024.11, it should be updated to import functions used from storage directly at custom_components/ramses_cc/broker.py, line 66: self._store: Store = hass.helpers.storage.Store(STORAGE_VERSION, STORAGE_KEY), please create a bug report at https://github.com/zxdavb/ramses_cc/issues
2024-05-19 12:24:11.706 WARNING (MainThread) [ramses_tx.schemas] Best practice is exactly one gateway (HGI) in the known_list: but, len([]) = 0
2024-05-19 12:24:11.706 WARNING (MainThread) [ramses_tx.schemas] It is strongly recommended to provide a known_list, and use it as a whitelist (device_id filter), configure: enforce_known_list = True
2024-05-19 12:24:11.706 INFO (MainThread) [ramses_tx.protocol] The known_list should include exactly one gateway (HGI), but does not (make sure you specify class: HGI)
2024-05-19 12:24:11.716 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:11.784 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:11.840 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:11.891 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:11.942 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:11.997 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.048 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.099 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.149 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.200 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.252 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.303 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.354 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.405 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.461 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.578 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.632 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.692 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.751 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.811 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.874 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.925 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:12.999 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:13.065 INFO (MainThread) [ramses_tx.transport] Tx:     b' I --- 18:000730 63:262142 --:------ 7FFF 016 0010018F90984E4476302E33312E3136\r\n'
2024-05-19 12:24:13.118 ERROR (MainThread) [custom_components.ramses_cc] There is a problem with the serial port: Never received an echo signature
2024-05-19 12:24:13.119 ERROR (MainThread) [homeassistant.setup] Setup failed for custom integration 'ramses_cc': Integration failed to initialize.
2024-05-19 12:24:13.119 INFO (MainThread) [homeassistant.setup] Setup of domain ramses_cc took 1.42 seconds

Also tried out 31.19 and the behavior is pretty much the same as 31.20.

[acseven]

Thank you for your effort in any case. I'll try to see if I can find something in the ProxMox communities.

Would it feasible to run Ramses_cc somehow in an isolated raspberry pi zero? Maybe publishing to MQTT?

[zxdavb]

Would it feasible to run Ramses_cc somehow in an isolated raspberry pi zero? Maybe publishing to MQTT?

Someone would have to write the code - it wouldn't be difficult. The other option would be to re-instate RFC2217. I don't have the time, sorry.

However, what you're wanting: an MQTT version of the evofw3 dongle, is just around the corner...

• see here: https://github.com/IndaloTech/ramses_esp?tab=readme-ov-file
• request to a stock update here: https://indalo-tech.onlineweb.shop/

[zxdavb]

I should still check your dongle outside of you VM environment - if it's a HW problem, then you can have it replaced.

But if you're going to buy a new one, I'd wait for the MQTT version from Indalo Tech.

[acseven]

Thank you for all the help so far. I'll try and check the USB dongle as you suggested first.

The Indalo Tech site store has been down all day, so I haven't been able to check your suggestion.

[acseven]

@zxdavb Do you know if the online shop is coming back?

[acseven]

I should still check your dongle outside of you VM environment - if it's a HW problem, then you can have it replaced.

But if you're going to buy a new one, I'd wait for the MQTT version from Indalo Tech.

Just tested on a raspberry pi, with almost nothing other than a base HASS install, and it's the exact same behavior. I guess this an hardware fault with the dongle

[zxdavb]

Do you know if the online shop is coming back?

https://freewebstore.status.io/

I have been trying to setup ramses-cc with a indalo-tech USB stick...
I guess this an hardware fault with the dongle

In that case, would contact Pete (who sold you the dongle), and ask for next steps. Maybe you could raise an issue on this github repo:
https://github.com/ghoti57/evofw3/issues

[acseven]

I've reached out to Peter and there's an issue with the online store.