mhthies/smarthomeconnect

Python 3 AsyncIO-based home automation and interfacing framework

smarthomeconnect's Issues

web: Add image ui widget with placeable widgets

examples: Create example for using position of sun and weather forecast

examples: Create complex example with multiple modules

web: Use importlib to load static web ressources

Goal is, to make SHC "zip_safe", i.e. make it executable directly from an unextracted wheel-package. This requires to load ressource files in a way that transparently extracts them from from the package if required, e.g. importlib.ressources.

For reference, see

• https://setuptools.pypa.io/en/latest/userguide/miscellaneous.html#setting-the-zip-safe-flag
• https://setuptools.pypa.io/en/latest/userguide/datafiles.html#accessing-data-files-at-runtime

However, I'm not sure if this is possible with our current approach of serving our static web ressources from a single static directory (see shc/web/interface.py:112).

Add Icinga2 API interface

The interface should allow to create connectors for reading the state of hosts or services, monitored by Icinga.

(I'd like to have LED lights in my appartment, changing from green to red when one of my web servers goes bad.)

Make stateful objects Reading to allow initialization from persitance

• Hysteresis
• AbstractRamp (? – for really long ramps?)

web: Switch widget changes visual state incorrectly when confirmation is denied

Create "stripe chart" log widget for boolean and enum variables

Idea: A bar of colored stripes, one stripe for each value change, with its color representing the value (True/False or an enum value) and its with representing the duration until the next value chage.

Current implementation idea: Use Chart.js' API to create a new chart type, possibly based on a stacked bar chart.

Follow up to #9

web: don't start websocket on /monitoring page

Currently, the Javascript for active elements on the web page is loaded by default for every page based on our base.html template, including the monitoring page. However, the monitoring page does not provide dynamic contents (all values are server-side rendered into the template), so the Javascript is not necessary.

As a solution, we could call the on-load JS code on normal SHC UI pages explicitly as a JS function from the HTML template. A Jinja template variable could be used to enable/disable calling this function.

Additionally, we could provide an auto-reload feature for the monitoring page.

shc_client: Only subscribe to API objects if local subscribtions exist

Add client interface library for websocket API

Simultaneous updates of connected variables may cause inconsistent states

When multiple variables are connectected (e.g. in server-client scenarios) and two value updates arrive at the same time (from "different sides"), the value updates may probably "cross over" and result in inconsistent states and different orders of outgoing updates.

examples: Create example of complex logics

Including expressions, logic handlers, internal Variables, BreakableSubscription, Hysteresis, and TimerSwitch.

web: Add support for HTML title suffix

Should be an optional parameter for shc.web.interface.WebServer, taking a suffix string, format string or formatting function for the title.

Add MQTT interface (incl. JSON-over-MQTT)

Ideally, we aim to be compatible with HomeAssistant's MQTT protocol integrations and ESPHome

Add a SceneManager utility class

The idea is to create SceneManager objects, that somehow behave like KNX scenes: You connect a number of connectable objects to the SceneManager (which provides any number of subscribable + reading connectors). Than you can call a method to store the current value of the connected objects as a "scene" with a specific number. Later, you can call another method to recall a stored scene, i.e. publish the stored value to all connected scenes.

The SceneManager should ensure persistence of the stored scenes somehow, e.g. using the JSON-representation of the stored values and writing it to a file (or some other persistent store).

The interface would look like this:

class SceneManager:
    def __init__(persistence_file: Path): ...
    def connector(type_: Type[T], name: str) -> SceneConnector: ...
    async def store_scene(scene_number: int) -> None: ...
    async def recall_scene(scene_number: int) -> None: ...

class SceneConnector(Reading[T], Subscribable[T], Generic[T]): ...

Maybe, there might also be a current_scene connector which allows to read, write and subscribe to the currently selected scene as an int value. In this case, it would also be nice if the SceneConnectors would be writable as well and the scene manager would update the current_scene value if the current values of the connected objects (coincidentally) resemble a specifc stored scene.

Create log plot widget for numeric log variables

Requires #7

Current approach: Use Chart.js

expressions: Support string and bytes operations

Allow writhing logic handler functions without (or just one) parameters

In most cases, logic handler functions do not need the origin argument or even the value argument. (This is because the logic handlers are typically triggered by a timer or by multiple Subscribable objects, so the current values of relevant variables need to be read within the logic handler anyways.) Thus, it would be nice if the value and origin parameters could simply be omitted when defining a logic handler function.

I.e. If a logic handler is defined without parameters, it will automatically be triggered/called without arguments. If a logic handler is defined with just one parameter, the triggering value will be passed as a single argument, but the origin list will be omitted.

Implementation idea: This magic dropping of arguments should be implemented in the @shc.handler() decorator. We could examine the decorated function using inspect.signature() (or so) and later call the logic handler in an appropriate way, when triggered.

Fix LogList web widget

• Correct reconnect behaviour
• Fix styling
• Allow value and timestamp formatting

examples: Create web ui standalone example without external interfaces

Add monitoring script for evaluating the /monitoring endpoint in Nagios (or compatible monitoring systems)

The monitoring script should be a single-file Python 3 script, only requiring on the Python standard library, if possible. It shall comply with the Nagios Plugin API (text output and exit code): https://assets.nagios.com/downloads/nagioscore/docs/nagioscore/3/en/pluginapi.html
It shall provide command line parameters to configure the URL of the SHC server to be monitored and aspects of the server to be monitoried. Supported aspects should be:

• the overall SHC server, representing the overall status as calculated by the SHC server, based on the interface criticality
• only the SHC server itself (i.e. is the server reachable and returns valid JSON data) – similar to monitoring it with the check_http script
• only a single SHC interface (out of the provided JSON data)
  • should return the interface's metrics as Nagios perfdata

tasmota: Combine energy monitoring connectors to a single tuple-typed one

This will allow to process all individual measurments (current, voltage, power, powerfactor, …) as a single value update, e.g. to write it into a single InfluxDB row. For individual processing, the tuple can be split using an UpdateExchange with its fields.

knx: clear origin in local feedback when sending value to group address

In KNX GroupVariable objects we re-publish each value update which is written to the object, effectively creating a "local feedback". This way, other objects subscribed to the GroupVariable receive the value update as it came from the KNX bus. However, we keep the origin list intact, such that the value update is not written back to the orginating object (or any other object in the update's path).

This behaviour helps avoiding feedback loops, but it poses two problems:

1. It will be inconsistent with the MQTT interface, where we cannot prevent feedback (without origin reference), since the MQTT broker will return our sent messages without any hint about their origin.
2. When a concurrent, conflicting update comes from the KNX bus, it will result in a inconsistent state of KNX bus devices and our internal variable (and even devices connected via other interfaces). (see also #27)

Thus, we should probably clear the origin value for the local value feedback, i.e. make it look like the value came straigt from the KNX bus.

interfaces: Reconnect backoff timeout should be resetted upon successful connection

SupervisedClientInterface implements an exponential backoff timeout before reconnect, which is used by interfaces like shc_client, mqtt and others. However, the exponential backoff time is not resetted, once the connection can successfully be re-established, resulting in a overly long reconnection timeout when the next interruption occurs.

Add InfluxDB interface for data logging

Create file-based persistence store

The persistence store shall allow to store the current values of connected objects in a file or directory on the harddrive to persist them across restarts of the SHC application and the host computer. Therefore, it shall provide any number of named, readable + writable objects which write every received value update to the persistence file.

Optimally, changed values are stored immediately to persist them even with unexpected crashes/shutdowns. Still, the implementation should be somehow optimized to avoid parsing, updating serializing and rewriting the whole persistence store on each value update.

iPhone / Android interface

I've seen the Web-UI and was wondering whether there is any kind of iPhone or Android App to control the widget via smarthomeconnect?

BTW I love your approach of handling the house automation in pure python scripts. I have no need for local UIs etc. to maintain the system. I've been using openhab now for 10 years and I'm looking for a more adequate developer solution. smarthomeconnect looks very promising, great work 👍 But a smart phone interface is a must, especially for WAF 😄

web: Remove server-side template rendering

The web widgets use client-side JavaScript for dynamically changing the UI from received object state. Some of them already generate most of the HTML layout of the widget in JavaScript as well, e.g. by using a third-party JS library (like Iro or Chart.js). The other widgets use server-side template rendering with Jinja2 to generate the required HTML elements. However, these HTML templates are all quite simple, so we could reduce complexity by removing the template rendering and let all widgets' JavaScript code generate the HTML elements.

We can keep the base.htm template for rendering the overall web page with placeholders for each widget and an embedded JSON structure containing the specification of the individual widgets or move to a full single-page application, where the specification of the page layout and widget configurations is fetched as JSON from an API endpoint and all HTML rendering is done in client-side JavaScript.

Make Python package dependencies of interfaces optional

• Don't import interface modules automatically
• Use feature_requires in setup.py to make dependencies optional

web: Add support for serving and including favicon

Should be implemented as an option for shc.web.interface.WebServer, taking a file path of the icon file.

log: Refactor log widgets in to web module

Ramp generators: Interrupting a ramp with a new target value can result in an incomplete ramp

I experienced this with the RGBWHSVRamp: When fading from one white-ish color to another color and interrupting this ramp with black, the ramp seems to miss the last step sometimes, such that black is never reached completely.

Add CalDAV interface

The CalDAV interface should be able to read a specific calendar from a CalDAV calendar server. It should allow to create Readable + Subscribable bool-typed connector objects that tell whether currently any event, matching a specific pattern, is running according to the calendar. The connector-specific patterns could allow to consider only busy (not free) events, or only events having their summary or description matching a specific string pattern.

In addition, the interface could allow to create similar connectors that return the datetime timestamp of the next matching event.

Add web ui gauge widget

Maybe something like this:

Change Interface.get_status() to return a subscribable status connector

It should be possible to subscribe to each interface's status if the interface supports it. The status connector should be at least Readable[InterfaceStatus] and optionally Subscribable[InterfaceStatus]. The normal HTTP-based monitoring will read from these status connectors, whereas subscribing allows to react to interface status changes.

specify and implement log variable aggregation

shc_client: Conflicting updates from server and client can result in inconsistent states

We should introduce a republishing of server-to-client-updates at the client side, when we are waiting for the server to acknowledge a concurrent client-to-server update. Similar to the origin-resetting mechanism for (stateful) Subscribable objects within SHC, this should resolve the state inconsistency.

Add setup.py for packaging and installation

examples: Create example for KNX specifics

Including usage of TwoWayPipe and stateless connections

Add automatic reconnect to interfaces

Add error propagation to write() implementations

This would enable showing error messages in the user interface and report errors back to interfaces (such as the HTTP API) when value update propagation to one or more interfaces failed.

Requires reasonable _write() timeouts in interfaces.

web: Improve layout of HideRow

Add simulation mode to interfaces to allow offline tests

Add Redis interface

The Redis interface should allow creating Readable + Writable connectors for using Redis String-values associated to a specific key for persistence.

In addition, the interface could allow reading and writing to/from Redis streams, possibly allowing to use Redis as a datalog store or communication interface with other applications.

Build JS/CSS dependencies from source and minify custom CSS/JS code

This should be done automatically when packaging the Python code (ref. #5).

web: Add optional confirmation dialog for ui action widgets

Allow timers to be dynamically disabled/enabled

Maybe add an attribute enabled: bool and method enable(enabled: bool) -> None to all timers.
An additional wrapper class, inheriting from Readable[bool] and Writable[bool], could transform this into a connectable object.

It could also be made Reading[bool] to allow initialization from persistence.

Add @expression decorator to transform simple function into ExpressionHandler