import file in pyscript directory

[custom-components]

import file in pyscript directory

[craigbarratt]

The sys.path needs to be set to the directories you want to import from (in addition to setting allow_all_imports).

First, you shouldn't put modules or packages in the <config>/pyscript directory, since pyscript will import every file there and treat it as a script. You should create a new folder, eg <config>/pyscript_modules. Add this near the top of your pyscript file:

import sys

sys.path.append("config/pyscript_modules")

You can then put your native Python modules or packages in config/pyscript_modules and your pyscript scripts can import them from there.

I recently added some (incomplete) documentation here. I need to add some examples.

[dlashua]

This ALMOST did what I wanted. The issue, now, is that none of the pyscript features are available in that module.

The goal is to make pyscripts easily sharable among users. If I offer a file to download to provide some functionality/automation/etc and the user has to configure that thing INSIDE of that same file, then any configuration they've made is overwritten when I provide an updated file.

So I want the guts of the automation to be in one file, with the configuration of that automation in another file. Ideally, within the configuration in YAML/ConfigUI, perhaps in the pyscript section of configuration.yaml. But, that last piece is a "not quite yet" goal.

I'm imagining code something like this:

# followme.py
@app
def followme(config={}):
    leader = config.get('leader')
    if leader is None:
        log.error('no leader')
        return

    followers = config.get('followers', [])
    
    @state_trigger("True or {}".format(leader))
    def follow_me_trigger():
        value = state.get(leader)
        if value == 'on':
            for item in followers:
                homeassistant.turn_on(entity_id=item)
        else:
            for item in followers:
                homeassistant.turn_off(entity_id=item)

    return follow_me_trigger

# usermanagedfile.py
followme1 = app.followme({
    'leader': 'input_boolean.test_1',
    'followers': ['input_boolean.test_2']
})

# maybe, eventually, in configuration.yaml
pyscript:
  apps:
    test_1:
      app: followme
      config:
          leader: input_boolean.test_1
          followers:
              - input_boolean.test_2

I'm thinking, for right now, maybe I can get away with making "followme" a "service". And "configuration" can be done with an automation that runs on homeassistant start and passes leader and followers to the service, which then adds the 'state_trigger" in a way that pyscript can see it.

[dlashua]

I just tested this. It's ugly, but initial tests show that it works.

in pyscripts:

# followme.py
registered_triggers = []

@service
def follow_me(**config):
    leader = config.get('leader')
    if leader is None:
        log.error('leader is required in config')

    followers = config.get('followers', [])

    log.error('creating trigger for {}'.format(leader))

    @state_trigger("True or {}".format(leader))
    def follow_me_trigger():
        log.error('follow me triggered for {}'.format(leader))
        value = state.get(leader)
        if value == 'on':
            for item in followers:
                homeassistant.turn_on(entity_id=item)
        else:
            for item in followers:
                homeassistant.turn_off(entity_id=item)

    registered_triggers.append(follow_me_trigger)

in configuration.yaml automations

- alias: follow_me test_1
  trigger:
    - platform: event
      event_type: service_registered
      event_data:
        domain: pyscript
        service: follow_me
  action:
    - service: pyscript.follow_me
      data:
        leader: input_boolean.test_1
        followers:
          - input_boolean.test_2

[craigbarratt]

Good suggestions and ideas.

Here are some proposals:

• Support an optional subdirectory <config>/pyscript/modules that contains pyscript modules that can be imported in the usual manner. These are not autoloaded.
• Support an optional subdirectory <config>/pyscript/apps that contains pyscript scripts that are autoloaded.
• Bind pyscript's yaml config settings to a variable like pyscript.config. Apps can use a @startup trigger to extract their configuration settings and implement any behavior based on those settings. However, I'm not sure if HASS supports dynamic config settings (ie, ones that aren't known until during startup). Maybe each pyscript app will need to specify its own config schema, or perhaps HASS allows some sort of wildcarding (ie, allow any yaml below pyscript -> apps)?

[craigbarratt]

Ok, it's possible to add arbitrary config settings that pyscript could just pass along to the apps scripts. Those scripts would be responsible for validating the structure and types of those settings, eg using voluptuous.

[dlashua]

Optional subdirectories for modules and apps sounds wonderful. This would provide a lot of flexibility, I think.

And leaving config validation up to the apps seems perfectly fine. And it would be extra great if the YAML in configuration.yaml is reloaded when PyScript is reloaded, therefore allowing app configuration changes without having to restart Home Assistant.

[craigbarratt]

I just pushed 2aaac02 which implements the suggestions above:

• Supports an optional subdirectory <config>/pyscript/modules that contains pyscript modules that can be imported in the usual manner. These are not autoloaded.
• Supports an optional subdirectory <config>/pyscript/apps that contains pyscript scripts that are autoloaded (not tested yet).
• Bind pyscript's yaml config settings to a variable pyscript.config. Apps can use a @time_trigger("startup") trigger to extract their configuration settings and implement any behavior based on those settings. This variable updates to the latest yaml config when you call the pyscript.reload service. There is no structure enforced on the config settings, but a recommendation is to use an entry apps, with the name of the application below that.

[dlashua]

This seems to work wonderfully with one issue:

configuration:

pyscript:
  allow_all_imports: true
  apps:
    - app: follow_me
      leader: input_boolean.test_1
      followers:
        - input_boolean.test_3
    - app: follow_me
      leader: input_boolean.test_2
      followers:
        - input_boolean.test_4

follow_me.py

registered_triggers = []

def follow_me(**config):
    leader = config.get('leader')
    if leader is None:
        log.error('leader is required in config')
        return

    followers = config.get('followers', [])

    log.error('creating trigger for {}'.format(leader))

    @state_trigger("True or {}".format(leader))
    def follow_me_trigger():
        log.error('follow me triggered for {}'.format(leader))
        value = state.get(leader)
        if value == 'on':
            for item in followers:
                homeassistant.turn_on(entity_id=item)
        else:
            for item in followers:
                homeassistant.turn_off(entity_id=item)

    registered_triggers.append(follow_me_trigger)

@time_trigger('startup')
def follow_me_config():
    log.error(pyscript.config)
    if "apps" not in pyscript.config:
        return

    for app in pyscript.config['apps']:
        if "app" not in app:
            continue
        if app['app'] != 'follow_me':
            continue

        log.error('loading follow_me app with config {}'.format(app))
        follow_me(**app)

The issue is, everything works great when Home Assistant starts. But, something about the way I'm using registered_triggers means that, when I reload pyscript, all the existing triggers are removed (as expected), and I see the log messages of "creating trigger for X". But, the triggers never fire. Once pyscript is reloaded, these "automations" don't work any more.

[dlashua]

I tried it this way too:

def follow_me(**config):
    leader = config.get('leader')
    if leader is None:
        log.error('leader is required in config')

    followers = config.get('followers', [])

    log.error('creating trigger for {}'.format(leader))

    @state_trigger("True or {}".format(leader))
    def follow_me_trigger():
        log.error('follow me triggered for {}'.format(leader))
        value = state.get(leader)
        if value == 'on':
            for item in followers:
                homeassistant.turn_on(entity_id=item)
        else:
            for item in followers:
                homeassistant.turn_off(entity_id=item)

    trigger_reference = 'trigger_{}'.format(id(follow_me_trigger))
    log.error(trigger_reference)
    globals()[trigger_reference] = follow_me_trigger
    log.error(globals())

I compared the output of globals() from Home Assistant start to the output on pyscript.reload. They appear to be the same.

Sadly, while I grok python pretty well, the code for pyscript is very complex. I tried to find the issue myself but failed.

[dlashua]

Ah Ha! As I was writing that last comment, I had a thought. What if the reason it doesn't work on reload is because the functions don't exist in the global scope when the file finishes evaling (since the function doesn't run until the "startup" trigger.

So I changed the code to this and it works perfectly:

def follow_me(**config):
    leader = config.get('leader')
    if leader is None:
        log.error('leader is required in config')

    followers = config.get('followers', [])

    log.error('creating trigger for {}'.format(leader))

    @state_trigger("True or {}".format(leader))
    def follow_me_trigger():
        log.error('follow me triggered for {}'.format(leader))
        value = state.get(leader)
        if value == 'on':
            for item in followers:
                homeassistant.turn_on(entity_id=item)
        else:
            for item in followers:
                homeassistant.turn_off(entity_id=item)

    trigger_reference = 'trigger_{}'.format(id(follow_me_trigger))
    log.error(trigger_reference)
    globals()[trigger_reference] = follow_me_trigger
    log.error(globals())


log.error(pyscript.config)
if "apps" in pyscript.config:
    for app in pyscript.config['apps']:
        if "app" not in app:
            continue
        if app['app'] != 'follow_me':
            continue

        log.error('loading follow_me app with config {}'.format(app))
        follow_me(**app)

[dlashua]

I converted the code back to using registered_triggers.append() and it still works just fine.

This does exactly what I need, though I think it would be cleaner if there was a method like:

pyscript.register_function(func_ref_here)

This would give a direct, readable, supported way of expressing exactly what the code is trying to do.

[craigbarratt]

The first example you included doesn't work on reload because of a bug. Sorry about that. The reload code fails to set the global context back to auto start, so trigger closures created after reload don't get started, as you noted. Your final example works because all the trigger functions get created before the script finishes, and every pending trigger is started at the end of the reload, which is why that version avoids the bug. I just pushed 88496cf to fix this, so your earlier startup trigger version should work now too.

[craigbarratt]

On the config, I was wondering whether making the app name the top level, and multiple instances could be listed under that, would be clearer. We'll need some convention as people develop different apps:

pyscript:
  allow_all_imports: true
  apps:
    - follow_me:
      - leader: input_boolean.test_1
        followers:
         - input_boolean.test_3
      - leader: input_boolean.test_2
        followers:
         - input_boolean.test_4

That would make the config code more compact, eg:

for app in pyscript.config.get("apps", {}).get("follow_me", []):
    follow_me(**app)

or with some error checking:

for app in pyscript.config.get("apps", {}).get("follow_me", []):
    if "leader" not in app:
        log.error(f"config yaml missing 'leader' setting for follow_me in {app}")
    elif "followers" not in app or not isinstance(app["followers"], list):
        log.error(f"config yaml expected 'followers' setting as list for follow_me in {app}")
    else:
        follow_me(app["leader"], app["followers"])

and then follow_me could have explicit arguments and skip its error checking.

[craigbarratt]

I added some documentation for these new features.

[dlashua]

I agree that having a top level app name looks nicer and is easier to parse when the YAML config is all in one place. However I tend to like to break my YAML up by room/area. So, in reality, my config would look more like this:

# configuration.yaml
pyscript:
  allow_all_imports: true
  apps: !include_dir_merge_list app_config

# app_config/office.yaml
- app: follow_me
  leader: input_boolean.test_1
  followers:
    - input_boolean.test_4
    
- app: follow_me
  leader: input_boolean.test_2
  followers:
    - input_boolean.test_3

# app_config/hvac.yaml
- app: calc_conditional_avg
  entity_id: sensor.avg_upstairs_occupied_temperature
  unit_of_measurement: "°F"
  friendly_name: Avg Upstairs Occupied Temperature
  precision: 1
  conditions:
    - condition: binary_sensor.master_occupied
      entity: sensor.master_temperature
    - condition: binary_sensor.celeste_occupied
      entity: sensor.celeste_temperature
    - condition: binary_sensor.jackson_occupied
      entity: sensor.jackson_temperature
    - condition: binary_sensor.james_occupied
      entity: sensor.james_temperature
    - condition: binary_sensor.dance_occupied
      entity: sensor.dance_temperature
    - condition: binary_sensor.bathkids_occupied
      entity: sensor.bathkids_temperature
    - condition: binary_sensor.bathmaster_occupied
      entity: sensor.bathmaster_temperature

- app: calc_conditional_avg
  entity_id: sensor.avg_downstairs_occupied_temperature
  unit_of_measurement: "°F"
  friendly_name: Avg Downstairs Occupied Temperature
  precision: 1
  conditions:
    - condition: binary_sensor.office_occupied
      entity: sensor.office_temperature
    - condition: binary_sensor.dining_occupied
      entity: sensor.dining_temperature
    - condition: binary_sensor.kitchen_occupied
      entity: sensor.kitchen_temperature
    - condition: binary_sensor.living_occupied
      entity: sensor.living_temperature
    - condition: binary_sensor.bathguest_occupied
      entity: sensor.bathguest_temperature

This style of configuration separation doesn't work well when you need entries under the same key in separate files. That being said, if you prefer the app name keyed convention, some separation can still happen. It would just break down to one file per app (and then multiple files from there, if preferred) as opposed to whatever arbitrary breakdown the user might prefer (in my case, by room/purpose).

I do like the idea of doing all of the config error checking in the yaml loading portion of the code