epics-containers/ibek

IOC Builder for EPICS and Kubernetes

ibek's Issues

Object id additions

At the moment we want to make a pmac.DlsPmacAsynMotor have a pmac.Geobrick as a PORT. But what if we wanted to allow a pmac.PowerBrick to be passed as well? iocbuilder allowed us to make a pmac.DeltaTau that pmac.Geobrick could subclass from, should be allow the same inheritance here?

Once we have decided that, we need to put enough information in the schema to allow the GUI to restrict selection of PORT to pmac.DeltaTau instances. This is not a supported concept in JSON schema, so we need to invent our own vocab like https://gregsdennis.github.io/json-everything/usage/vocabs-unique-keys.html

changes to ibek-defs

We need a way to provide versioning of ibek.support.yaml files and patch files. After discussion with @coretl we have the following approach to try out:

(examples below for the asyn support module)

• the repo ibek-defs will become 2 repos:

  • ibek-patch:

    • this contains patch script asyn.sh as before and any other files this needs to refer to
    • if breaking changes are made for e.g. asyn R4-44-2, then this file is copied to asyn<R4-44-2.sh
    • then the changes are made to the original asyn.sh
    • explicit copies into /ctools of the ibek-patch/asyn folder will be performed before each install (as before for ibek-defs)
    • modules.py install will automatically execute the patch file that is relevant based on the version of asyn in its 3rd argument
    • invocations of modules.py install will add to a file /repos/epiics/support/modules.json which will list the versions of modules
      installed by modules.py

  • ibek-yaml:

  • contains asyn.ibek.support.yaml

  • also will contain previous versions of the file using asyn<R4-44-2.ibek.support.yaml naming convention

  • after all support module install and compiles are complete we copy ibek-support into the container at repos/ibek-support

  • a new modules.py command add-yaml will soft link yaml files into each of the support modules at e.g. /repos/epics/support/asyn/ibek/asyn.ibek.support.yaml

  • the file modules.json is used to determine which version of the yaml file is linked.

  • if the ibek folder already exists in the support module then this step does nothing as that is assumed to have a support.yaml file that versions with the support module.

• both of thes repos are added to every ioc-XXX using git submodules.

ibek ioc autocompletion is picking up incorrect arguments

For example in this file https://github.com/epics-containers/ibek/blob/main/tests/samples/example-srrfioc08/SR-RF-IOC-08.ibek.ioc.yaml.

If you hit control space when editing an ipac.Hy8002 object you will see the args for ipac.Hy8002 but also the args for an asyn.AsynIP (at least that is what it looks like).

Remove helm chart generation

ibek will only run inside the container and charts will be made manually

Merge module.py into ibek

module.py provides some functions for building the generic-ioc containers.

both ibek and module.py now reside in epics-base container, see https://github.com/epics-containers/epics-base/blob/main/support/module.py

It would make sense to merge the functions of module.py into ibek as it is the 'in container' home of python things to support the building of support modules.

Not urgent.

Screen ideas

motor.screens.yaml would contain

screens:
- name: pmac.DlsPmacAsynMotor
  screen: motor.bob
  macros:
  - motor: $(P)$(M)

Consider a VS Code extension to allow allow schema location to be picked up from $schema

This lets us put the schema in ioc.yaml so we can edit it without having to configure a VS Code project and point at the schema for a particular IOC. We can use this $schema location to get the IOC image for the helm chart too.

There is an extension API that might help:
https://github.com/redhat-developer/vscode-yaml/wiki/Extension-API#register-contributor

Add an "entity_disabled" argument to each created dataclass

Defaults to False

Like type: https://github.com/epics-containers/ibek/blob/master/ibek/ioc.py#L102

If set then the render code should ignore this element

create substitution file

We have a requirement to build a substitution file for database generation instead of just a series of msi calls in a script.

This involves a few changes to the epics-containers ioc-template so I'd like to get this done soon.

@niamhdougan is it OK if I make this change or would you be interested/available?

Move system test to pytest

Currently the system test is a bash script.

Convert it to a pytest using:-

• ioca for channel access client
• https://docker-py.readthedocs.io/en/stable/services.html for launching the IOC container
• https://github.com/gilesknap/mciwb/blob/main/tests/conftest.py for inspiration

Enums

Would it be possible to provide Enum Types to constrain the values of args.

e.g. We could do with them in this ASYN def file

ibek/tests/samples/yaml/asyn.ibek.defs.yaml

Lines 37 to 50 in a82a480

	- type: int
	name: bits
	# TODO we should add enums to ibek's allowed data types
	# TODO this would require we define the enum values int 'type:' too
	description: Bits [8,7,6,5]
	default: null
	- type: str
	name: parity
	description: Parity [null,even,odd]
	default: null
	- type: int
	name: stop
	description: Stop Bits [1,2]
	default: null

Description field in definitions are not exposed in vscode popups

Database Generation

From a discussion with Tom.

We should provide the ability to generate a DB file using Jinja.

For example the PSC IOC SR-05I-PC-IOC-01 has the generated DB file pasted below.

We could do this by

• accumulating a list of PV names in global state using utils.set_var / get_var.
• adding a make_file entry to defs that creates a file and populates it with a multiline string. (this could be used to generate any file - not just DB files)
• using make_file to make a db from jinja for loops iterating over the PV names list above.

@coretl I'm still concerned that we will need to support epicsDB-builder for some IOCs. We could take the tack of using it instead of ibek. Maybe that is the only viable option as integrating ibek and builder sounds hard.

 For example the PSC IOC SR-05I-PC-IOC-01 has the following generated DB file:
# This file was automatically generated on Thu 18 Aug 2022 18:10:59 BST from
# source: /dls_sw/prod/R3.14.12.7/support/pscBuilder/2-2-3/python/build-psc.py
# 
# *** Please do not edit this file: edit the source file instead. ***
# 

record(calc, "SR05I-PC-TRIM-01:GRPSTATE")
{
    field(CALC, "0")
    field(INPA, "SR05I-PC-TRIM-01:L4DEVSTATE CP MS")
    field(INPB, "SR05I-PC-TRIM-02:L4DEVSTATE CP MS")
    field(INPC, "SR05I-PC-TRIM-03:L4DEVSTATE CP MS")
    field(INPD, "SR05I-PC-TRIM-04:L4DEVSTATE CP MS")
    field(INPE, "SR05I-PC-TRIM-05:L4DEVSTATE CP MS")
    field(INPF, "SR05I-PC-TRIM-06:L4DEVSTATE CP MS")
    field(INPG, "SR05I-PC-TRIM-07:L4DEVSTATE CP MS")
    field(INPH, "SR05I-PC-TRIM-08:L4DEVSTATE CP MS")
    field(INPI, "SR05I-PC-TRIM-09:L4DEVSTATE CP MS")
    field(INPJ, "SR05I-PC-TRIM-10:L4DEVSTATE CP MS")
    field(INPK, "SR05I-PC-TRIM-11:L4DEVSTATE CP MS")
    field(INPL, "SR05I-PC-TRIM-01:GRPSTATE2 CP MS")
    field(PINI, "YES")
    field(SCAN, "Passive")
}

record(calc, "SR05I-PC-TRIM-01:GRPSTATE2")
{
    field(CALC, "0")
    field(INPA, "SR05I-PC-TRIM-12:L4DEVSTATE CP MS")
    field(INPB, "SR05I-PC-TRIM-13:L4DEVSTATE CP MS")
    field(INPC, "SR05I-PC-TRIM-14:L4DEVSTATE CP MS")
    field(INPD, "SR05I-PC-VSTR-11:L4DEVSTATE CP MS")
    field(INPE, "SR05I-PC-VSTR-12:L4DEVSTATE CP MS")
    field(INPF, "SR05I-PC-HSTR-11:L4DEVSTATE CP MS")
    field(INPG, "SR05I-PC-HSTR-12:L4DEVSTATE CP MS")
    field(PINI, "YES")
    field(SCAN, "Passive")
}

Change from entity_disabled to entity_enabled

Tied to this issue

ibek ids names vary and this is confusing

If I want to refer to another object in a ibek.ioc.yaml file then I refer to it by its ID. However id might be 'name' or 'device' or any other arg on the object I'm referring to. There is no way to find out which arg to refer to without looking in the support.yaml.

It is true to say that 'name' and 'device' are the full set of options presently. Perhaps this could be resolved simply by listing the set of arg names that are used as ids (and hope that nothing needs to use two of these arg names??!!)

@AlexanderWells-diamond hit this issue working on his first IOC

Improve validation and error handling

Some improvements suggested in conversation with @coretl

• in getattrs make a nice error when a non-existent attribute is requested (I am type X with attrs ... but you have asked for Attr Y)
• also remove need to say object.name instead render object as its id
• Provide a validation capability for defs e.g.

defs:
  - name: Psc
    args:
      - name: carrier
        type: object
        description: IPAC carrier name

      - name: interrupt_vector
        type: object
        description: Interrupt Vector reserved with epics.InterruptVectorVME, count=3

    validate:
      - assert: interrupt_vector.count == 3
        message: Interrupt_vector must be reserved with epics.InterruptVectorVME, count=3

Poor connection between function arguments and ibek entity arguments

Consider this function definition in Hy8401ip.ibek.support.yaml

    script:
      - type: function
        name: Hy8401ipConfigure
        header: |
          #   IpSlot 0=A 1=B 2=C 3=D
          #   ClockRate  0=1Hz  1=2Hz  2=5Hz  3=10Hz 4=20Hz 5=50Hz 6=100Hz7=200Hz 8=500Hz 9=1kHz 10=2kHz11=5kHz 12=10kHz 13=20kHz 14=50kHz 15=100kHz
        args:
          CardId: "{{ card_id }}"
          IPACid: $({{ carrier.name }})
          IpSiteNumber: "{{ ip_site_number }}"
          InterruptVector: $({{ vector.name }})
          InterruptEnable: "{{ int_enable | int }}"
          AiType: 0
          ExternalClock: "{{ external_clock | int }}"
          ClockRate: "{{ clock_rate }}"
          Inhibit: "{{ inhibit | int }}"
          SampleCount: 1
          SampleSpacing: 1
          SampleSize: "{{ sample_size }}"

With example resulting entry in the st.cmd

# Hy8401ipConfigure CardId IPACid IpSiteNumber InterruptVector InterruptEnable AiType ExternalClock ClockRate Inhibit SampleCount SampleSpacing SampleSize
#   IpSlot 0=A 1=B 2=C 3=D
#   ClockRate  0=1Hz  1=2Hz  2=5Hz  3=10Hz 4=20Hz 5=50Hz 6=100Hz7=200Hz 8=500Hz 9=1kHz 10=2kHz11=5kHz 12=10kHz 13=20kHz 14=50kHz 15=100kHz
Hy8401ipConfigure 40 $(IPAC4) 0 $(Vec0) 0 0 0 15 0 1 1 0

In script->function->args we have made up UpperCamelCase names for the arguments that don't directly relate to definition arguments in the support.yaml file. There is no description anywhere of the meaning of AiType SampleCount SampleSpacing. IPACid and InterruptVector are also made up names that have no direct description.

Our first user @AlexanderWells-diamond certainly found this confusing. Not sure there is much we can do - but perhaps at least mandate function args names are snake_cases and match their definition arg where appropriate (but this feels repetative).

Move sphinxext into a sphinx-apischema project

There is some code from scanspec to move there too

Convert from apischema to pydantic

After dls-controls/scanspec#52