virtuos/opencast-camera-control

Configure the Cameras for Opencast Recordings

Opencast Camera Control

Control PTZ camera to move to certain presets when starting a scheduled recording.

This tool communicates with Opencast to get scheduled events for a list of configured capture agents. When an event starts, cameras configured to be part of the agent's setup are then automatically turned to an active position specified by a configurable camera preset. When the event ends, the camera is turned back to a neutral position.

This allows you, for example, to automatically turn cameras to the wall when no recordings are active.

opencast-camera-control.mp4

(Camera moves when a one minute scheduled recording starts and returns to point at the wall at the end)

Installation

PIP

We release Opencast Camera Control via the Python Package Index. This means you can easily install the tool via pip:

❯ pip install opencast-camera-control
❯ opencast-camera-control

RPM Repository

On RHEL 9 like distributions (CentOS Stream, Rocky, Alma, …) you can use the RPM repository to install Opencast Camera Control. Install the repository by adding a file /etc/yum.repos.d/opencast-camera-control.repo:

[opencast-camera-control]
name = Opencast camera control el$releasever repository
baseurl  = https://raw.githubusercontent.com/virtUOS/opencast-camera-control/rpm-el$releasever/
enabled  = 1
gpgcheck = 1
gpgkey = https://raw.githubusercontent.com/virtUOS/opencast-camera-control/rpm-el$releasever/opencast-camera-control.key

Then activate the EPEL repository and install opencast-camera-control:

❯ dnf install -y epel-release
❯ dnf install -y opencast-camera-control

The RPM packages provide a Systemd unit to run the tool as service:

❯ systemctl start opencast-camera-control.service
❯ systemctl enable opencast-camera-control.service

Container

We also provide a container image. A simple docker compose example would look like this

---
services:
  camera-control:
    image: ghcr.io/virtuos/opencast-camera-control:0.3.0
    container_name: opencast-camera-control
    ports:
      - '8000:8000'
    volumes:
      - './your_config.yml:/etc/camera-control.yml'

Configuration

Take a look at the camera-control.yml configuration file. All available options are documented in there.

The tool uses the first configuration it can find. It looks for files in the following order:

• ./camera-control.yml
• ~/camera-control.yml
• /etc/camera-control.yml

You can provide custom configuration files using the --config option:

❯ opencast-camera-control --config custom-config.yml

Supported Cameras

The tool supports PTZ cameras from Panasonic and Sony. The following cameras have been confirmed to be working:

• Panasonic AW-UE50
• Panasonic AW-UE70
• Sony SRG-300SE
• Sony SRG-X120

Opencast User

To improve security, you can limit the access rights for the Opencast user by creating a user which has only the role ROLE_CAPTURE_AGENT assigned.

Metrics

You can enable an OpenMetrics endpoint in the configuration. This can give you insight into current camera positions and will report occurring errors.

The resulting metrics data will look like this:

# HELP request_errors_total Number of errors related to HTTP requests
# TYPE request_errors_total counter
request_errors_total{ressource="http://camera-3-panasonic.example.com",type="ConnectionError"} 77.0
request_errors_total{ressource="http://camera-panasonic.example.com",type="ReadTimeout"} 12.0
# HELP request_errors_created Number of errors related to HTTP requests
# TYPE request_errors_created gauge
request_errors_created{ressource="http://camera-3-panasonic.example.com",type="ConnectionError"} 1.707571882114209e+09
request_errors_created{ressource="http://camera-panasonic.example.com",type="ReadTimeout"} 1.7075718871156712e+09
# HELP agent_calendar_update_total Nuber of calendar update
# TYPE agent_calendar_update_total gauge
agent_calendar_update_total{agent="test_agent"} 4.0
# HELP agent_calendar_update_time Time of the last calendar update
# TYPE agent_calendar_update_time gauge
agent_calendar_update_time{agent="test_agent"} 1.707571943100096e+09
# HELP camera_position Last position (preset number) a camera moved to
# TYPE camera_position gauge
camera_position{camera="http://camera-2-panasonic.example.com"} 10.0

opencast-camera-control's Issues

Create RPM repository

…to make the installation easier.

Metric for expected camera position

…in addition to the metric for the actual position.

Reminder: remove python setup patch in RHEL10

This patch is only needed in RHEL9: `

opencast-camera-control/.github/rpm/python-confygure.spec

Line 35 in b3728e8

# This is a workaround for the Python toolchain in EL9 not supporting the new pyproject.toml

We should remove this, once we do not use RHEL9 any more.

Document methods and classes

We should make sure to have proper doc strings everywhere

Add video of how this works

…to show users what to expect.

Create executable

Ensure pip creates an executable when installing the tool.

Basically just copy and adjust pyproject.toml and setup.py from https://github.com/virtUOS/deterrers-cli

Update schedule regularly

The schedule of recordings may change on short notice. Only updating it every few days is not enough. What if I plan a new recording in one hour?

https://github.com/virtUOS/cameraConfiguration/blob/f4812dea08ee510bf9cd4885812e52fbb923d984/main.py#L150-L151

Add open metrics endpoint

Mostly to indicate errors but we may also add some other stats.

Set camera defaults

It would be helpful to set some default values for cameras instead of setting the same value over and over again.
This could look like this:

camera:
  __default:
    - type: panasonic
      user: camerauser
      password: secret

Complain about missing configuration file

Ensure that we complain about missing configuration files instead of stumbling over missing configuration keys.
Right now, this is what a missing configuration file causes:

# opencast-camera-control
Traceback (most recent call last):
  File "/usr/local/bin/opencast-camera-control", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/site-packages/occameracontrol/__main__.py", line 83, in main
    for agent_id, agent_cameras in config_rt(dict, 'camera').items():
  File "/usr/local/lib/python3.9/site-packages/confygure/__init__.py", line 119, in config_rt
    raise KeyError(f'Missing configuration key {args}')
KeyError: "Missing configuration key ('camera',)"

Use flake8 for static code analysis

Several places in the code are potentially problematic.

For example, here the indentation suddenly changes. Since Python uses indentation for structure this can easily lead the weird side effects:
https://github.com/virtUOS/cameraConfiguration/blob/5b83de76ea2af24cb1090ec187faddfaa8a0a43c/main.py#L43-L47

In general, it is good practice to follow Python's official style guide (PEP8). A good tool to check for this is flake8.
Run it with:

❯ flake8 main.py

Pass on camera object/dict

Instead of handling all of the camera properties as separate arguments like this:
https://github.com/virtUOS/cameraConfiguration/blob/cba428b95d89d05b38fd520794952993446634c6/main.py#L111
…we should just hand over a camera object/dict containing all of the relevant information. This could just be the camera dict loaded from the configuration.

Simplify range comparison

Python allows you to compare several values at one, making it unnecessary to split range comparisons.
You can turn something like this:
https://github.com/virtUOS/cameraConfiguration/blob/f4812dea08ee510bf9cd4885812e52fbb923d984/main.py#L24
into something like this:

if 0 <= preset < 101: 

Seperrate control and calendar updates

A calendar update could potentially take some time (network issues, …) which may cause the control to be delayed. We should move the calendar updates for all cameras to a separate thread, so we can sequentially update all calendars.

What happens if one deletes an active recording in Opencast

This may or may not be an edge-case where we do not yet handle changing camera positions correctly:

• Schedule an event in Opencast
• Event is being picked up by camera control
• Event starts recording and camera turns to active position
• Someone deletes the scheduled recording in Opencast
• Camera control picks up new schedule
• Recording ends

Does the camera ever turn back?
We should check this.

Time formatting in logs

Sometimes, seconds arent really helpful:

Mar 26 15:00:53 ...[6091]: INFO:occameracontrol.camera:[...] Next event `...` starts in 590346 seconds

Having this formatted as hh:mm:ss would be more helpful.

Document supported camera types

Add a list of cameras we have tested

Publish on PyPi

• Turn this into a Python package
• Publish on pypi.org

Verify agents exist

Right now, a typo in a capture agent name can mean that the camera control never gets any events. It only ever gets an empty result. This can make it hard to spot errors in capture agent names.

Maybe we could check the configuration against the registered agents in Opencast and log an error or maybe even fail if a configured agent does not exist.

We can get the agents from Opencast like this:

❯ curl -s -u admin https://develop.opencast.org/capture-admin/agents.json | jq
Enter host password for user 'admin':
{
  "agents": {
    "agent": [
      {
        "name": "pyca@ansible-pyca",
        "state": "idle",
        "url": "http://localhost:5000",
        "time-since-last-update": 1606,
        "capabilities": ""
      },
      {
        "name": "pyca@ansible-pyca-debian",
        "state": "idle",
        "url": "http://localhost:5000",
        "time-since-last-update": 22591,
        "capabilities": ""
      },
      {
        "name": "pyca@jani",
        "state": "offline",
        "url": "http://localhost:5000",
        "time-since-last-update": 53842561,
        "capabilities": ""
      }
    ]
  }
}

Update position on a regular basis

We should update the position on a regular basis in case something happens to the camera (power loss, …). The update interval should be configurable, maybe with every 60s as a good default?

Create container

We should create a containerized version of this

Set HTTP timeouts

We need to make sure that we set proper timeouts for all HTTP requests to ensure our control loops do not hang.

Handle not only HTTP errors

We should make sure that the loops always restart or the whole service fails. One thread failing is bad.

Automate configuration tests

It would be great if we could add a test run to verify that all cameras and agents are configured correctly.
Running the test should:

• For each configured capture agent, schedule a recording in 5 min with a length of 60 seconds
  • We may need to ask users for an account with more permissions to run this test
• Check that all cameras are in neutral position
• After the events begin, check that all cameras are in active position
• After the events stop, check that all cameras are in neutral position

No stack traces for read timeouts

Similar to #34, we don't need full stack traces for read timeouts.

We want something like

 ERROR:occameracontrol.metrics:Failed to communicate with camera 'arec-15-421' @ 'http://camera1-15-421.example.com': requests.exceptions.ReadTimeout: HTTPConnectionPool(host='camera1-15-421.example.com', port=80): Read timed out. (read timeout=5)

Instead of getting

 ERROR:occameracontrol.metrics:Failed to communicate with camera 'arec-15-421' @ 'http://camera1-15-421.example.com'
 Traceback (most recent call last):
   File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 446, in _make_request
     six.raise_from(e, None)
   File "<string>", line 3, in raise_from
   File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 441, in _make_request
     httplib_response = conn.getresponse()
   File "/usr/lib64/python3.9/http/client.py", line 1377, in getresponse
     response.begin()
   File "/usr/lib64/python3.9/http/client.py", line 320, in begin
     version, status, reason = self._read_status()
   File "/usr/lib64/python3.9/http/client.py", line 281, in _read_status
     line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
   File "/usr/lib64/python3.9/socket.py", line 704, in readinto
     return self._sock.recv_into(b)
 socket.timeout: timed out
 During handling of the above exception, another exception occurred:
 Traceback (most recent call last):
   File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 439, in send
     resp = conn.urlopen(
   File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 756, in urlopen
     retries = retries.increment(
   File "/usr/lib/python3.9/site-packages/urllib3/util/retry.py", line 532, in increment
     raise six.reraise(type(error), error, _stacktrace)
   File "/usr/lib/python3.9/site-packages/urllib3/packages/six.py", line 709, in reraise
     raise value
   File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 700, in urlopen
     httplib_response = self._make_request(
   File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 448, in _make_request
     self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
   File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 337, in _raise_timeout
     raise ReadTimeoutError(
 urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='camera1-15-421.example.com', port=80): Read timed out. (read timeout=5)
 During handling of the above exception, another exception occurred:
 Traceback (most recent call last):
   File "/usr/lib/python3.9/site-packages/occameracontrol/__main__.py", line 60, in control_camera
     camera.update_position()
   File "/usr/lib/python3.9/site-packages/occameracontrol/camera.py", line 147, in update_position
     self.move_to_preset(self.preset_inactive)
   File "/usr/lib/python3.9/site-packages/occameracontrol/camera.py", line 86, in move_to_preset
     response = requests.get(url, auth=auth, params=params, timeout=5)
   File "/usr/lib/python3.9/site-packages/requests/api.py", line 76, in get
     return request('get', url, params=params, **kwargs)
   File "/usr/lib/python3.9/site-packages/requests/api.py", line 61, in request
     return session.request(method=method, url=url, **kwargs)
   File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 544, in request
     resp = self.send(prep, **send_kwargs)
   File "/usr/lib/python3.9/site-packages/requests/sessions.py", line 657, in send
     r = adapter.send(request, **kwargs)
   File "/usr/lib/python3.9/site-packages/requests/adapters.py", line 529, in send
     raise ReadTimeout(e, request=request)
 requests.exceptions.ReadTimeout: HTTPConnectionPool(host='camera1-15-421.example.com', port=80): Read timed out. (read timeout=5)

Error resistance

The script will turn the camera only in the one second when the recording starts:
https://github.com/virtUOS/cameraConfiguration/blob/f4812dea08ee510bf9cd4885812e52fbb923d984/main.py#L130
and the one second the recording stops:
https://github.com/virtUOS/cameraConfiguration/blob/f4812dea08ee510bf9cd4885812e52fbb923d984/main.py#L136

If anything happens during this one second, the camera control fails, and we will either lose the recording or have the camera still pointed at the room at the end. The tool should make sure that the command is executed. Being somewhat late is still far better than not executing the command at all.

Add a camera option to verify a recording has started

This may need some discussion, but it may be a good idea to verify that a recording has actually begun before turning the camera.

Log only HTTP errors? Or log stack traces?

Right now, we log stack traces of HTTP errors. Do we want to log just the errors instead?

Allow users to configure HTTP authentication types

Right now, we assume HTTP Basic auth for Panasonic cameras and HTTP Digest auth for Sony cameras. But the cameras actually allow users to configure the authentication type. Therefor, we sould allow that as a camera configuration option as well.

Use a logger

Instead of using complicated print statements and variables to control which one to show, use Python's logging standard library. It supports variable replacement and log levels. That way, you can just turn something like this:
https://github.com/virtUOS/cameraConfiguration/blob/f4812dea08ee510bf9cd4885812e52fbb923d984/main.py#L32-L33
into

logger.debug(...)

See, for example, the logger usage in the Opencast Podcast project.

Make profile configurable

Instead of hard-coding the profile, we should allow users to specify the active and neutral profiles in the configuration.
This could look somewhat like this:

    - url: http://camera.example.com
      type: panasonic
      user: admin
      password: admin
      profile:
        active: 1
        inactive: 10

Use float/soconds for timestand

If we just use floats, we don't need to convert them to milliseconds/int every time.

License the tool

The tool needs a license.
I suggest GPL?

Camera control restart causes camera position reset

If the camera control tool is restarted, it turns all cameras to an inactive position. This is unfortunate, if there is an active recording. Once we get the schedule, the camera is then again turned to the active position.

This is how a restart looks like:

camera-control-restart-ps.mp4

The problem is likely that the first position change is issued before we got the schedule. We may want to wait for the schedule before starting to set camera positions.