nununo/pycandle2017

Candle (2017) in Python

pycandle2017's Issues

Review README.md

Including:

• Spell check it (noted in Raspberri which should be Raspberry).
• Improve development notes or create a separate document and link to it.
• Review the configuration section: inputs.agd.source is outdated.
• Rewrite / improve the introduction: the first section should describe the experience.
• Update the Running section to mention that a dbus-daemon process should also be running.

Review player manager _change_play_level() implementation

Level triggering mis-behaviour

With the change in 074fbbe (argh, a PR would definitely have been cleaner!) the following sequence is now possible:

• Input stable, with AGD output stable: level=0.
• Input raises such that AGD triggers level=1.
• While level 1 is playing:
  • Input goes down such that AGD triggers level=0.
  • Player manager forces a cross-fade from level=1 to level=0.

The last step does not make sense:

• Level 0 is "always there" so fading it feels silly.
• Going back to level 0 (rest) should always act naturally and not be forced ahead of time.

Possible fixes:

• Have AGD not trigger changes to level 0.
• Have player manager ignore changes to level 0.

Race condition between player manager and individual players.

Player manager general behaviour:

• On start(self):
  • Connects to dbus.
  • Spawns one player instance per level:
    • Spawns a new process.
    • Tracks/waits process start.
    • Tracks/waits process connection to dbus.
    • Instructs it to pause, via dbus.
  • Unpauses level 0 player.
• On level(self, new_level, comment=''):
  • It unpauses the respective level player.
• Then _player_ended(self, level):
  • Tracks player termination and spawns a new player instance associated to that level.

The race condition:

• Player manager started and level 0 is playing.
• level(new_level=X) called: player unpaused, level X video is played, fades out and player process terminates.
• Player manager detects that via _player_ended(level=X): spawns new level X player.
• level(new_level=X) called before just spawned level X player is ready.
• Player manager tries to unpause player X which fails because the process/dbus attachment isn't completed yet.

This would be within acceptable limits if a subsequent level X trigger would work. Unfortunately it does not, so this must be fixed.

Possible fix will need to not only track player termination but also player readiness.

Below is a player.* debug level log of the occurrence:

18:14:03.001381 [player.mngr#info] new_level=1 comment='network'
18:14:03.003097 [player.each.com.nunogodinho.vela2017-1-01#debug] asking player to play/pause
18:14:03.017497 [player.each.com.nunogodinho.vela2017-1-01#debug] asked player to play/pause
18:14:03.019412 [player.each.com.nunogodinho.vela2017-1-01#info] fade in starting
18:14:03.020830 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 0
18:14:03.058684 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 96.381676197052
18:14:03.099508 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 200.49487352371216
18:14:03.161713 [player.each.com.nunogodinho.vela2017-1-01#info] fade in completed
18:14:05.940658 [player.each.com.nunogodinho.vela2017-1-01#info] fade out starting
18:14:05.942076 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 255
18:14:05.969577 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 240.94610452651978
18:14:06.010768 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 219.9284291267395
18:14:06.040127 [player.mngr#info] new_level=1 comment='network'
18:14:06.052054 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 198.8754916191101
18:14:06.092972 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 178.01369905471802
18:14:06.133929 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 157.1235752105713
18:14:06.175132 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 136.11124992370605
18:14:06.216207 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 115.15485763549805
18:14:06.257382 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 94.16709423065186
18:14:06.298490 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 73.2254147529602
18:14:06.339461 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 52.30732440948486
18:14:06.380493 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 31.374034881591797
18:14:06.391978 [player.mngr#info] new_level=1 comment='network'
18:14:06.421738 [player.each.com.nunogodinho.vela2017-1-01#debug] alpha 10.33787727355957
18:14:06.484176 [player.each.com.nunogodinho.vela2017-1-01#info] fade out completed
18:14:06.567999 [player.dbus#debug] signal data=(':1.24', ':1.24', '')
18:14:06.574260 [player.dbus#debug] signal data=('com.nunogodinho.vela2017-1-01', ':1.23', '')
18:14:06.577890 [player.dbus#debug] signal data=(':1.23', ':1.23', '')
18:14:06.580275 [player.proc.com.nunogodinho.vela2017-1-01#debug] stdout: b'Video codec omx-h264 width 1280 height 720 profile 77 fps 25.000000\nSubtitle count: 0, state: off, index: 1, delay: 0\nV:PortSettingsChanged: [email protected] interlace:0 deinterlace:0 anaglyph:0 par:1.00 display:0 layer:1 alpha:0 aspectMode:0\nhave a nice day ;)\n'
18:14:06.585271 [player.proc.com.nunogodinho.vela2017-1-01#debug] player process ended; exit_code=0
18:14:06.587023 [player.mngr#info] player level=1 ended
18:14:06.588351 [player.mngr#info] creating player level=1
18:14:06.590380 [player.each.com.nunogodinho.vela2017-1-01#info] spawning player 'com.nunogodinho.vela2017-1-01'
18:14:06.591918 [player.dbus#info] tracking dbus player 'com.nunogodinho.vela2017-1-01'
18:14:06.593308 [player.mngr#debug] executable is '/usr/bin/omxplayer.bin'
18:14:06.595053 [player.mngr#debug] executable is '/usr/bin/omxplayer.bin'
18:14:06.603210 [player.proc.com.nunogodinho.vela2017-1-01#debug] player process started
18:14:06.607334 [player.dbus#info] waiting player 'com.nunogodinho.vela2017-1-01' start
18:14:06.712040 [player.mngr#info] new_level=1 comment='network'
18:14:06.713216 [player.each.com.nunogodinho.vela2017-1-01#debug] asking player to play/pause
18:14:06.714924 [twisted.internet.defer#critical] Unhandled error in Deferred:
18:14:06.715876 [twisted.internet.defer#critical] 
	Traceback (most recent call last):
	  File "/home/tiago.montes/pyVela2017.venv/lib/python3.4/site-packages/twisted/internet/defer.py", line 1532, in unwindGenerator
	    return _inlineCallbacks(None, gen, Deferred())
	  File "/home/tiago.montes/pyVela2017.venv/lib/python3.4/site-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
	    result = g.send(result)
	  File "/home/tiago.montes/pyVela2017/player/player.py", line 200, in play
	    yield self.play_pause()
	  File "/home/tiago.montes/pyVela2017.venv/lib/python3.4/site-packages/twisted/internet/defer.py", line 1532, in unwindGenerator
	    return _inlineCallbacks(None, gen, Deferred())
	--- <exception caught here> ---
	  File "/home/tiago.montes/pyVela2017.venv/lib/python3.4/site-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
	    result = g.send(result)
	  File "/home/tiago.montes/pyVela2017/player/player.py", line 190, in play_pause
	    yield self._dbus_player.callRemote(
	builtins.AttributeError: 'NoneType' object has no attribute 'callRemote'
	
18:14:06.791978 [player.dbus#debug] signal data=(':1.25', '', ':1.25')
18:14:06.794413 [player.dbus#debug] signal data=('com.nunogodinho.vela2017-1-01', '', ':1.25')
18:14:06.795625 [player.dbus#info] player 'com.nunogodinho.vela2017-1-01' started
18:14:06.796671 [player.each.com.nunogodinho.vela2017-1-01#debug] getting dbus player object
18:14:06.797847 [player.each.com.nunogodinho.vela2017-1-01#debug] got dbus player object
18:14:06.801525 [player.dbus#debug] signal data=(':1.26', '', ':1.26')
18:14:06.897648 [player.each.com.nunogodinho.vela2017-1-01#info] duration is 3.52s
18:14:06.899038 [player.each.com.nunogodinho.vela2017-1-01#debug] asking player to play/pause
18:14:06.917616 [player.each.com.nunogodinho.vela2017-1-01#debug] asked player to play/pause
18:14:06.983720 [player.mngr#info] new_level=1 comment='network'

Increase max log lines

Race condition if DBus process unexpectedly goes away

Following up on #30, #31, #32, and playing around with starting, stopping, process relationships and their unexpected exits, found yet another "ugly way" things can fail (as in, it will hang in there for ever).

Reproducing:

• Processes running: dbus-run-session, dbus-daemon, python and 4x omxplayer.bin.
• dbus-daemon suddenly goes away.
• Result:
  • No further candle2017.py to omxplayer.bin communication is possible:
  • Interaction is gone: no more video play/pause/fade control.
  • Stops may fail waiting forever on the omxplayer.bin processes to go away from DBus.
  • omxplayer.bin unexpected stops aren't properly handled: player manager respawns then, but waits forever for them to appear on DBus, thus never pauses them, etc.)

Without detecting DBus disconnections, the race condition solution is to implement a timeout on the last step above. Stop will work, albeit somewhat slowly, depending on the timeouts.

Detecting DBus disconnections, there are two possible solutions:

• DBus manager "marks" all names as gone, including ones that it will be asked about in the future; this fixes the race condition and ensures a fast stop.
• Player manager initiates an "emergency" top, avoiding _stop_via_dbus() and going directly to _stop_via_sigterm(); this avoids the race condition and ensures a fast stop.

DBus disconnections can be detected with:

from txdbus import client

dbus_conn = yield client.connect(...)
dbus_conn.notifyOnDisconnect(callable_receiving_two_args)

Calibrate sensors

Sensors are currently not calibrated. Particularly for level 1.

Implement runtime arduino input threshold adjustments

Notes:

• Might be useful.
• Needs thinking, UI wise.
• Should web changed thresholds be saved back to the settings.json file?

PS: By the way, should web changed log levels be saved back as well? (not as important, of course)

Review code for consistency (naming, etc)

Review UI logger names and their order.

Throw away event manager.

Convert it instead to a dependency on wires which is mostly the same code, somewhat improved and tested, which I just published on PyPI.

Web client: Responsive to iPad

USB Mouse / Gamepad / Joystick input support

I was driving back home when it struck me that these are commonly available devices useful to:

• Test and play around.
• Prove the current code design.

I investigated a little bit and I guess I have a solution.

Reconsider: trigger level 1 while playing level 2 clips?

While working on #93, I stumbled upon a behaviour I think might need to be reviewed:

• Recent changes to player_manager allow triggering a level 1 clip while a level 2 clip is playing.
• Given the nature of the clips we've been using, I think this leads to strange, unnatural responses.

Change level directly from the command line

Dynamic input wiring.

Implementing #62 will probably lead to some duplication regarding the raw audio input processing, where some variation of the arduino input's "aggregated derivative" will be implemented.

Implementing #62 also brings "challenges" like: how do we get this input's raw/derivative/whatever data plotted via the web?

The general idea about dynamic input wiring is:

• Separate the "aggregate derivative" into an input all by itself (will need some nicer name).
• Configuration in settings.json will tell how to arrange/wire things up, via new output/input entries defined in each input.

Here's an sketch:

inputs: {
    "arduino": {
        "device_file": ...,
        "baud_rate": ...,
        "output": "arduino-raw",
    },
    "agd": {
        "thresholds": [10, 20, 30],
        "input": "arduino-raw",
        "output": "arduino-agd"
    },
   "web": {
        ...
        "chart-1-input": "arduino-raw",
        "chart-2-input": "arduino-agd",
    }
}

With the way the current event manager works, implementing this is trivial.

PS: May need some additional thinking with regards to the web UI's "chart names" and "stuff"!

Let's integrate the first non-beta wires release.

Once the API is stable the change will be trivial:

• requirements.txt.
• Any wiring/unwiring syntax.

Standing by on tmontes/python-wires#24

Implement web log level control

Translate the project "subject" line to english.

See attached picture.

inputs.network.interface is configurable: update sample settings + docs.

Once #90 is merged, I'll quickly address this.

Update movie files

Re-design DBus session dependency

Current scenario:

• candle2017.py checks for the DBUS_SESSION_BUS_ADDRESS variable in the environment.
• If not present, it respawns itself under dbus-run-session.

Observation:

• The self-respawning is not very elegant.
• Due to intrinsic dbus-run-session limitations, this leads to somewhat inelegant clean stops.

Idea (after a little investigation):

• Let's forget about dbus-run-session and use dbus-damon directly instead.
• Maybe player/dbus_manager.py can handle it all:
  • Spawns (configurable) dbus-daemon with the --session and --print-address flags.
  • Tracks the bus address output and adds it to the environment.
  • Which will be ready for all the players after player manager launches them.
  • It will need a stop method to stop the spawned dbus-daemon.
• Player manager will need to stop DBus manager after cleaning up all processes.

Benefits:

• Much cleaner and easier to grasp design.
• Simpler system level integration.

Efforts:

• Not much.
• Need to confirm out-of-control process failure scenarios are still handled properly.

Integrate with Raspberry Pi start up and shutdown

Multiple triggers of same level need fixing

Integrate start/stop with system startup/shutdown.

(thankfully, #30 and #31 are in place)

Inputs should have configuration toggle switch.

Given the multitude of inputs, and the fact that some do not work concurrently, it would be nice to be able to disable them in the configuration instead of having to delete them.

Maybe an "enabled" key per each input entry is enough, taking values true or false.

Arduino input sensor: more efficient aggregated derivative calculation?

• Currently using a collections.deque() to track the most recent _INPUT_SIZE readings.

• On every new sensor reading:

  • The deque is updated.
  • The aggregated derivative is calculated from scratch, iterating the whole deque.

• This is computationally intensive and non-scalable (memory and processor-wise).

• Thoughts:

  • With the current algorithm I don't think we can send the deque away.
  • But maybe we can avoid iterating through it every time by tracking the "current aggregated derivative" (call it current_agd), incrementally updating it as in:
    • If reading is zero, set current_agd to 0 and clear the deque().
    • Otherwise, add reading to current_agd and, if len(deque) == _INPUT_SIZE, subtract its oldest value from current_agd; then append the reading to the deque().
  • This speeds up the calculation and makes it almost O(1) instead of the current O(n) (clearing the deque not O(1), but certainly much faster than our current approach; the other deque operations are O(1): len() and getting the oldest item).

PS: Not that we desperately need those CPU cycles, though. :)

Review: stop level N-1 clip after level N clip fade in.

Guarantees that when level N clip ends, we don't get a trailing piece of a level N-1 longer clip!

Add an audio based input

Scratching an itch, useful for:

• Testing with no arduino / bend sensor kit.
• Stressing current code architecture: is it clean or can/should it be improved.

Preliminary notes at https://github.com/tmontes/pyVela2017/blob/xplore-asla-input/inputs/alsaaudio/README.md

Improve input architecture

Status quo:

• Input manager calls "arbitrary" functions to initialize each input: somewhat messy.
• It also isn't ready to "finalize" inputs.
• More: if input initialization were to be asynchronous, input manager wouldn't know how to handle it.

Idea:

• Create a "base input class" with two methods "initialize" and "cleanup" (or some such).
• Re-factor each existing input into a derived class.
• Adjust input manager to account for that.

Once done:

• Call "input manager" cleanup on stop (this will be more important once I share a crazy idea I've been secretly exploring: having a microphone based input) ;)

TEASER: https://github.com/tmontes/pyVela2017/blob/xplore-asla-input/inputs/alsaaudio/README.md

Setup PI as an access point

One of the sensors is not working

Avoid the Web vs AGD config order dependency.

For correct web based AGD threshold visualisation and changes, the web input must be declared before the AGD input in settings.json.

This stems from the fact that the input instantiation order is directly sourced from the configuration.

Possible fixes:

• Hard code some kind of "instantiation order" in player manager - ugly, not sustainable.
• Add a "depends_on" attribute to the inputs - boring, error-prone, maybe silly.
• Have the web input "ask" AGD for the threshold levels, whenever they're needed (probably just the first time a web client connects) - this sounds clean and simple.

Move webserver into into inputs?

I'm in a "make things simpler and with less code" tonic...

Preamble

I have a WIP branch that simplifies the webserver by quite a bit: it mostly throws the http module away and handles everything in what has been the websocket module to date (but renamed to server on that branch).

The benefits are:

• There is a single HTTP endpoint serving both HTML and the websocket protocol.
• The javascript client can then, easily, connect back without needing a hardcoded a TCP port.
• This makes webserver configuration (port and interface) possible and trivial.
• Again, less code.

That branch is holding on #55 which imposed other types of simplifying changes to websocket.py.

Result

With such a simpler and now easily configurable web server, maybe we can move it to the inputs package and take advantage of its ability to "setup" inputs based on the settings.json configuration.

While the webserver is not strictly an "input", I think it fits nicely along with the other inputs.

Thoughts?

Improve cleanup resiliency on crazy exit conditions.

Reproducing steps:

• Launch candle2017.py.
• There should be the following processes:
  • dbus-run-session
    • dbus-daemon
    • python
      • omxplayer.bin [x4]
• Send a SIGTERM to the dbus-daemon process with kill SIGTERM <pid>.
• Confim the dbus-daemon is gone.
• Send a SIGTERM to python.
• It will try to stop the players via DBus, timing out on each (dbus-daemon gone).
• Assuming the processes are gone, it exits.
• The omxplayer.bin processes are left running, orphaned, however.

Improvement:

• In player/player.py handle the TimeOut condition and try to send a SIGTERM to the associated spawned process, tracking it if needed.

Plot arduino input level thresholds along with actual readings.

Review log messages and levels

Sending SIGTERM to process leaves omxplayer.bin processes running.

README says that stopping can be achieved by sending a SIGTERM to the Python process.

This procedure leaves the spawned omxplayer.bin processes running.
Fixing needs to be done via Twisted event handling.

Configure fade times