jcarrano/rtfi

Audio visualization & analysis using the RTFI

Resonator Time-Frequency Image

by Juan I Carrano

Here, watch a video of it in action!

Introduction

This is an implementation of the RTFI, as described in [Zhou]. It consists of a bank of band-pass filters, whith the same Q-factors and exponentially spaced center frequencies. The filter are single complex pole resonators, that is they are not symmetric in the Z-plane. They take a real input, but have a complex state and a complex output. The advantage of this is that the envelope of the filtered signal can be obtained by calculating the norm of the output samples. In order to make the calculation of 900 filters feasible in real time, the multirate approach suggested by Zhou is employed.

History

0.0.3 (Jun '19)

• Add Makefile
• Fix compilation issues with C99
• Reorganize repo

0.0.2 (Nov '18)

Revive project. Assign version number and refomat README.

March '13

Repackaged to include extra_libraries.

Mid 2012

Initial version. Derived from a course project in Signals and Systems (the original code was MATLAB and was not real time.

Usage

The only supported sample frequencies are 44100Hz, 48000Hz and 96000Hz. This is because filter coefficients are only calculated for those frequencies.

To start the program:

# (if necessary) Launch the jack daemon
$ jackd -r -d alsa
# Launch rtfi:
$ ./rtfi [width] [height]
# or go fullscrenn with
$ ./rtfi f

Within the program you can use the following key controls:

ESC, q

exit

UP, DOWN

move y axes. Holding SHIFT moves by a greater amount

LEFT, RIGHT

change between ARTFI, AES, PES; SPES and NPES

Building

Requirements

jack-audio-connection-kit

Compilation

type:

$ make

To rebuild the autogenerated files use:

$ make generated

To clean use make clean. To wipe everything use make wipe.

Implementation details

This code uses the jack-audio-connection-kit and SDL. The filter bank runs as a JACK callback, and outputs a ARTFI frame (consisting of 900 bins) every 10ms to a circular buffer capable of holding 6 frames. The drawing thread reads from this buffer. Synchronization is done by semaphores, where the value of the semaphore represents the number of frames waiting to be read. The drawing thread processes the ARTFI data to obtain AES, PES, SPES and NPES. Finally, there is a UI thread processes keyboard events.

The script rtfi.py calculates the coefficients and generates the C-files containing the tables.

Current limitations & issues

The multirate processing does not take into account the delays introduced by decimating filters. Because of this, the lower-frequency bins of the ARTFI are delayed with respect to the higher frequency ones. This degrades the note detection capabilities for fast and short notes. Low frequency content and percussion also affects the note detection ability. Equalizing the signal before feeding it to the RTFI can alleviate the problem. This can be done easily with jack-rack or similar software.

[Zhou]

R. Zhou and M. Mattavelli, "A new time-frequency representation for music signal analysis: resonator time-frequency image," in Proceedings of the 9th International Symposium on Signal Processing and Its Applications (ISSPA '07), Sharijah, UAE, February 2007.

rtfi's Issues

Resizing the window?

Very neat visualiser!

How about the ability to resize the visual when the window is resized?