danilobellini/audiolazy

Expressive Digital Signal Processing (DSP) package for Python

AudioLazy

Development
Last release
PyPI status

Real-Time Expressive Digital Signal Processing (DSP) Package for Python!

Laziness and object representation

There are several tools and packages that let the Python use and expressiveness look like languages such as MatLab and Octave. However, the eager evaluation done by most of these tools make it difficult, perhaps impossible, to use them for real time audio processing. To avoid such eagerness, one can make the calculations only when data is requested, not when the path to the data is given. This is the core idea in laziness that allows:

• Real-time application (you don't need to wait until all data is processed to have a result);
• Endless data sequence representation;
• Data-flow representation;
• Task elimination when a reverse task is done: instead of doing something to then undo, nothing needs to be done, and no conscious optimization need to be done for that.

Another difficulty concerns expressive code creation for audio processing in blocks through indexes and vectors. Sometimes, that's unavoidable, or at least such avoidance would limit the power of the system that works with sequence data.

Block sequences can be found from sample sequences being both objects, where the latter can be the result of a method or function over the former. The information needed for such is the block size and where would start the next block. Although one can think about the last block and the exact index where it would start, most of the time spent in steps like this one happens to be an implementation issue that just keep the focus away from the problem being worked on. To allow a thing like an endless data sequence, there should be no need to know when something stops.

Probably an engineer would find the use of equations and structures from electrical engineering theory much cleaner to understand than storing everything into data arrays, mainly when common operations are done to these representations. What is the product of the filter with numerator [1, 7, 2] and denominator [1, 0.5, 0.2] as its system equation with the one that has the arrays reversed like [2, 7, 1]? That might be simple, and the reversed would avoid questions like "what comes first, the zero or the [minus] two exponent?", but maybe we could get more efficient ourselves if we had something easier: multiplication could be written once and for all and with a representation programmers are used to see. This would be even more expressive if we could get rid from the asymmetry of a method call like filt1.multiply_by(filt2), since multiplication in this case should be commutative. The use of natural operators is possible in a language that allows operator overloading, but for such we need to describe those equations and structures as objects and object relationships.

The name Hz can be a number that would allow conversion to a default DSP internal rad/samples unit, so one can write things like freq = 440 * Hz. This isn't difficult in probably any language, but can help in expressiveness, already. If (almost) everything would need data in "samples" or "rad/sample" units, constants for converting these from "second" and "hertz" would help with the code expressiveness. A comb filter comb.tau(delay=30*s, tau=40*s) can represent a comb filter with the given delay and time constant, both in samples, but with a more clear meaning for the reader than it would have with an expression like [1] + [0] * 239999 + [alpha]. Would it be needed to store all those zeros while just using the filter to get a frequency response plot?

It's possible to avoid some of these problems with well-chosen constants, duck typing, overloaded operators, functions as first-class citizens, object oriented together with functional style programming, etc.., resources that the Python language gives us for free.

What does it do?

Prioritizing code expressiveness, clarity and simplicity, without precluding the lazy evaluation, and aiming to be used together with Numpy, Scipy and Matplotlib as well as default Python structures like lists and generators, AudioLazy is a package written in pure Python proposing digital audio signal processing (DSP), featuring:

• A Stream class for finite and endless signals representation with elementwise operators (auto-broadcast with non-iterables) in a common Python iterable container accepting heterogeneous data;
• Strongly sample-based representation (Stream class) with easy conversion to block representation using the Stream.blocks(size, hop) method;
• Sample-based interactive processing with ControlStream;
• Streamix mixer for iterables given their starting time deltas;
• Multi-thread audio I/O integration with PyAudio;
• Linear filtering with Z-transform filters directly as equations (e.g. filt = 1 / (1 - .3 * z ** -1)), including linear time variant filters (i.e., the a in a * z ** k can be a Stream instance), cascade filters (behaves as a list of filters), resonators, etc.. Each LinearFilter instance is compiled just in time when called;
• Zeros and poles plots and frequency response plotting integration with MatPlotLib;
• Linear Predictive Coding (LPC) directly to ZFilter instances, from which you can find PARCOR coeffs and LSFs;
• Both sample-based (e.g., zero-cross rate, envelope, moving average, clipping, unwrapping) and block-based (e.g., window functions, DFT, autocorrelation, lag matrix) analysis and processing tools;
• A simple synthesizer (Table lookup, Karplus-Strong) with processing tools (Linear ADSR envelope, fade in/out, fixed duration line stream) and basic wave data generation (sinusoid, white noise, impulse);
• Biological auditory periphery modeling (ERB and gammatone filter models);
• Multiple implementation organization as StrategyDict instances: callable dictionaries that allows the same name to have several different implementations (e.g. erb, gammatone, lowpass, resonator, lpc, window);
• Converters among MIDI pitch numbers, strings like "F#4" and frequencies;
• Polynomials, Stream-based functions from itertools, math, cmath, and more! Go try yourself! =)

Installing

The package works both on Linux and on Windows. You can find the last stable version at PyPI and install it with the usual Python installing mechanism:

python setup.py install

If you have pip, you can go directly (use -U for update or reinstall):

pip install audiolazy

for downloading (from PyPI) and installing the package for you, or:

pip install -U .

To install from a path that has the setup.py file and the package data uncompressed previously.

For the bleeding-edge version, you can install directly from the github repository (requires git for cloning):

pip install -U git+git://github.com/danilobellini/audiolazy.git

For older versions, you can install from the PyPI link or directly from the github repository, based on the repository tags. For example, to install the version 0.04 (requires git for cloning):

pip install -U git+git://github.com/danilobellini/[email protected]

The package doesn't have any strong dependency for its core besides the Python itself (versions 2.7, 3.2 or newer) as well as its standard library, but you might need:

• PyAudio: needed for playing and recording audio (AudioIO class);
• NumPy: needed for doing some maths, such as finding the LSFs from a filter or roots from a polynomial;
• MatPlotLib: needed for all default plotting, like in LinearFilter.plot method and several examples;
• SciPy (testing and examples only): used as an oracle for LTI filter testing and for the Butterworth filter example;
• Sympy (testing only): used for testing linear filters with time-varying matrices of symbolic coeffs where the Stream samples are these matrices;
• tox for testing all at once, or pytest, pytest-cov and pytest-timeout for testing in a single environment (testing only): runs test suite and shows code coverage status;
• wxPython (example only): used by one example with FM synthesis in an interactive GUI;
• Tkinter (example only): needed for the pitch follower based on the zero-crossing rate example GUI;
• Music21 (example only): there's one example that gets the Bach chorals from that package corpora for synthesizing and playing;
• Sphinx (documentation only): it can create the software documentation in several different file formats.

Beside examples and tests, only the filter plotting with plot and zplot methods needs MatPlotLib. Also, the routines that needs NumPy up to now are:

• Root finding with zeros and poles properties (filter classes) or with roots property (Poly class);
• Some Linear Predictive Coding (lpc) strategies: nautocor, autocor and covar;
• Line Spectral Frequencies lsf and lsf_stable functions.

Getting started

Before all examples below, it's easier to get everything from audiolazy namespace:

from audiolazy import *

All modules starts with "lazy_", but their data is already loaded in the main namespace. These two lines of code do the same thing:

from audiolazy.lazy_stream import Stream
from audiolazy import Stream

Endless iterables with operators (be careful with loops through an endless iterator!):

>>> a = Stream(2, -2, -1) # Periodic
>>> b = Stream(3, 7, 5, 4) # Periodic
>>> c = a + b # Elementwise sum, periodic
>>> c.take(15) # First 15 elements from the Stream object
[5, 5, 4, 6, 1, 6, 7, 2, 2, 9, 3, 3, 5, 5, 4]

And also finite iterators (you can think on any Stream as a generator with elementwise operators):

>>> a = Stream([1, 2, 3, 2, 1]) # Finite, since it's a cast from an iterable
>>> b = Stream(3, 7, 5, 4) # Periodic
>>> c = a + b # Elementwise sum, finite
>>> list(c)
[4, 9, 8, 6, 4]

LTI Filtering from system equations (Z-transform). After this, try summing, composing, multiplying ZFilter objects:

>>> filt = 1 - z ** -1 # Diff between a sample and the previous one
>>> filt
1 - z^-1
>>> data = filt([.1, .2, .4, .3, .2, -.1, -.3, -.2]) # Past memory has 0.0
>>> data # This should have internally [.1, .1, .2, -.1, -.1, -.3, -.2, .1]
<audiolazy.lazy_stream.Stream object at ...>
>>> data *= 10 # Elementwise gain
>>> [int(round(x)) for x in data] # Streams are iterables
[1, 1, 2, -1, -1, -3, -2, 1]
>>> data_int = filt([1, 2, 4, 3, 2, -1, -3, -2], zero=0) # Now zero is int
>>> list(data_int)
[1, 1, 2, -1, -1, -3, -2, 1]

LTI Filter frequency response plot (needs MatPlotLib):

(1 + z ** -2).plot().show()

The matplotlib.figure.Figure.show method won't work unless you're using a newer version of MatPlotLib (works on MatPlotLib 1.2.0), but you still can save the above plot directly to a PDF, PNG, etc. with older versions (e.g. MatPlotLib 1.0.1):

(1 + z ** -2).plot().savefig("my_plot.pdf")

On the other hand, you can always show the figure using MatPlotLib directly:

from matplotlib import pyplot as plt # Or "import pylab as plt"
filt = 1 + z ** -2
fig1 = filt.plot(plt.figure()) # Argument not needed on the first figure
fig2 = filt.zplot(plt.figure()) # The argument ensures a new figure
plt.show()

CascadeFilter instances and ParallelFilter instances are lists of filters with the same operator behavior as a list, and also works for plotting linear filters. Constructors accepts both a filter and an iterable with filters. For example, a zeros and poles plot (needs MatPlotLib):

filt1 = CascadeFilter(0.2 - z ** -3) # 3 zeros
filt2 = CascadeFilter(1 / (1 -.8 * z ** -1 + .6 * z ** -2)) # 2 poles
# Here __add__ concatenates and __mul__ by an integer make reference copies
filt = (filt1 * 5 + filt2 * 10) # 15 zeros and 20 poles
filt.zplot().show()

Linear Predictive Coding (LPC) autocorrelation method analysis filter frequency response plot (needs MatPlotLib):

lpc([1, -2, 3, -4, -3, 2, -3, 2, 1], order=3).plot().show()

Linear Predictive Coding covariance method analysis and synthesis filter, followed by the frequency response plot together with block data DFT (MatPlotLib):

>>> data = Stream(-1., 0., 1., 0.) # Periodic
>>> blk = data.take(200)
>>> analysis_filt = lpc.covar(blk, 4)
>>> analysis_filt
1 + 0.5 * z^-2 - 0.5 * z^-4
>>> residual = list(analysis_filt(blk))
>>> residual[:10]
[-1.0, 0.0, 0.5, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]
>>> synth_filt = 1 / analysis_filt
>>> synth_filt(residual).take(10)
[-1.0, 0.0, 1.0, 0.0, -1.0, 0.0, 1.0, 0.0, -1.0, 0.0]
>>> amplified_blk = list(Stream(blk) * -200) # For alignment w/ DFT
>>> synth_filt.plot(blk=amplified_blk).show()

AudioLazy doesn't need any audio card to process audio, but needs PyAudio to play some sound:

rate = 44100 # Sampling rate, in samples/second
s, Hz = sHz(rate) # Seconds and hertz
ms = 1e-3 * s
note1 = karplus_strong(440 * Hz) # Pluck "digitar" synth
note2 = zeros(300 * ms).append(karplus_strong(880 * Hz))
notes = (note1 + note2) * .5
sound = notes.take(int(2 * s)) # 2 seconds of a Karplus-Strong note
with AudioIO(True) as player: # True means "wait for all sounds to stop"
  player.play(sound, rate=rate)

See also the docstrings and the "examples" directory at the github repository for more help. Also, the huge test suite might help you understanding how the package works and how to use it.

Project files

There are several files and directories in the AudioLazy repository (as well as in the source distribution):

File/Directory	Description
audiolazy/	AudioLazy package modules source code
audiolazy/tests/	Testing subpackage
docs/	Documentation generation scripts
examples/	Examples for some AudioLazy use cases
images/	Images referenced by some reStructuredText documentation file
math/	Proof for some formula used by AudioLazy using Sympy CAS
CHANGES.rst	AudioLazy History, a.k.a. change log
conftest.py	Configuration for py.test, to work properly with doctests on StrategyDict strategies and on an environment missing Numpy
COPYING.txt	License file
MANIFEST.in	List of extra distributed files to be included in the tarball that doesn't need to be installed together with the package
README.rst	Some general information about the AudioLazy project
setup.py	General Python setup script for installation, testing, etc.
tox.ini	Configuration for tox, py.test and pytest-cov
.travis.yml	Travis-CI configuration (not in PyPI tarball/"egg" source distribution)

The examples and the math directories might be useful for an AudioLazy user. All Python files in these two directories are scripts intended to run on both Python 2 and 3 unless they need something not yet available for Python 3 (e.g. wxPython), most of them have some external dependency.

---

Copyright (C) 2012-2016 Danilo de Jesus da Silva Bellini

License is GPLv3. See COPYING.txt for more details.

audiolazy's Issues

Support for newer python versions (> 3.7)

Bumping the python version from 3.6 fails because of a missing import from the collections library:

ImportError                               Traceback (most recent call last)
Cell In [2], line 7
      5 from scipy import signal
      6 import IPython.display as ipd
----> 7 from audiolazy import levinson_durbin
      8 import speech_test
      9 get_ipython().run_line_magic('matplotlib', 'inline')

File /opt/conda/lib/python3.10/site-packages/audiolazy/__init__.py:67
     16 """
     17 AudioLazy package
     18 
   (...)
     61 under the terms of the GPLv3.
     62 """
     64 # Some dunders and summary docstrings initialization
     65 __modules__, __all__, __doc__ = \
     66   __import__(__name__ + "._internals", fromlist=[__name__]
---> 67             ).init_package(__path__, __name__, __doc__)
     69 # Import all modules contents to the main namespace
     70 exec(("from .{} import *\n" * len(__modules__)).format(*__modules__))

File /opt/conda/lib/python3.10/site-packages/audiolazy/_internals.py:117, in init_package(package_path, package_name, docstring)
    102 """
    103 Package initialization, to be called only by ``__init__.py``.
    104 
   (...)
    114 used by the package to import every module into the main package namespace.
    115 """
    116 module_names = get_module_names(package_path)
--> 117 modules = get_modules(package_name, module_names)
    118 dunder_all = dunder_all_concat(modules)
    119 for module in modules:

File /opt/conda/lib/python3.10/site-packages/audiolazy/_internals.py:59, in get_modules(package_name, module_names)
     57 def get_module(name):
     58   return __import__(".".join([package_name, name]), fromlist=[package_name])
---> 59 return [get_module(name) for name in module_names]

File /opt/conda/lib/python3.10/site-packages/audiolazy/_internals.py:59, in <listcomp>(.0)
     57 def get_module(name):
     58   return __import__(".".join([package_name, name]), fromlist=[package_name])
---> 59 return [get_module(name) for name in module_names]

File /opt/conda/lib/python3.10/site-packages/audiolazy/_internals.py:58, in get_modules.<locals>.get_module(name)
     57 def get_module(name):
---> 58   return __import__(".".join([package_name, name]), fromlist=[package_name])

File /opt/conda/lib/python3.10/site-packages/audiolazy/lazy_analysis.py:23
     20 from __future__ import division
     22 from math import sin, cos, pi
---> 23 from collections import deque, Sequence, Iterable
     24 from functools import wraps, reduce
     25 from itertools import chain

ImportError: cannot import name 'Sequence' from 'collections' (/opt/conda/lib/python3.10/collections/__init__.py)

ALSA underrun above rate=22000

Hey Danilo love the library. Hoping to make use of the lazy eval to do some DSP for a visual-audio art piece I'm working on.

When I try to run the examples I get static noise and stdout contains errors about ALSA lib pcm.c:8526:(snd_pcm_recover) underrun occurred

I eventually found that lowering rate on AudioIo.play avoids these errors.

sound = WavStream("static/mono.wav")
print(sound.channels, sound.rate)

with AudioIO(True, api="alsa") as player: 
    player.play(sound, rate= int(sound.rate // 3))  # Plays correctly

with AudioIO(True) as player:
    player.play(sound, rate= int(sound.rate // 2))

The first play is slowed down 3x but otherwise plays correctly. During the second play an error is printed about a hundred times to the console and the sound is just strongly static (with a little of the correct signal mixed in) for ~1 second (the clip is 22 seconds long).

Any idea why this might would be happening?

"OSError: [Errno -9985] Device unavailable"

While trying to use the I/O wire example provided with the master (io_wire.py) with any other audio API than the default, I always get this error (in PyCharm CE, W7 x64 with Python 3.5.1)):
(The [...]'s are redactions on my path, the actual traceback contains the full path.)

Traceback (most recent call last):
  File "C:/[...]/audiolazy_master/examples/io_wire.py", line 46, in <module>
    pr.play(pr.record())
  File "C:\[...]\lib\site-packages\audiolazy\lazy_io.py", line 270, in play
    new_thread = AudioThread(self, audio, **kwargs)
  File "C:\[...]\lib\site-packages\audiolazy\lazy_io.py", line 401, in __init__
    **kwargs)
  File "C:\[...]\lib\site-packages\pyaudio.py", line 750, in open
    stream = Stream(self, *args, **kwargs)
  File "C:\[...]\lib\site-packages\pyaudio.py", line 441, in __init__
    self._stream = pa.open(**arguments)
OSError: [Errno -9985] Device unavailable

Process finished with exit code 1

Here is the modified script for the wire:

[...]
from audiolazy import chunks, AudioIO
import sys

# Choose API via command-line
api = sys.argv[1] if sys.argv[1:] else None
print(api)

# Amount of samples per chunk to be sent to PortAudio
chunks.size = 1 if api == "jack" else 16

with AudioIO(True, api="asio") as pr: # A player-recorder
  pr.play(pr.record())

When using the I/O wire example provided with the PyAudio master (PyAudio-0.2.9.tar/test/wire_full.py), the wire works perfectly fine, the ASIO device is detected and the wire established. Here is the script, with the index of my ASIO device being 12:

import pyaudio
import sys

CHUNK = 256
WIDTH = 2
CHANNELS = 1
RATE = 44100
RECORD_SECONDS = 5

if sys.platform == 'darwin':
    CHANNELS = 1

p = pyaudio.PyAudio()

stream = p.open(format=p.get_format_from_width(WIDTH),
                channels=CHANNELS,
                rate=RATE,
                input=True,
                input_device_index=12,
                output_device_index=12,
                output=True,
                frames_per_buffer=CHUNK)

print("* recording")

for i in range(0, int(RATE / CHUNK * RECORD_SECONDS)):
    data = stream.read(CHUNK)
    stream.write(data, CHUNK)

print("* done")

stream.stop_stream()
stream.close()

p.terminate()

Here's the info for the device, as given by as given by p.get_device_info_by_index(12):

{'defaultLowInputLatency': 0.0014512471655328798, 'name': 'Yamaha Steinberg USB ASIO', 'defaultHighInputLatency': 0.0014512471655328798, 'index': 12, 'structVersion': 2, 'maxOutputChannels': 2, 'defaultLowOutputLatency': 0.0014512471655328798, 'maxInputChannels': 2, 'defaultHighOutputLatency': 0.0014512471655328798, 'hostApi': 2, 'defaultSampleRate': 44100.0}

Sorry if it it's a PICNIC.

Log plot sampling

The LinearFilter.plot method samples nicely for linear plots, but when asked to output logarithmic plots (i.e., when fscale == "log") the freqs (and freqs_label)
sampling values should also be logarithmically distributed to keep the pixel-distance distribution linear, like what happens in iso226_plot.py.

AttributeError: module 'audiolazy.lazy_lpc' has no attribute 'nautocor'

Hi,i'm using audiolazy to process lpc feature,but i get error below:

AttributeError: module 'audiolazy.lazy_lpc' has no attribute 'nautocor'
Anyone knows the reason,please help me.

Tests fail for pytest 3.2 and later

The tests are failing for pytest 3.2 and later, since the location of the XFailed class has moved.

audiolazy/tests/__init__.py:21: in <module>
    from _pytest.skipping import XFailed
E   ImportError: cannot import name XFailed

[Question] AR prediction with ZFilter FIR

Iam trying to understand how the ZFilter object does its operation in the fitted time series (with Levinson Durbin AutoCorrelation coefficients) when you put the time series in the object caller in order to do Auto Regression, ie, x[t] = a[t-1]*x[t-1]+a[t-2]*x[t-2]+...+a[t-order]*x[t-order]

so i call:

Order=...
zf=levinson_durbin(acorr(timeseries[N-Order-1:N-1]),order=Order) # here iam fitting the 'a' coefficients according to the last Order elements of timeseries

fitted=zf(timeseries[N-Order-1:N-1]).take(Order)) # what exactly the FIR filter is doing here ? when 
 i call take to get Order elements based on the timeseries window of size Order ? 

Seems that the first element of the fitting by FIR, when i call take, is almost exactly the the same of the original series, while the others are not.

I would like to know what the FIR is doing depending upon the argment of .take, and why i cannot take more then Order elements, since i would like to do the AutoRegressive prevision, then use it to do another prevision with the same fitted FIR (like in the AR formula given above).

Thank you for your time and patience.

Plot methods testing

Some AudioLazy methods lacks automated testing. Coveralls says the least covered file is the lazy_filters.py, and most of the tests it's lacking is about its interface with Matplotlib.

It seems difficult to test a plot. But some tests might be done on the output figure to see at least whether:

• Whether the output result is indeed a figure
• Assert if the amount of axes is ok
• Axes data should match expected values (for different input parameters)
• A circle should be printed in the zplot

A much more difficult test would be matching the resulting image with an image analysis procedure to "see" (literally) whether the elements (and their displacement) are correct. Testing pixel-by-pixel wouldn't help if small details are changed (e.g. font face in different systems), so that shouldn't be done.

For now (and for this issue), at least the basic internal information itemized above should be tested.

requirements.txt

I think will be nice use it for contributors build your own environment. =)

http://www.pip-installer.org/en/latest/requirements.html

Issue with STFT overlap_add: ValueError: need at least one array to concatenate

Hello,

I tried both the robotize and roll magnitude examples and experienced the same error. From robotize:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib64/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/sevagh/.local/lib/python3.8/site-packages/audiolazy/lazy_io.py", line 414, in run
    for chunk in chunks(self.audio,
  File "/home/sevagh/.local/lib/python3.8/site-packages/audiolazy/lazy_io.py", line 89, in chunks
    for block in blocks(seq, size, padval=padval):
  File "/home/sevagh/.local/lib/python3.8/site-packages/audiolazy/lazy_misc.py", line 99, in blocks
    for el in seq:
  File "/home/sevagh/.local/lib/python3.8/site-packages/audiolazy/lazy_analysis.py", line 790, in overlap_add
    gain = np.sum(np.abs(np.vstack(steps)), 0).max()
  File "<__array_function__ internals>", line 5, in vstack
  File "/home/sevagh/.local/lib/python3.8/site-packages/numpy/core/shape_base.py", line 283, in vstack
    return _nx.concatenate(arrs, 0)
  File "<__array_function__ internals>", line 5, in concatenate
ValueError: need at least one array to concatenate

The stacktrace shows Python 3.8, so I downgraded to 3.6 and get the same error.

Thanks.