sonidosmutantes/apicultor

[Deprecated] py2.7 API.Cultor: Another BigData approach to sound collections

About the project

• Interactive DEMO:
  • Cloud Instrument raspicultor + custom controller version video. Interactive demo retrieving sounds from the Cloud using MIR descriptors and processing them in realtime (using raspicultor aka raspberry pi + apicultor).
  • Experimental session with sounds from the cloud. First live demo with an standard midi controller.
• Embedded instrument running in a Bela or Raspberry Pi: cloudpicultor.
• Cloud Instrument mention in ArCiTec prize, Technology applied to Arts (2018).
• Publication: Sound recycling from public databases @ Audio Mostly 2017
• "Forgotten pieces from unknown people" and "Dialectic in suspense" constructed using APICultor, were showed in NIPS 2017 Art Gallery (Machine Learning for Creativity and Design, Long Beach, California, USA)
• "Sound recycling from public databases" was presented at Technarte Los Angeles 2017.
• Recycled sounds: Sound miniatures

APICultor born to realize interdisciplinary performances based on sounds of the web platform http://redpanal.org. The system is also useful to use with any other sound database on the internet or even running it locally.

The sound is processed digitally using different live-coding techniques. A pre-analysis based on Music Information Retrieval (MIR) stored in a database and accessed via a web-service REST API is combined with real-time processing and synthesis, random processes and human control via external interfaces.

Examples available with SuperCollider, Pyo and pure data.

Spanish version: README_es.md

News

• 2018
  • Migration to Python3 and setup.py install thanks to MarsCrop!
  • MusicEmotionMachine by MarsCrop (in development)
  • Cloud Instrument ready to play with an external controller, running in a dedicated device like Raspberry Pi or Bela Platform
• 2017:
  • SuperCollider patch to play in realtime using sounds from public repositories (databases on the web). Realtime synthesis, wav retrieving using MIR descriptors, OSC & MIDI communication between process and controllers.
• 2016: First demos (extracting MIR features, database build, MIR State Machine, etc)

Developers

See Development Guidelines.

License

Free Software shared with GPL v3, see LICENSE.

Cloud Instrument

Using a desktop computer, a Raspberry pi or bela platform.

See cloud_instrument/README.md

Interactive DEMO: Cloud Instrument. Interactive demo retrieving sounds from the Cloud using MIR descriptors and processing them in realtime (using raspicultor aka raspberry pi + apicultor).

UI

Open Stage Control User Interface

Custom MIDI Controller design

Yaeltex custom MIDI controllers

With a SuperCollider synthesizer/effects processor running in a Raspberry pi, plus an external sound card for high fidelity.

Performances

Sonidos Mutantes

Interdisciplinary performances based on sounds of the web platform Redpanal.org

Proofs of concept:

• APICultor + No Input + Granular synthesis live performance.

• Compositions & recycled samples

  • API-Cultor compositions
  • Proyecto de reciclado de sonidos libres de la web en RedPanal.org (sonido + referencia al original).

• Generative music using MIR state machine a free/libre sounds from Freesound.org:

  • "Feature Thinking" (with Creative Commons sounds from Freesound.org) by hordia.
  • State Machine @ CASo (Centro de Arte Sonoro) https://www.youtube.com/watch?v=sG1YUc8PQV4

• Live performances using this tool:

  • Jam con guitarras + fx chain y sintes analógicos: Perfo mutante en La Siesta del Fauno
  • Closing performance of the Workshop of Experimental Sound: http://redpanal.org/a/banda-de-mutantes-cierre-taller/
  • La Noche de los Museos La Casa del Bicentenario: Performance 29/10/2016 Con visuales 3D (Blender game engine) ) Reseña: enjambrebit-y-la-banda-de-mutantes/

• "Remixes que toman audios libres de RedPanal.org para categorizarlos según diferentes tipos de emociones. Luego se elige una y se sincronizan las pistas, cambiando las tonalidades. De ser posible se separan de fuentes dentro de las mismas" (by Mars Crop)

  • Beats intro jazz
  • [Bass & DJ] (http://redpanal.org/a/sm-bass-guitar-plays-with-dj/)

Components

• Mock web service with API REST to provide audio samples using MIR descriptors as parameters
• State machine, with each state defined by several MIR descriptors.
• Interaction with the free internet sound database http://redpanal.org
• API REST
• Webscrapping by tag
• Algorithms MIR to extract mean values or by frame of audio samples
• Segmentation algorithms using different criteria.
• Classify algorithms and clustering of samples of the sound database
• Server OSC
• Examples in Supercollider, pyo
• Examples with MIDI and OSC controller. Locale and remote.

Dependencies

Tested under Linux, Mac OS (>10.11) and Windows 10.

Debian, Ubuntu 15.04 and 16.04 (and .10). And Docker images. Raspian @ Raspberry Pi

See INSTALL.md

apicultor's Issues

Incluir los documentos en un MANIFEST.in

una forma sería removiendo backspacers para no confundir la inclusión de los archivos, alguien entiende mejor los MANIFEST como para no complicarla con los filenames?

Efecto 'freeze' en supercollider + LFO

reproducir un sample en modo freeze

se recibe el path del mismo vía mensaje osc

MIDI in supercollider

Fire events with an external MIDI controller

(dev branch)
https://github.com/sonidosmutantes/apicultor/blob/dev/examples/playing_server_midi.scd

Integrar API en redpanal.org

Meter la API rest en redpanal (accediendo a su db actual) y dejar de usar el mock

Definir el "guión" para la obra

Estructurar

Solve py2 vs py3 (scripts and dependencies)

Manejar varias "capas" en SuperCollider

cada una con diferente sample + freeze effect

Pan and spacialization in SC

Random and MIDI controlled localization

Build (para instalar en el sistema)

Habia hecho algo al respecto, pero falta el setup.py @MarsCrop

listar tags

no encuentro la forma de hacerlo ni online ni por CLI desde WebScrapingDownload.py

Normalizar volumen de los samples

O en el sound server (SuperCollider) utilizar algún tipo de AGC (automatic gain control)

Comunicación con supercollider en otra máquina desde VM

Probar en un windows host. Ver el listener, si tiene que ser * o alcanza con localhost.
(funciona si esta todo corriendo en la misma máquina)

Agregar descriptores

ver run_MIR_analysis.py

Generar samples para drones

a mano y automáticos (por comparación/similaridad) o en base a descriptores

dockerize

https://hub.docker.com/_/ubuntu/

Haria mas facil probar el setup en no-ubuntu distros. No tiene porque ser 1 imagen. Pueden ser mas o reutilizar otras online.

Segmentación

Segmentar los archivos largos generando 'samples' usando essentia o librosa

Calcular descriptor LFC para los samples y contrastar resultados (análisis)

Objetivo: aplicaciónes musicales

LFC (Low Frec Content)

Revisar los resultados

Buscar un umbral?

Hacer tests hasta cubrir el 100% de los métodos

Escribir más tests y obtener alguna métrica de code coverage

Report generation (samples used during a performance)

Authors and license

Log file or report

Start/Stop performance or after exit