sinshu/rustysynth

A SoundFont MIDI synthesizer written in pure Rust

RustySynth

RustySynth is a SoundFont MIDI synthesizer written in pure Rust, ported from MeltySynth.

Features

• Suitable for both real-time and offline synthesis.
• Supports standard MIDI files with additional features including dynamic tempo changing.
• No dependencies other than the standard library.

Demo

This is a demo video to show the synthesizer running on rust-sfml in real-time.

https://www.youtube.com/watch?v=o9rPTJIPmVk

Installation

RustySynth is available on crates.io:

cargo add rustysynth

Examples

An example code to synthesize a simple chord:

// Load the SoundFont.
let mut sf2 = File::open("TimGM6mb.sf2").unwrap();
let sound_font = Arc::new(SoundFont::new(&mut sf2).unwrap());

// Create the synthesizer.
let settings = SynthesizerSettings::new(44100);
let mut synthesizer = Synthesizer::new(&sound_font, &settings).unwrap();

// Play some notes (middle C, E, G).
synthesizer.note_on(0, 60, 100);
synthesizer.note_on(0, 64, 100);
synthesizer.note_on(0, 67, 100);

// The output buffer (3 seconds).
let sample_count = (3 * settings.sample_rate) as usize;
let mut left: Vec<f32> = vec![0_f32; sample_count];
let mut right: Vec<f32> = vec![0_f32; sample_count];

// Render the waveform.
synthesizer.render(&mut left[..], &mut right[..]);

Another example code to synthesize a MIDI file:

// Load the SoundFont.
let mut sf2 = File::open("TimGM6mb.sf2").unwrap();
let sound_font = Arc::new(SoundFont::new(&mut sf2).unwrap());

// Load the MIDI file.
let mut mid = File::open("flourish.mid").unwrap();
let midi_file = Arc::new(MidiFile::new(&mut mid).unwrap());

// Create the MIDI file sequencer.
let settings = SynthesizerSettings::new(44100);
let synthesizer = Synthesizer::new(&sound_font, &settings).unwrap();
let mut sequencer = MidiFileSequencer::new(synthesizer);

// Play the MIDI file.
sequencer.play(&midi_file, false);

// The output buffer.
let sample_count = (settings.sample_rate as f64 * midi_file.get_length()) as usize;
let mut left: Vec<f32> = vec![0_f32; sample_count];
let mut right: Vec<f32> = vec![0_f32; sample_count];

// Render the waveform.
sequencer.render(&mut left[..], &mut right[..]);

Yet another example code to synthesize a MIDI file in real-time with TinyAudio:

// Setup the audio output.
let params = OutputDeviceParameters {
    channels_count: 2,
    sample_rate: 44100,
    channel_sample_count: 4410,
};

// Buffer for the audio output.
let mut left: Vec<f32> = vec![0_f32; params.channel_sample_count];
let mut right: Vec<f32> = vec![0_f32; params.channel_sample_count];

// Load the SoundFont.
let mut sf2 = File::open("TimGM6mb.sf2").unwrap();
let sound_font = Arc::new(SoundFont::new(&mut sf2).unwrap());

// Load the MIDI file.
let mut mid = File::open("flourish.mid").unwrap();
let midi_file = Arc::new(MidiFile::new(&mut mid).unwrap());

// Create the MIDI file sequencer.
let settings = SynthesizerSettings::new(params.sample_rate as i32);
let synthesizer = Synthesizer::new(&sound_font, &settings).unwrap();
let mut sequencer = MidiFileSequencer::new(synthesizer);

// Play the MIDI file.
sequencer.play(&midi_file, false);

// Start the audio output.
let _device = run_output_device(params, {
    move |data| {
        sequencer.render(&mut left[..], &mut right[..]);
        for (i, value) in left.iter().interleave(right.iter()).enumerate() {
            data[i] = *value;
        }
    }
})
.unwrap();

// Wait for 10 seconds.
std::thread::sleep(std::time::Duration::from_secs(10));

Todo

• Wave synthesis
  • SoundFont reader
  • Waveform generator
  • Envelope generator
  • Low-pass filter
  • Vibrato LFO
  • Modulation LFO
• MIDI message processing
  • Note on/off
  • Bank selection
  • Modulation
  • Volume control
  • Pan
  • Expression
  • Hold pedal
  • Program change
  • Pitch bend
  • Tuning
• Effects
  • Reverb
  • Chorus
• Other things
  • Standard MIDI file support
  • MIDI file loop extension support
  • Performace optimization

License

RustySynth is available under the MIT license.

rustysynth's Issues

Use smaller integer types for MIDI events

Synthesizer::note_on, Synthesizer::note_off, Synthesizer::process_midi_message and friends expect input parameters that are a byte each, i.e. u8 or i8. However, currently, they all take i32, making it difficult to guess what ranges of values are expected.

Most notably, note_on will fail silently with a velocity larger than i8::MAX.

Would it be possible for these public APIs (and ideally their internals as well, so that the library doesn't consume more memory than needed) to be adjusted to use smaller integer types that better represent their actual size?

BaseAudioOutputDevice/run_output_device() persistence

The examples mention running the run_output_device() function to play audio. It starts the render loop from JS by scheduling regular callbacks that call the Sequencer to generate the samples.

Should the dyn BaseAudioOutputDevice be persisted in the application, and run forever, or is it recommended to reopen/recreate the audio output device every time when one wants to play a sound?

No audio on Windows (using Tauri)

I'm making an application with Tauri, and running the demo code (using an appropriate soundfont) produces no sound whatsoever.

It doesn't produce any errors either, which could have been helpful.

I'm on Windows. I don't think using Tauri should affect the output in any way.

Embedded?

This is awesome!

Does this work on embedded / microcontrollers ? Like Raspberry Pi Pico or something?

What's the smallest/simplest chip you've run this on?

No support for sf3

I tried using ms_basic.sf3 from MuseScore. Everything loads correctly but most of the presets sounds like bad white noise.

I converted ms_basic.sf3 to an sf2 file using Polyphone. rustysynth handles ms_basic.sf2 correctly.

Possibly, this is because sf3 uses ogg vorbis samples.

Use more informative error types

It would be much easier for downstream crates to handle errors if they were returned as enums rather than String errors in the form Box<dyn Error>. Not only are the current errors impossible to match on, they are also incompatible with error handling solutions such as anyhow.

If this change would be welcome, I would be willing to PR it?

Skip bytes if the track end event ends before the chunk length

I have this midi file that ends the track chunk with multiple track end events:

Currently, the MidiFile struct will see the track end and immediately attempt to process the next event as an MTrk header giving me an InvalidChunkType for "??/?". Tracking the number of bytes read and skipping excess bytes (the extra event) allows the file to be fully read + play.

I have a fork that wraps the reader to track bytes read for resolving how many bytes to skip, not sure if there's a better solution.

Event callbacks

For the integration in applications, a client will need to know the offset of notes being played so they can update a UI. Can one register a listener/callback somewhere to listen for MIDI events when playing?

flathub

flatpak is the main linux distribuition platform

flathub.org

Mix individual instrument .sf2 files

Hi, thank you for this great work! I have been waiting for a proper Rust SF2+MIDI synth. It will be perfect for parture.org

Would it be possible to assemble a synth from multiple individual instrument .sf2 files? That would allow only loading the neccessary instruments for a piece instead of a potentially large .sf2 that covers the complete General MIDI.