How to Implement a custom Audio Effect Node?

[Samigos]

How to Implement a custom Audio Effect Node?

[Samigos]

I’ve been trying to figure out how other effects work, like PitchShifter, so I can have a starting point, but I can’t find the actual algorithm implementation. Also, there is no example online anywhere! So I would really appreciate anyone's help!

[Samigos]

Here's what I've done so far:

#include "SoundpipeDSPBase.h"
#include "ParameterRamper.h"
#include "Soundpipe.h"
#include "CSoundpipeAudioKit.h"
#include "tuna.h"

enum TunaParameter : AUParameterAddress {
    TunaFilterPitchCorrectionParameterSpeed,
};

class TunaDSP : public SoundpipeDSPBase {
private:
    float speed = 0;
    ParameterRamper speedRamp;
    tuna *tuner;
    
    
public:
    TunaDSP() {
        parameters[TunaFilterPitchCorrectionParameterSpeed] = &speedRamp;
    }
    
    void setSpeed(float speed) {
        this->speed = speed;
        tuner->setpar(tuna::TUNA_SPEED, speed);
        reset();
    }
    
    void init(int channelCount, double sampleRate) override {
        SoundpipeDSPBase::init(channelCount, sampleRate);
        tuner = new tuna();
        tuner->setpar(tuna::TUNA_SPEED, 1); 
    }
    
    void deinit() override { 
        SoundpipeDSPBase::deinit(); 
    }
    
    void reset() override {
        SoundpipeDSPBase::reset();
        if (!isInitialized) return;
        tuner = new tuna(); 
    }
    
#define CHUNKSIZE 8     // defines ramp interval
    
    void process(FrameRange range) override {
        const float *inBuffers[2];
        float *outBuffers[2];
        inBuffers[0]  = (const float *)inputBufferLists[0]->mBuffers[0].mData + range.start;
        inBuffers[1]  = (const float *)inputBufferLists[0]->mBuffers[1].mData + range.start;
        outBuffers[0] = (float *)outputBufferList->mBuffers[0].mData + range.start;
        outBuffers[1] = (float *)outputBufferList->mBuffers[1].mData + range.start;
        //unsigned inChannelCount = inputBufferLists[0]->mNumberBuffers;
        //unsigned outChannelCount = outputBufferList->mNumberBuffers;
        
        if (!isStarted)
        {
            // effect bypassed: just copy input to output
            memcpy(outBuffers[0], inBuffers[0], range.count * sizeof(float));
            memcpy(outBuffers[1], inBuffers[1], range.count * sizeof(float));
            return;
        }
        
        // process in chunks of maximum length CHUNKSIZE
        for (int frameIndex = 0; frameIndex < range.count; frameIndex += CHUNKSIZE)
        {
            int chunkSize = range.count - frameIndex;
            if (chunkSize > CHUNKSIZE) chunkSize = CHUNKSIZE;
            
            tuner->process(inBuffers[0], inBuffers[1], outBuffers[0], outBuffers[1], chunkSize);
            
            // advance pointers
            inBuffers[0] += chunkSize;
            inBuffers[1] += chunkSize;
            outBuffers[0] += chunkSize;
            outBuffers[1] += chunkSize;
        }
    }
};

void akTunaFilterPitchCorrectionSetSpeed(DSPRef dspRef, float speed) {
    auto dsp = dynamic_cast<TunaDSP *>(dspRef);
    assert(dsp);
    dsp->setSpeed(speed);
}

AK_REGISTER_DSP(TunaDSP, "tuna")
AK_REGISTER_PARAMETER(TunaFilterPitchCorrectionParameterSpeed)

And a Node:

import Foundation
import AudioKit
import AudioKitEX
import CAudioKitEX

public class PitchCorrectionAudioKitNode: Node {
    let input: Node
    
    /// Connected nodes
    public var connections: [Node] { [input] }
    
    /// Underlying AVAudioNode
    public var avAudioNode = instantiate(effect: "tuna")
    
    // MARK: - Parameters
    
    /// Specification details for feedback
    public static let speedDef = NodeParameterDef(
        identifier: "speed",
        name: "Speed",
        address: akGetParameterAddress("TunaFilterPitchCorrectionParameterSpeed"),
        defaultValue: 0.6,
        range: 0.0 ... 1.0,
        unit: .percent
    )
    
    @Parameter(speedDef) public var speed: AUValue
    
    // MARK: - Initialization
    
    /// Initialize this pitch correction node
    ///
    /// - Parameters:
    ///   - input: Input node to process
    ///
    public init(
        _ input: Node,
        speed: AUValue = speedDef.defaultValue
    ) {
        self.input = input
        
        setupParameters()
        
        self.speed = speed
    }
}

Now it works.

[emurray2]

Hi @Samigos,

Thank you for your inquiry! What does tuner->process() do? I haven't seen anywhere in the code where the output buffers are assigned a new value which might explain why the audio is silent.

I think you've almost answered your own questions here. The best place to check for examples on these is the Soundpipe code itself. There are many nodes implemented there which do various things.

Additionally, I'm not sure if you're a member of the AudioKit Discord yet, but please send me an email if not. We would love to have you join and ask any questions you might have there 🙂!

[emurray2]

I’ve been trying to figure out how other effects work, like PitchShifter, so I can have a starting point, but I can’t find the actual algorithm implementation. Also, there is no example online anywhere! So I would really appreciate anyone's help!

https://github.com/AudioKit/SoundpipeAudioKit/blob/main/Sources/Soundpipe/modules/pshift.c

[Samigos]

Hi @emurray2!

The tuner->process() is from a static library. Here's its declaration:
- (void)processStereo:(const float *)srcL leftDestination:(float *)dstL rightSource:(const float *)srcR rightDestination:(float *)dstR numSamples:(int)nsmp;

Additionally, I'm not sure if you're a member of the AudioKit Discord yet, but please send me an email if not. We would love to have you join and ask any questions you might have there 🙂!

I'd love to join! Where can I send you my email?

[emurray2]

Hi @emurray2!

The tuner->process() is from a static library. Here's its declaration:

- (void)processStereo:(const float *)srcL leftDestination:(float *)dstL rightSource:(const float *)srcR rightDestination:(float *)dstR numSamples:(int)nsmp;

Additionally, I'm not sure if you're a member of the AudioKit Discord yet, but please send me an email if not. We would love to have you join and ask any questions you might have there 🙂!

I'd love to join! Where can I send you my email?

You can send it to my GitHub email: [email protected]

Something about the wording of this declaration confuses me and it's hard to tell what's happening without the implementation. The destination is usually where the output goes in most cases, so that shouldn't be const.

‘tuner->process(inBuffers[0], inBuffers[1], outBuffers[0], outBuffers[1], chunkSize);‘

Additionally, the order of parameters above is mixed up. It should be ‘tuner->process(inBuffers[0], outBuffers[0], inBuffers[1], outBuffers[1])‘

[Samigos]

Yeah you are right! I gave you the wrong declaration... here's the correct one
void process(const float *srcl, const float *srcr, float *dstl, float *dstr, int nsmp)

[emurray2]

Alright, I'm going to go ahead and close this now that you're in the Discord, but if you do find an issue with the AudioKit code itself related to this, please feel free to reopen it.