mochi-neko/voice-activity-detection-unity

A voice activity detection (VAD) library for Unity.

voice-activity-detection-unity

A voice activity detection (VAD) library for Unity.

Features

Records voice data from any sources (IVoiceSource, e.g. recording by UnityEngine.Microphone), detects voice activity by any logic, and provides voice data to any buffers (IVoiceBuffer, e.g. buffering to WAV file) when voice is active.

You can customize voice sources, voice buffers, and voice activity detection logics adjusting your use cases.

• Sources
  • UnityEngine.Microphone -> UnityMicrophoneSource
  • UnityEngine.AudioSource (OnAudioFilterRead callback) -> UnityAudioSource
  • Native microphone
• Buffers
  • Null (Detection only) -> NullVoiceBuffer
  • Wave file (by NAudio) -> WaveFileVoiceBuffer
  • AudioClip -> AudioClipBuffer
• Voice activity detection logics
  • Queueing-based simple VAD logic -> QueueingVoiceActivityDetector
    • Less memory usage but less stability
  • Cumulative VAD logic -> CumulativeVoiceActivityDetector
    • More stability but more memory usage and less noise robustness

How to import by UnityPackageManager

Add following dependencies to your /Packages/manifest.json.

{
    "dependencies": {
        "com.mochineko.voice-activity-detection": "https://github.com/mochi-neko/voice-activity-detection-unity.git?path=/Assets/Mochineko/VoiceActivityDetection#0.4.2",
        "com.cysharp.unitask": "https://github.com/Cysharp/UniTask.git?path=src/UniTask/Assets/Plugins/UniTask",
        "com.neuecc.unirx": "https://github.com/neuecc/UniRx.git?path=Assets/Plugins/UniRx/Scripts",
        "com.naudio.core": "https://github.com/mochi-neko/simple-audio-codec-unity.git?path=/Assets/NAudio/NAudio.Core#0.2.0",
        ...
    }
}

Samples

• VAD as component
• VAD with echo
• VAD by AudioSource
• VAD with OpenAI/Whisper API transcription
• VAD by cumulative logic

See also Samples.

Change log

See CHANGELOG.

3rd party notices

See NOTICE.

License

Licensed under the MIT license.

voice-activity-detection-unity's Issues

How to use this library as a newcomer to Unity

Hi, I'm new to Unity, how can I use this library, can I request a detailed manual?

Create fields

I am trying to attach namespace to a new created script, but unity does not see it
But it is working in provided sample scripts
Moreover , when I am trying to modify and create field of my type (exmp : public MyScript myscript) , unity can not find the reference name
How to fix this trick ?)