gtreshchev/audioanalysistools

Audio Analysis Tools plugin for Unreal Engine. Provides a variety of functions for analyzing audio data. Works in conjunction with the Runtime Audio Importer plugin.

Audio Analysis Tools

Real-time audio data analysis
Works in conjunction with the Runtime Audio Importer plugin

Explore the docs »

Marketplace . Releases
Discord support chat

Features

• Beat Detection
• Onset Detection
• Core Time Domain Features
• Core Frequency Domain Features
• No any static libraries and external dependencies
• Support for all available devices (Android, iOS, Windows, Mac, Linux, etc)

Legal info

Unreal® is a trademark or registered trademark of Epic Games, Inc. in the United States of America and elsewhere.

Unreal® Engine, Copyright 1998 – 2024, Epic Games, Inc. All rights reserved.

audioanalysistools's Issues

Pause an Play

Sorry for disturbing. Epic store is not suitable to communicate. So i come here.

Thanks for the fast replay. I created audio component in my actor and set it to wave sound. But but i can't connect to your audio analaysis BPs. Caue object reference is not the same. Please can you describe brefily. Here is my blueprint implmentation screenshot.

Is it possible to use this with Audio Capture for live input?

Not an issue, just wondering if this setup is supported?

Instead of:

Import sound wave -> Play sound wave -> Analyze audio data as the sound wave plays

Do:

Audio Capture -> Analyze audio data as the sound wave comes in

Implement frequency spectrum calculation

im wondering because im interested in making a spectral analyzer in blueprints.
I know blueprints are slower, but i could probably make a spectral analyzer in a few days in blueprints rather than a few months in C++. i also intend to multi thread this, so even if blueprints are slower it shouldn't be as much of an issue.
while i do intend to learn C++ in the future, right not im a bit crunched for time and need to get my current project finished in the next month or so.
in addition, once im done my project, i fully intend on making a video tutorial showing the coding process from scratch. there are very little FFT resources/tutorials online, so i think this could be very helpful.

Beat position calculation

Is there a way to work out the time position of beats in a song?

I'm trying to generate a map of the beats in a given song at runtime, to use as events, but I'm finding "IsBeat" functions more like an audio visualiser.
For example, given a simple beat like https://www.youtube.com/watch?v=HTmKgbT3PFA, an incremented int reaches 120 long before a minute passes, and watching the printed strings, it looks like it'd work really well as a visualiser.

Is there a function to work out slightly more precisely where a beat is?
Something like https://github.com/aubio/aubio/blob/master/src/tempo/tempo.c ?

Definitely going to use your plugins, thanks for your work! 😄

How to use the plugin?

I am kind of new to unreal, and I tried to follow the document and is confused. In the second step of "How to use the plugin" section, it told us to bind to "On Generate PCM data", there there's no such thing in my editor. A lot of funtions seems to be missing, not pretty sure something went wrong like I am using the wrong version of Unreal or the document is not updated so the name is wrong?

FFTSubbandsSize cannot be set higher than 512 or beat detection fails

If FFTSubbandsSize is set higher than 512, beat detection fails every frame.

FFT Alloc typo?

Is the second FFTStateStruct not supposed to be FFTComplexSamples at
https://github.com/gtreshchev/AudioAnalysisTools/blob/main/Source/AudioAnalysisTools/Private/Analyzers/FFTAnalyzer.cpp#L379 ?

Judging from https://github.com/mborgerding/kissfft/blob/master/kiss_fft.c#L342 at least.

Detect Build up

Hay bro..... this isn't actually an issue but, is there any way to detect building up of high beat in music using your plugin. I tried to figure out how for 3 days but no luck. Also ur plugin can return whether or not the audio is beating for the specified time(beat detection). how to get that beat frequency in addition to the bool return at that time.

Thanks very much for this free plugin 🙏

Build error With audio analysis tools

Hello bro.....when i try to build my project that contains audio analysis tools plugin , it returns this error

UATHelper: Packaging (Windows (64-bit)):   c:\Program Files\Epic Games\UE_4.27\Engine\Plugins\Marketplace\RuntimeAudioImporter\Source\RuntimeAudioImporter\RuntimeAudioImporter.Build.cs(7,14) : error CS0101: The namespace '<global namespace>' already contains a definition for 'RuntimeAudioImporter'
UATHelper: Packaging (Windows (64-bit)):   c:\Program Files\Epic Games\UE_4.27\Engine\Plugins\Marketplace\RuntimeAudioImporter\Source\RuntimeAudioImporterEditor\RuntimeAudioImporterEditor.Build.cs(5,14) : error CS0101: The namespace '<global namespace>' already contains a definition for 'RuntimeAudioImporterEditor'

Exception if you exit whilst AudioAnalysisTools calculation in progress

This isn't normally an issue, but I'm throwing all the data at the frame process at the moment.

If you exit whilst the processing is in progress, you get this exception:

Unhandled Exception: EXCEPTION_ACCESS_VIOLATION reading address 0x0000025b4b06e908

UnrealEditor_AudioAnalysisTools!CalculateButterfly_Generic() [\Plugins\AudioAnalysisTools\Source\AudioAnalysisTools\Private\Analyzers\FFTAnalyzer.cpp:247]
UnrealEditor_AudioAnalysisTools!DoWork() [\Plugins\AudioAnalysisTools\Source\AudioAnalysisTools\Private\Analyzers\FFTAnalyzer.cpp:316]
UnrealEditor_AudioAnalysisTools!DoWork() [\Plugins\AudioAnalysisTools\Source\AudioAnalysisTools\Private\Analyzers\FFTAnalyzer.cpp:291]
UnrealEditor_AudioAnalysisTools!DoWork() [\Plugins\AudioAnalysisTools\Source\AudioAnalysisTools\Private\Analyzers\FFTAnalyzer.cpp:291]
UnrealEditor_AudioAnalysisTools!DoWork() [\Plugins\AudioAnalysisTools\Source\AudioAnalysisTools\Private\Analyzers\FFTAnalyzer.cpp:291]
UnrealEditor_AudioAnalysisTools!DoWork() [\Plugins\AudioAnalysisTools\Source\AudioAnalysisTools\Private\Analyzers\FFTAnalyzer.cpp:291]
UnrealEditor_AudioAnalysisTools!DoWork() [\Plugins\AudioAnalysisTools\Source\AudioAnalysisTools\Private\Analyzers\FFTAnalyzer.cpp:291]
UnrealEditor_AudioAnalysisTools!DoWork() [\Plugins\AudioAnalysisTools\Source\AudioAnalysisTools\Private\Analyzers\FFTAnalyzer.cpp:291]
UnrealEditor_AudioAnalysisTools!DoWork() [\Plugins\AudioAnalysisTools\Source\AudioAnalysisTools\Private\Analyzers\FFTAnalyzer.cpp:291]
UnrealEditor_AudioAnalysisTools!<lambda_bc65446df753adec24c1f5a4d0787a41>::operator()() [\Plugins\AudioAnalysisTools\Source\AudioAnalysisTools\Private\Analyzers\FFTAnalyzer.cpp:272]
UnrealEditor_AudioAnalysisTools!ParallelForImpl::NewParallelForInternal<TFunctionRef<void __cdecl(int)>,<lambda_06f0d5d1c5fc9bccb0e7d1fd1e70ca42>,std::nullptr_t>'::2'::FParallelExecutor::operator()() [C:\Program Files\Epic Games\UE_5.0\Engine\Source\Runtime\Core\Public\Async\ParallelFor.h:566]
UnrealEditor_AudioAnalysisTools!LowLevelTasks::TTaskDelegate<void __cdecl(bool),48>::TTaskDelegateImpl<<lambda_1bf4a01354d569cb2631b3617c09e35e>,0>::CallAndMove() [C:\Program Files\Epic Games\UE_5.0\Engine\Source\Runtime\Core\Public\Async\Fundamental\TaskDelegate.h:171]
UnrealEditor_Core
UnrealEditor_Core
UnrealEditor_Core
UnrealEditor_Core
UnrealEditor_Core
UnrealEditor_Core
UnrealEditor_Core
kernel32

The fix is to check whether the engine is still running, I had the same issue with a plugin I made, I tried a number of things, checking to see if GEngine was valid, whether I could get a valid world pointer, the only thing that reliably worked was a bool, that would be checked before taking action, and set when the destructor was called, for example.
I had to use this for the editor preview, otherwise it skipped my destructor:

#if WITH_EDITOR
void OnPreEndPIE(const bool bWasSimulatingInEditor) { // Do your thing here }
#endif

If you have/find a better solution, I'd love to know, my bool workaround feels very "hacky"!

In-Engine FFT tools

There's a native plugin for KISS FFT that you might find interesting, and a number of built in analysers, which might be of interest.

Are these of any use? Personally, I'm interested in the FFTPeakPitchDetector, which can take a float buffer, and produce timestamps based on pitch ranges, I'm going to have to take a close look at that.