nibnu/wavcraft

Official repo for WavCraft, an AI agent for audio creation and editing

🌊WavCraft

Generate and edit the audio with a simple sentence.

This repo currently support:

• text-guided audio editing: edit the content of given audio clip(s) conditioned on text input
• text-guided audio generation: create an audio clip given text input
• audio scriptwriting: get more inspiration from WavCraft by prompting a script setting and let the model do the scriptwriting and create the sound for you.
• check if your audio file is synthesized by WavCraft

Content

• Usage
  • Installation
  • Audio edition using a single line
  • Audio edition via interaction
  • Check if an audio file is generated/modified by WavCraft
• Approach
• Acknowledgments
• Citing

Usage

Installation

source scripts/setup_envs.sh

Configure environment

export OPENAI_KEY=YOUR_OPENAI_KEY
export HF_KEY=YOUR_HF_KEY

Audio edition using a single line

python3 WavCraft.py basic -f \
--input-wav assets/duck_quacking_in_water.wav \
--input-text "Add dog barking."

Audio edition via interaction

python3 WavCraft-chat.py basic -f -c
[New session is create]
Add audio files(s) (each file starts with '+'): +assets/duck_quacking_in_water.wav
Enter your instruction (input `EXIT` to exit the process): "Add dog barking"

Check if an audio file is generated/modified by WavCraft

python3 check_watermark.py --wav-path /path/to/audio/file

Approach

WavCraft is an LLM-driven agent for audio content creation and editing. It applies LLM to connect various audio expert models and DSP function together. An overview of WavCraft architecture can be found bellow:

Disclaimer

This repository is for research purpose only. We are not responsible for audio generated/edited using semantics created by this model. Also, everyone use WavCraft must NOT disable the watermarking techniques in anyway.

Acknowledgments

We appreciate WavJourney, AudioCraft, AudioSep, AudioSR, AudioLDM, WavMark for their amazing code work.

Citing

If you found our work is helpful, please cite our work: