hyakuchiki/realtimeddsp

Realtime (streaming) DDSP in PyTorch compatible with neutone

realtimeddsp's Issues

Is the ddsp implementation different from the original tensorflow ddsp?

I tried to test ddsp of this project and ddsp of tensorflow with the same input, got different output, especially filtered_noise_synth.Below is my code.

    n_frames = 1000
    hop_size = 64
    n_samples = n_frames * hop_size
    sample_rate = 16000
    # Amplitude [batch, n_frames, 1].
    # Make amplitude linearly decay over time.
    amps = np.linspace(1.0, -3.0, n_frames)
    amps = amps[np.newaxis, :, np.newaxis]
    # Harmonic Distribution [batch, n_frames, n_harmonics].
    # Make harmonics decrease linearly with frequency.
    n_harmonics = 20
    harmonic_distribution = np.ones([n_frames, 1]) * np.linspace(1.0, -1.0, n_harmonics)[np.newaxis, :]
    harmonic_distribution = harmonic_distribution[np.newaxis, :, :]
    # Fundamental frequency in Hz [batch, n_frames, 1].
    f0_hz = 440.0 * np.ones([1, n_frames, 1])
    n_frames = 1000
    hop_size = 64
    n_samples = n_frames * hop_size
    sample_rate = 16000
    ee = Additive(n_samples=n_samples,sample_rate=sample_rate)
    amps = torch.FloatTensor(amps)
    harmonic_distribution = torch.tensor(harmonic_distribution)
    f0_hz = torch.tensor(f0_hz)
    audio = ee.forward(amps, harmonic_distribution, f0_hz)
    sf.write('tes_ddsp_torch_code2.wav', audio.cpu().detach().numpy()[0], 16000)
    n_frames = 250
    n_frequencies = 1000
    n_samples = 64000
    # Bandpass filters, [n_batch, n_frames, n_frequencies].
    magnitudes = [torch.sin(torch.linspace(0.0, w, n_frequencies)) for w in np.linspace(8.0, 80.0, n_frames)]
    magnitudes = 0.5 * torch.stack(magnitudes)**4.0
    magnitudes = magnitudes[None, :, :]
    filtered_noise_synth = FilteredNoise(n_samples=n_samples,
                                                    scale_fn=None)
    plt_fig(magnitudes.detach().numpy()[0])
    # Generate some audio.
    audio = filtered_noise_synth(magnitudes)
    sf.write('tes_ddsp_noise.wav', audio.numpy()[0], 16000)

Error in call to target and Exception: No Data Found

Hello,

I want to use custom timbres for your Neutone plugin, but using the collab notebooks online are timing out and are expensive.
For these reasons, I wanted to use your files directly from this repository.
However, I am getting errors as shown below and with the complete stack trace.
I have tried to use both .mp3 files as you had in your data folder as well as .wav files, all mono, 44.1khz, and 16bit.
Please help me to fix this issue so I can keep developing models for your plugin.

Thank you!

(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>python train.py name=Alto_Sax data.raw_dir='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/data/413204-mono.mp3' data.db_path='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/temp_db_dir'
[2023-02-18 10:06:22,729][pytorch_lightning.utilities.seed][INFO] - Global seed set to 0
Starting Preprocessing.
0it [00:00, ?it/s]
Error executing job with overrides: ['name=Alto_Sax', "data.raw_dir='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/data/413204-mono.mp3'", "data.db_path='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/temp_db_dir'"]
Error in call to target 'diffsynth.data.SliceDataset':
Exception('No data found @ C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/data/413204-mono.mp3')
full_key: data

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>python train.py name=Alto_Sax data.raw_dir='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/data/413204-mono.mp3' data.db_path='C:/Users/marlo/anaconda3/envs/RealtimeAudioDDSPv2/realtimeDDSP-master/temp_db_dir'
Traceback (most recent call last):
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 213, in run_and_report
return func()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\utils.py", line 453, in
lambda: hydra.run(
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\hydra.py", line 105, in run
cfg = self.compose_config(
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\hydra.py", line 594, in compose_config
cfg = self.config_loader.load_configuration(
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\config_loader_impl.py", line 141, in load_configuration
return self._load_configuration_impl(
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\config_loader_impl.py", line 235, in _load_configuration_impl
self.process_config_searchpath(config_name, parsed_overrides, caching_repo)
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\config_loader_impl.py", line 158, in process_config_searchpath
loaded = repo.load_config(config_path=config_name)
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\config_repository.py", line 349, in load_config
ret = self.delegate.load_config(config_path=config_path)
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\config_repository.py", line 92, in load_config
ret = source.load_config(config_path=config_path)
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\hydra_internal\core_plugins\file_config_source.py", line 31, in load_config
cfg = OmegaConf.load(f)
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\omegaconf\omegaconf.py", line 192, in load
obj = yaml.load(file, Loader=get_yaml_loader())
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml_init.py", line 81, in load
return loader.get_single_data()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\constructor.py", line 49, in get_single_data
node = self.get_single_node()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\composer.py", line 36, in get_single_node
document = self.compose_document()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\composer.py", line 58, in compose_document
self.get_event()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\parser.py", line 118, in get_event
self.current_event = self.state()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\parser.py", line 193, in parse_document_end
token = self.peek_token()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\scanner.py", line 129, in peek_token
self.fetch_more_tokens()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\scanner.py", line 223, in fetch_more_tokens
return self.fetch_value()
File "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\lib\site-packages\yaml\scanner.py", line 577, in fetch_value
raise ScannerError(None, None,
yaml.scanner.ScannerError: mapping values are not allowed here
in "C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master\configs\config.yaml", line 3, column 9

(RealtimeAudioDDSPv2) C:\Users\marlo\anaconda3\envs\RealtimeAudioDDSPv2\realtimeDDSP-master>