andimarafioti / audiocontextencoder Goto Github PK
View Code? Open in Web Editor NEWA context encoder for audio inpainting
A context encoder for audio inpainting
I secretly trained this network over night since we all wanted to (didn't we?). It takes the STFTs from the sides without gap information and produces the mag stft for the gap with sides information. Some curves on the l2Loss of the trainable variables and the reconstruction loss (gray is this net, orange is the one we had going from mag to mag, please take into account that one produced only 7 frames so reconstruction loss is not scaled)
And for the SNRs (remember we had some weird SNRs on that one, I think they were still being computed on 1D):
Using the original phase we get SNRs of 21.3 dBs. I'm trying to see what I can do with Griffin Lim but the word around the institute is it produces bad SNRs.
Use normalization instead of strange l2 loss?
The full dataset pipeline should be organised in a user friendly way.
The user want to do
./make_nynthdataset.py /path/fileout
and everything should be done automatically. (download, extract, create tfRecord,...)
logs_path = '../logdir_real_cae/' + self._name # write each run to a diff folder.
Instead of using the SNR of the reconstructed 1D signal, take SNR of spectograms. This should be pretty straight forward taking the reduce_sum on the euclidean norm in the frequency and time axis.
I start to have trouble understanding your code. It is very hard to follow how the data flow in your code. Do you see what I mean?
Also, this cannot be more confusing:
signal = aModel.output()
with tf.name_scope('Energy_Spectogram'):
fft_frame_length = 512
fft_frame_step = 128
stft = tf.contrib.signal.stft(signals=signal, frame_length=fft_frame_length, frame_step=fft_frame_step)
sides_stft = tf.stack((stft[:, :15, :], stft[:, 15+7:, :]), axis=3)
mag_stft = tf.abs(sides_stft) # (256, 15, 257, 2)
aModel.setOutputTo(mag_stft)
I would be cool if you could re-organise your code in a way that I (or other people) can more easily grap what you are doing.
I do not think it is a major change, but maybe a clever re-oganisation.
What do you think?
excuse me. i found so many files in you project, but the readme.md is not clear, you said: "To train the network, execute in the parent folder python paperArchitecture.py", i even can not find it . please tell me more information about your training process. Thank you so much.
Nicki wants me to try to use the magnitude network with the original phase from the signal to see the result
Save the mean and std of the individual signals, correct for that before inputing and re apply it before computing the loss. It's a hand-crafted skip connection. Should be applied on both dims simultaneously for STFTs.
I ran this over the weekend. Because the target of this network has 11 windows instead of the 7 from the last experiments, the reconstruction losses are not directly comparable. But here they are anyway (red is this new one):
The SNRs should be comparable:
After reconstruction though, we find that exclusively for the gap, the SNR is ...
1 quarter of the dataset, STFT with a redundancy of 4
I added a dropout feature to the sequential model. Preliminary tests on it are a bit hard to asses.
I trained two equivalents networks for 800k steps with a learning rate of 1e-3. In orange there's a network with dropout = 0.3 for the linear layer and 0.1 for all conv and deconv layers except the last deconv. In blue is the same network without any dropout.
I think the sudden change in the orange one in the training SNR comes when I restarted the training with dropout = 0.3 for the linear layer (before it was 0.5, I'm not really sure)
It seems to work well since the performance on the validation test is better with dropout and worse on the training set.
What do you think? Should I run more tests? Are this parameters good for you? (30% on the linear layer and 10% on convs)
I also tried the same net w/only dropout=50% on convs (blue):
I get the following error when trying to unpickle ''magnitude_network_parameter.pkl"
Traceback (most recent call last):
File "/home/akclee/miniconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3319, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 2, in
data = pickle.load(f)
File "/home/akclee/.pycharm_helpers/pydev/_pydev_bundle/pydev_import_hook.py", line 21, in do_import
module = self._system_import(name, *args, **kwargs)
ModuleNotFoundError: No module named 'architecture'
Everything is in the title...
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.