dtaoo/vggish

An implementation of vggish in keras with tf backend

vggish's Issues

ckpt to h5

@DTaoo The original checkpoint file is provided here. How did you generate a HD5 file without retraining the model ?

No 'sound' module

Hi, I run the vggish.py and got the ModuleNotFoundError like this:
ModuleNotFoundError: No module named 'sound'

and I did not find where is the sound module.
Did I miss something?

Could you please provide the details of the pre-trained models?

Did you train the models on the AudioSet for AED? How did you get the raw audio dataset?
Could you explain how to train the model without the fc layers?
Thanks!

TXT Files for testing

Could you please load the txt files

Weight import error for top layers

When using include_top=True, load_weights=True and your weight file, i get the following error:

ValueError: Dimension 0 in both shapes must be equal, but are 63488 and 12288. Shapes are [63488,4096] and [12288,4096]. for 'Assign_163' (op: 'Assign') with input shapes: [63488,4096], [12288,4096].

Loading weights without top layers seems to work fine.

Any ideas?

Add keras model module to VGGish project

This repo is amazing, I mean, VGG-ish is a powerfull model, but tensorflow is a little bit complicated. However, Keras is far more easier to learn. Now a days keras is part of tensorflow itself, so here goes my question/suggestion:

have you considered adding this particular module to the original VGGish project?

For me, it looks like a great idea to keep it all together, I'd definitely do it, but perhaps you already tried it

Thanks

unable to read .h5 files of weights

The weight files provided in the link, pose error when read or when loaded in the model directly.
My code:
import h5py
filename_wt='vggish_audioset_weights.h5'
f = h5py.File(filename_wt, 'r')

ERROR:
OSError: Unable to open file (unable to open file: name = 'vggish_audioset_weights.h5', errno = 22, error message = 'Invalid argument', flags = 0, o_flags = 0)

Can you share the train.txt file with us?

evaluation.py VGGish is not being fitted

Hello

I've been reading your evaluation.py example, and I can't understand why weights aren't loaded when the VGGish model is made, nor it's fit after the model is made.

As I see it, those lines:

sound_model = VGGish(include_top=False, load_weights=False)

should be:

sound_model = VGGish(include_top=False, load_weights=True)

or after that, it should be fitted before used (which currently isn't).

With the current implementation (i.e. load_weights=False), I run it on a subset of Speech Commands dataset, and I get the following results:

Report for testing
              precision    recall  f1-score   support

          no       0.90      0.90      0.90       386
         yes       0.90      0.90      0.90       397

    accuracy                           0.90       783
   macro avg       0.90      0.90      0.90       783
weighted avg       0.90      0.90      0.90       783

however, only switching load_weights=True, I get those:

Report for testing
              precision    recall  f1-score   support

          no       0.92      0.98      0.95       386
         yes       0.98      0.92      0.95       397

    accuracy                           0.95       783
   macro avg       0.95      0.95      0.95       783
weighted avg       0.95      0.95      0.95       783

As I see it, if the networks are not initialized with given weights, nor fitted, then its weights are just random initializations and therefore, the features computed by the model are just random computations on the input and therefore they don add too much value to the model itself.

I would love to hear your thoughts on this

error raised when loading model_weights "include_top=True"

here is the code
model = VGGish(include_top=True)
model.load_weights(WEIGHTS_PATH_TOP)

here is the bug
ValueError: Dimension 0 in both shapes must be equal, but are 63488 and 12288. Shapes are [63488,4096] and [12288,4096]. for 'Assign_12' (op: 'Assign') with input shapes: [63488,4096], [12288,4096].

what's wrong?