Comments (3)
It is possible to train acoustic models with any kind of input, if you can store the features in Kaldi format.
A CNN acoustic model trained in Keras can be used to extract posterior features by forward pass on the test features (similar to nnet-forward
). For this your network should, after some convolutional layers, convert the 3D signal to 2D and use a Dense
layer with softmax
at the output. Then its outputs can be converted to likelihoods and can be sent to the decoder using latgen-faster-mapped
.
from keras-kaldi.
Just curious, If I have fbank features extracted, and GMM force-alignment trained, and I would like to train CNN on top of it.
Can I simply replace your
m = keras.models.Sequential([
keras.layers.LSTM(256, input_shape=(learning['spliceSize'],trGen.inputFeatDim), activation='tanh', return_sequences=True),
keras.layers.LSTM(256, activation='tanh', return_sequences=True),
keras.layers.LSTM(256, activation='tanh'),
keras.layers.Dense(trGen.outputFeatDim, activation='softmax')])
(in trian*.py)
to something like the following
m = Sequential()
m.add(Convolution2D(150,8,8), input_shape=trGen.inputFeatDim)
m.add(MaxPooling2D(6,6))
m.add(Flatten())
m.add(Dense(1024))
m.add(Activation('relu'))
m.add(Dense(output_dim=treGen.outputFeatDim)
m.add(Activation('softmax'))
and have the rest of the files remained the same?
Is there anything else you would like us to know that requires modification for this pipeline to work? like do I need to modify dataGenerator
?
Thanks in advanced!
from keras-kaldi.
Yes, but I guess Convolution1D
makes sense in the case of filterbank features (because we want each filter of the Kernel to move across time and capture sound patterns by looking at the frequencies, and so we don't want the Kernel to move on the frequency axis). You could try that. The batch_size
could be kept None
, size
could be your context and input_dim
could be the number of filters in the filterbank. And then you can flatten the layer's output and use Dense
layer(s) with a softmax
at the output. You can use dataGenSequences
for this purpose. I haven't tested any code though. I will try to include a CNN example in a later revision.
from keras-kaldi.
Related Issues (15)
- question final.mdl HOT 6
- getBinaryLabels function not defined HOT 1
- Can this code work on a tensorflow trained model HOT 6
- Phone error rates below ~50% HOT 2
- Any pretrained model available? HOT 2
- problem during runing run_kt.sh HOT 2
- Problem with 'dataGenSequences' object has no attribute shape HOT 3
- exp/tri2b Data Processing HOT 2
- ImportError HOT 2
- self.inputFeatDim in steps_kt/dataGenerator.py HOT 10
- samples_per_epoch in steps_kt/train.py HOT 2
- Data generator error when finishing epoch HOT 6
- subprocess error HOT 2
- Training an cnn acoustic model with already trained model in keras HOT 22
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from keras-kaldi.