Comments (3)
How much data shall I use for fine tuning to get decent results to avoid initial over fitting? Does 50 files of 5 sec work in training? what's the general rule when dealing with over fitting in case of transformers? Do we really need more data to fine tune with or is it hyper parameters?
from ssast.
hi there,
Hey I'm curious on why you don't have Librispeech960 Pre-trained on Frame base. I saw you were recommending Frame based models. Do you have Pre-trained Librispeech on Frame?
We do have AudioSet+Librispeech pretrained checkpoint for frame based AST, see https://github.com/YuanGongND/ssast#pretrained-models. One conclusion in our ablation study is that this checkpoint would be better than the model trained solely on Librispeech, even on speech tasks.
Note that for speech tasks, we do not mean ASR, but speech classification, e.g., command recognition, emotion recognition, etc.
How much data shall I use for fine tuning to get decent results to avoid initial over fitting? Does 50 files of 5 sec work in training? what's the general rule when dealing with over fitting in case of transformers? Do we really need more data to fine tune with or is it hyper parameters?
It is hard to estimate as there are many factors (e.g., how many classes, how easy it is to sepearate sounds). You would need to try, but 50 files is a very small number. The smallest dataset we tested is ESC-50 (50 classes, each 40 samples, total 2000 samples).
-Yuan
from ssast.
Hey Thanks Yuan,
Nice answer, I guess I got the idea on what factors I should be looking for when deciding smallest dataset. kudos!
I appreciate your answer to the Librispeech model, I guess I should have framed the question a little different. Anyway, from what I understand Frame-400 trained on both Audioset and Librispeech should perform better than others for Speech classification.
Looking at ablation study in paper and table 2, I can't find whether Libripseech(only) has been trained with patch or frame . From table 5 I can see that Librispeech only has been paired with patch.
It would be nice to the benchmarks for Librispeech only with frames for speech tasks. It's just that I'm unable to find it either on paper or Github readme. Apologies for inconvenience.
Best Regards,
Jaydeep
from ssast.
Related Issues (20)
- Stereo audio HOT 1
- Model fails to converge on transfer to audio backtesting problem HOT 5
- speech commands v1 HOT 3
- Extract audio representations for future use HOT 13
- sed HOT 1
- python and cuda version HOT 3
- Host on Huggingface HOT 2
- Which python version are you using? HOT 1
- For discriminative loss, is the true NCE batch size the number of masked patches? HOT 2
- AttributeError: 'ASTModel' object has no attribute 'unfold' HOT 8
- I've been stuck on the first epoch in a tiny model for 8+ hours using 300 5s spectrograms for training. Is this normal? HOT 3
- Are the model weights loaded into the ASTModel with stochasticity? HOT 2
- Data Loading Logs HOT 2
- performance and loss of the frame-based model HOT 2
- audio file length HOT 4
- SSAST for embedding model
- question: example usage for inference, rather than training HOT 1
- Use SSAST pretrained model to inference
- experiment setup
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ssast.