Comments (5)
@sreenivasaupadhyaya @Mortyzhou-Shef-BIT
Hi there,
I agree SSAST is a cool modeling approach. However, it is not my repo/project and I do not have bandwidth to help you on your use case. Commenting out the lines above as suggested by @YuanGongND worked for my use case. Probably best to close this thread as resolved.
Hugo
from ssast.
I am wondering if modifying these two lines and using finetuningavgtok
as the task
in forward
could help your application (basically comment out the mlp head)?
ssast/src/models/ast_models.py
Lines 262 to 264 in b589c9c
Btw, I would suggest first trying the original inference code to see if the prediction accuracy is as expected and then extract the embedding. This helps avoid mistakes in the inference pipeline (e.g., input normalization, model parallelization, etc).
-Yuan
from ssast.
Perfect, thanks so much.
I ended up defining 'finetuningavgtok_embed' as basically 'finetuningavgtok' but with the last line commented as another task option in ASTModel. Not sure if that will be better or if the one with the last two lines commented, as you suggested, would be better. But regardless, some options for embeddings to test!
Hugo
from ssast.
Hi @HugoBothaMD , I am also interested to use the encoder part of this model for audio event classification task. It would be of great help if you could guide me how i could make use of the model/code in this repo for my application. I would like to have the below setup.
I would like to generate the embeddings for my audio events of 1 second duration.
def get_embeddings_ssast(audio_event):
#define the ssast model
#load pretrained weights
#create a alternate model which returns the embeddings
#generate the embeddings
#return the mebeddings
I would like to use these embeddings as input features in my classifier.
from ssast.
Hi @HugoBothaMD , I am also interested to use the encoder part of this model for audio event classification task. It would be of great help if you could guide me how i could make use of the model/code in this repo for my application. I would like to have the below setup.
I would like to generate the embeddings for my audio events of 1 second duration. def get_embeddings_ssast(audio_event): #define the ssast model #load pretrained weights #create a alternate model which returns the embeddings #generate the embeddings #return the mebeddings
I would like to use these embeddings as input features in my classifier.
Hii, Do you finish it? Thank you
from ssast.
Related Issues (20)
- Bug with input_dim and pure inference HOT 11
- Stereo audio HOT 1
- Model fails to converge on transfer to audio backtesting problem HOT 5
- speech commands v1 HOT 3
- Extract audio representations for future use HOT 13
- sed HOT 1
- python and cuda version HOT 3
- Host on Huggingface HOT 2
- Which python version are you using? HOT 1
- For discriminative loss, is the true NCE batch size the number of masked patches? HOT 2
- Librispeech960 Pretrained Model HOT 3
- AttributeError: 'ASTModel' object has no attribute 'unfold' HOT 8
- I've been stuck on the first epoch in a tiny model for 8+ hours using 300 5s spectrograms for training. Is this normal? HOT 3
- Are the model weights loaded into the ASTModel with stochasticity? HOT 2
- Data Loading Logs HOT 2
- performance and loss of the frame-based model HOT 2
- audio file length HOT 4
- SSAST for embedding model
- question: example usage for inference, rather than training HOT 1
- Use SSAST pretrained model to inference
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ssast.