Comments (3)
I think I see the issue. In ast_models.py
, in the pretrain_stage==True
loop, we compute
self.p_f_dim, self.p_t_dim = self.get_shape(fstride, tstride, input_fdim, input_tdim, fshape, tshape)
num_patches = self.p_f_dim * self.p_t_dim
self.num_patches = num_patches
which results in self.num_patches=24
, which is then passed to gen_maskid_patch()
as sequence_len
. Because mask_size=400
in run.sh
, the following loop is infinite:
while len(list(set(mask_id))) <= mask_size:
start_id = randrange(sequence_len)
# this improves the efficiency, but might change the pretrained model
# while start_id in mask_id:
# start_id = randrange(sequence_len)
cur_mask = []
for i in range(0, cur_clus):
for j in range(0, cur_clus):
mask_cand = start_id + self.p_t_dim * i + j
if mask_cand > 0 and mask_cand < sequence_len:
cur_mask.append(mask_cand)
mask_id = mask_id + cur_mask
So, is the solution to simply reduce mask_patch
in run.sh
file? What exactly are the consequences in reducing the mask_patch
to (for example) 10?
from ssast.
please check the paper. The masked patch should be something 50%-80% of the total #patch.
You are running it on CPU (Intel UHD and AMD GPUs do not support cuda). So it will be slow anyways. Maybe training on colab (GPU) is a better idea, even with the free plan.
-Yuan
from ssast.
This answers my question; thank you so much for your help, Yuan! I suggest mentioning that mask_patch
should also be modified in the src/pretrain/run_mask_{frame,patch}.sh
files. At the moment, in your "Pretrain on custom dataset" section, you do not include this as a parameter to be modified. Additionally, we would need to modify the --n_class
and --label-csv
arguments in the terminal command.
from ssast.
Related Issues (20)
- Bug with input_dim and pure inference HOT 11
- Stereo audio HOT 1
- Model fails to converge on transfer to audio backtesting problem HOT 5
- speech commands v1 HOT 3
- Extract audio representations for future use HOT 13
- sed HOT 1
- python and cuda version HOT 3
- Host on Huggingface HOT 2
- Which python version are you using? HOT 1
- For discriminative loss, is the true NCE batch size the number of masked patches? HOT 2
- Librispeech960 Pretrained Model HOT 3
- AttributeError: 'ASTModel' object has no attribute 'unfold' HOT 8
- Are the model weights loaded into the ASTModel with stochasticity? HOT 2
- Data Loading Logs HOT 2
- performance and loss of the frame-based model HOT 2
- audio file length HOT 4
- SSAST for embedding model
- question: example usage for inference, rather than training HOT 1
- Use SSAST pretrained model to inference
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ssast.