VAME question

[lincellularneuroscience]

VAME question

[kvnlxm]

Hi Mohammad,
thank you for posting your issue here. I'm wondering on how many data points you trained and tested VAME? Moreover, what is the frame rate in your experiment? As VAME is a probabilistic machine learning model it needs quite a lot data to learn the underlying distribution of the data.

To check if VAME works well on your machine can you test your installation with the demo data set we provided and see if it reconstructs the signal after about 200 Epochs. Note that you have to change the num_features back to 12

If this works for you, I would suggest to increase the hidden_size_layer_1 to 512 and to change the kmeans_lambda parameter to 0.1 and see if this works better for you as you have 18 input signals instead of 12

I just tested VAME on my machine (Ubuntu 18.04, GPU: Nvidia GTX 1080 ti) with a small data set size and I got a reconstruction of the input signal. The reconstruction and loss figures are attached.

Best,
Kevin

[MHYaghoubi]

Hi Kevin
Thanks a lot for the detailed explanation. I trained the code on about 25,000 frames, with a data acquisition rate of 30 Hz. I am hoping to re-run this code on a 3 or 4 times larger data set soon.

I tried the code on the demo that you have provided and I got the results that are attached. It works better than on my data but still, the reconstruction doesn't capture the fast changes in the traces.

I also changed the parameter as you mentioned and re-ran the code on my data set but unfortunately, it didn't help much. I hope it could improve as I increase the size of the test data set.

Thanks
Mohammad

[kvnlxm]

Hey,

so this looks already better then before. And knowing your frame rate is 30 Hz I have some suggestions which hopefully help:
time_window: 15 (which represents half a second of data and worked well for us)
prediction_steps: 5
kmeans_lambda: 0.1 or 0.001 (because this loss term seems to be very high compared to the KL-Loss)
hidden_size_layer_1: 256 (going back to default)

Moreover, as long training times are crucial for RNNs I also would suggest to give it at least 500 Epochs or even more. We trained our RNN on over a million datapoints for about 24 hours. Just to give you an idea.

Best,
Kevin

PS: If this still doesn't seem to fit with your data you could share some of it (e.g. google colab) and I take a look myself in order to make it work for you :)

[MHYaghoubi]

Hi Kevin

Thanks for the points in your response. It helped a bit. Still my data is 10 times smaller than yours. So hopefully I could get better results with more data.

I have another question about the final results.
The length of the numpy file that saves the results for each video (the cluster ID of each frame, if I'm not wrong) at the very end, doesn't match with the length of the video itself. could you pleas elaborate on this?

Thanks,

[kvnlxm]

Hi Mohammad,

I think your question is related to Issue #12 . I just fixed the vame.behavior_segmentation(config, model_name='VAME', cluster_method='kmeans', n_cluster=[30]) function and it should now store the labels for each individual video.

[MHYaghoubi]

Hi Kevin

I increased my data set to 70 sessions of recording, each session 5 minutes long, and also I increased the frame rate from 30 to 60 Hz, simply by linearly interpolating between frames. Now my data is comparable to what you have, about 1000,000 frames.

But still when I train the network by 500 Epochs which tool about 10-15 hours on GeForce RTX 2080 Ti, the reconstructed and main trajectories are very different. please see the attachment.

I should mention that the latent space captures a few behavioral properties (I will email you a video showing this). But still, I am not satisfied with reconstruction quality.

[kvnlxm]

Hi Mohammad,

I looked into the data you sent me. The first thing I noticed was a big spike in the dlc labels right at the beginning of each data set, which resulted in a training/test data set with some anomalies:

Closer look at one spike:

Having these spikes/anomalies in your data will cause the model to try to capture this data as well, and as their magnitude is a lot higher than the signal you are interestet in, you get some poor reconstruction results. I cleaned the data you gave me and achieved actually some nice reconstruction:

Moreover, I segmented your data and found a cluster size of 8 to be a good fit. The videos are looking quite nice and the UMAP embedding also shows a good seperation of the data:

If you go ahead and clean your data in this way, you should be able to have better results. Make sure that your behavior videos are also starting with the same frame as your cleaned and aligned data

Best,
Kevin