Run real time on less powerful devices?

[cTatu]

Run real time on less powerful devices?

[vb000]

Hi,

It's required only for the training. If you do not wish to load spynet weights before inference, you change these two lines:

NeuriCam/model/keyvsrc/net.py

Lines 34 to 35 in 976a9a2

	spynet_pretrained=os.path.join(os.path.dirname(__file__),
	'spynet_20210409-c6c1bd09.pth'),

to

 spynet_pretrained=None, 

[vb000]

Hi @cTatu,

There are a few options that could be explored to run it in realtime, on a smaller form factor device:

• Reducing the channel count from 64 to 48, 32 etc.
• Using only 2-levels in the grid propagation instead of 4. With only 2-levels, the model could be considered as a bidirectional RNN for the most part, except for the cross connections between levels.
• And of course, quantization (with/without retraining).

Accuracy trade-off required for these options has to be empirically evaluated. Hope this helps!

[cTatu]

Yes very helpful thank you very much!

[cTatu]

Hello again,

I have a question about spynet. It is necessary for inference as well? or just for training?

Thank you