neuroailab/barrel

For barrel cortex modelling

barrel's Issues

Pre-training networks related

1. Download pre-trained networks for predicting normals.
2. Test them on 3dworld dataset to see the performances.
3. Use pre-trained networks to predict responses of V4 and IT neurons.
   -- Ask Pouya and Jonas for how to do dimension reduction on network outputs.
4. Fine-tuning them for category tasks.

From predicting normals to predicting categories

1. Combining the loss function of category recognition.
2. Train networks predicting category from normals (shallow network, smaller number of examples)
   --Compare this from pixels->labels and pretrained deep neural networks

Network architecture space searching

Formulate large parameter space of models with different bypass settings, conv filters, etc. And search this space.

Possibly use different loss functions: l2_loss, dot product. Search the structure space in every loss function.

Model training for whisker-normalnet

Several proposals about the network structures:

1. concatenate all time sequences at the channel dimension and train the networks as traditional convolutional neural network.
2. a smaller network for every frame of responses and concatenate the output of those small networks for each frames and then train another network from them to our targets。
3. Use LSTM in traditional convolutional neural networks to equip them with dynamics across time dimention or used simple recurrent connections inside the same layer of network conveying information from one frame to another.

Similar hyper-parameters searching for whisker-normalnet as issue #3.

Train existed model in three datasets

After fixing or avoiding the multi-thread bug in tfutils, transfer to tfrecords to train the networks.

Also, use datasets from scannet, scenenet to train the networks. The networks can be trained to predict normals in two dataset and then tested on the third dataset.

Generate dataset

-- Use objects from shapenet
-- Only use half of the whiskers
-- Touch the whiskers from different direction and different surface of the objects.
-- Record the feedback from sensors, maybe just lower units.
-- Total time should be fixed.

Comparising experimental data with network outputs

Parameter optimizing for whiskers

Use hyperopt or other parameter optimizing techniques to optimize hyper-parameters.
-- For every whisker, use a fixed force to push the top to the bottom. Record how close it goes and then let it go. Record how far every dot on that whisker travels before coming back. Also record the recovering time.
-- Test different weights of the three items.