sg-s/crabsort

🦀🦀🦀 Sort spikes from extra-cellular recordings using neural networks. Fully automated.

🦀 crabsort

Installation

Using git

Clone these repos:

# bash
git clone https://github.com/sg-s/crabsort
git clone https://github.com/sg-s/puppeteer
git clone https://github.com/sg-s/srinivas.gs_mtools

and add the all to your MATLAB path.

Updating and uninstalling

crabsort supports built-in methods to upgrade and update:

% matlab
crabsort.update
crabsort.uninstall

Usage

Video tutorial by Mara Rue walking through how to use crabsort

Keyboard actions

Key	Action
a	Scroll to beginning of file
z	Scroll to end of file
Spacebar	Jump to next uncertain spike (as predicted by Neural Network)
g	generate data for Neural network
⇧ + ↑	jump to the weirdest spike
⇧ + ↓	jump to a next less weird spike
p	Predict spikes using Neural network
r	reset zoom
0	Set channel as having no spikes
↑	Select channel above currently chosen channel
↓	Select channel below currently chosen channel
→	Load next file in dataset
←	Load next file in dataset
p	Predict spikes using Neural network
⇧ + →	jump to the next file with unsorted data on this channel

License

GPL v3

If you plan to use crabsort for a publication, please write to me for appropriate citation.

crabsort's Issues

redo button should be moved to a menu item

it's too easy to click right now

training the neural network should stop when accuracy > .999

it's silly otherwise

scrolling is a little borked

as it's possible to scroll past the end of the data stream

crabsort should allow multi-step elimination and classification

this way, different combinations of methods can be used to weed out spurious spikes

crabsort should refuse to create a new window if there is already one existing.

saving and loading files are too slow

i think we can optimizing saving to .crabsort files -- using savefast?

automation should allow for multiple passes through the data

with different settings -- so first we pass everything through, then use time...

but why? this can be done in one step

consider moving the spike distance setting into a GUI element

and out of the pref file.

need to allow channels to be hidden

nerve names need to be sorted in dropdown

as soon as spikes are found and classified, the manual override should be made visible

so turn it on after clusterCallback and after predict

change hero image to something less likely to trigger an epileptic attack

"watch me" should automatically turn on and off

turn on when switching to a channel with no automate_info and no spikes

turn off when switching to a channel with automate_info

-ve training data generation should happen in two steps:

1. "spike-like" things that aren't spikes -- find these by using findSpikes with a more generous tolerance
2. random noise -- randomly pull out snippets from the trace.

there is a bug in conversion of ms -> timestep

in before and after preference settings

need to abstract away ABF-specific things

right now, we assume a very specific structure for self.metadata that is true only for ABF files

closing main window should close tensorflow window

need a new drop down menu item on automate

that prints all the automate info in a human-friendly fashion

use uimenu's Check property to hide and show some channels

though rescaling the remaining channels is going to be painful-- we will have to do a complete redraw on the figure

all text strings should respect the font size pref

colours for 2 units on the same channel can sometimes be very similar

crabsort should make sense of intracellular recordings

• should show the 0 mV line
• use a fixed y-lim?

add a new X axis at the top of the first plot

with units in ms

there should exist a method to check that everything is sorted

-- checks that every named channel in every file contains at least one spike

training the neural network should be much more general

Given some time series with labelled spikes,

1. remember the spike times and labels
2. keep dropping the spike prominence detection till we double the # of spikes
3. we have now generated our negative training dataset

open problems:

1. how do we know whether to use + or - prom?

write a method to check all files in a folder

and see if they load.

this simply loops over all files, and checks them. if they don't load, it is marked as such

XTicks for all inner plots should be turned off

otherwise they appear as ghostly white apparations

since tf_model_dir is hardcoded

as a full path, moving the directory, or transferring to a different computer breaks everything.

maybe use relative paths?

automate should use tensorflow when available

this should happen right after the findSpikes step

things that are common to the entire dataset

and not a particular file should be stored elsewhere

like

1. channel names
2. tf_data
3. tf_model_names
4. automate_info

training should show PCA with labels

use a different marker for different units, and paint them in green when correctly classified, and in red when incorrectly classified. maybe also show the training data in black?

should be able to load .smr files

tensorflow should not show channels that have no automate_info

image thread

crabsort should play more nicely with giant files

if you encounter a 1GB file, it's silly to load it into memory. maybe first convert it to a non-ABF format, then load it piecemeal?

if crabsort opens a file in a read-only directory, it throws an error

and refuses to close.

need a way to stop the automation once it is started

crabsort should respect pref.m

Right now, it's totally ignored.

need the ability to mark some sections as bad

for example, when an electrode pops out, or if there is noise

when automate is running, all controls should be disabled

need a new set of control to control automation

loadData in tensorflow window does not load data from all files

findSpikes can be much faster

and not slow to a crawl on large files

-- only find spikes in the visible trace. we can find all the spikes later

new neural network interface

should have:

1. control for n_steps
2. control for model_name
3. train button
4. done button
5. visual display of accuracy over time.

annotate_info should refer to popupmenu by name, not value

because the items in that list can change over time, their order can change, etc

menu handles should be stored in a vector

the spike prominence slider can get "stuck"

when moved to low values, and millions of spikes are found.

save data in a nicer way

empty .crabsort files are too big

need to allow removing spikes manually

removing spikes from modify should add this

as -ve training data to automate_info