kushalkolar/caiman-napari-prototype

prototype caiman plugin for napari

caiman-napari-prototype's Issues

Ideas

Basic design

Using a batch manager would probably be the most useful way to implement this. Mesmerize users like it a lot because it saves them time and helps with parameter optimization.

-> See if anyone has already made a batch/process manager for napari

pandas extensions should make it very easy to create a batch processor!

Implementation

• Use the hook napari_experimental_provide_dock_widget() to wrap the GUIs for the various caiman functionalities. This allows us to create dock widgets that can interact with the napari viewer. It's very similar to how dock widgets & plugins interact with the mesmerize viewer, so we can probably just take the ui files and param creation functions from mesmerize itself and put the various caiman modules (NoRMCorr, CNMF etc.) each in tab widgets or similar.
• See if it's possible to use napari.qt.threading.thread_worker, else use QProcess like mesmerize to run all the dview stuff inside.
  -> Update, @thread_worker doesn't work, napari just hangs. QProcess works well, probably a better solution anwyways

Usage

1. User must open the movie using our plugin. With the napari API, there is no way to get the originating file path for any image(s) that are currently open in the viewer. For now support the same image filetypes that are supported by caiman. Allow the user to view these movies using lazy methods such as zarr.
   -> After we get basic functionality working, allow the user to open any arbitrary format that napari supports. In our plugin the user can specify the plugin to use to open the image. See plugin kwarg in napari.Viewer.open().
   -> Maybe add an attribute like origin_path to napari.layers.Image, might be useful for other napari devs.

2. User sets params for NoRMCorr/CNMF(E) in our plugin, adds this to a batch, repeats for all movies.

3. When the batch runs, the filepath is used to get each image and memmap files are created.

4. Similar to mesmerize, the user clicks on a batch item to view its output

5. If the output is a motion corrected video, our plugin loads the memmap file for visualizing the movie in the viewer.

6. If the user wants to perform CNMF(E) on the motion corrected video, the same memmap file is used. Else a memmap file is created.

7. For CNMF(E) outputs, napari.layers.Shapes is used to draw the ROIs. Rejected & accepted components in two separate layers.
   -> Use pyqtgraph to show the traces? napari doesn't have a way to plot timeseries data.

I'm not sure of how to

1. Make memmap files directly lazy readable with napari, there's a closed issue that mentions it so it has been thought of: napari/napari#103
2. Get meta data (sampling rate) which are required for CNMF(E) etc. regardless of file format? Napari might have a standardized way to get these (should be easy if the input data using OME).

Other thoughts

• Online analysis for very large movies. Visualizing them is very easy with zarr.
• Interactive component evaluation and selection based on quality metrics.
• Session registration
• A way to interact with DataJoint?
  • Pandas extensions for some basic functionalities when data are pulled from DJ?
  • DJ users will probably be able to do any integration by themselves?

License

From the grant's Policies section:

All new code must be released under a permissive open source license (MIT, BSD 2-Clause, BSD 3-Clause, or Apache v2.0). All pre-existing and derivative code must be licensed under the most permissive license possible, given the licensing terms of the pre-existing code.

Caiman is under GPL v2, will this be an issue if the plugin only interfaces with caiman?

Should we go with Apache 2.0?

TODO list: Final prototype

If we can submit the prototype for the grant, I can add these features within a few days:

• Add NoRMCorre with hard-coded params, use QProcess just like for CNMF and put the mcorr video in a different layer within the napari viewer.

• UI entry for for CNMF (can just use the same UI template file that exists in mesmerize).

• Use a tab widget or stack widget where each page shows one of:

  • NoRMCorre with 2 buttons (buttons open movie, button to perform mcorr)
  • CNMF with the UI entry, button to choose movie, the param entry UI and button to perform CNMF
  • QTextBrowser that just shows the stdout from running QProcesses