hoa-2/slicerneurosegmentation

NeuroSegmentation extension for 3D Slicer

slicerneurosegmentation's Issues

Streamline segmentation interface

Slicer currently contains many features that are unnecessary for the segmentation process.

The interface can be simplified by creating a Slicelet

Start with QuickSegment. If additional features are needed, then another Slicelet can be developed using QuckSegment as a starting point.

Slicer segmentation manual: Segmentation_Slicer_v15.pdf

Subtasks:

• Add tag/filtering options to qMRMLSegmentsTableView #9

Add module for importing FreeSurfer data

Models from FreeSurfer exist in a coordinate system that is dependent on the originating volume.
Generally the origin of the model coordinate system exists at the center of the mri volume.

A module to import FreeSurfer data in the correct coordinate system could be developed to allow users to select data from FreeSurfer and import it correctly. Users would choose a folder containing saved freesurfer data, and then select the desired files to import.

Relevant links:
https://surfer.nmr.mgh.harvard.edu/fswiki/CoordinateSystems
http://www.grahamwideman.com/gw/brain/fs/coords/fscoords.htm
http://surfer.nmr.mgh.harvard.edu/fswiki/FsAnat-to-NativeAnat

Segmentation Performance Optimization

When there are hundreds of drawn segments, the performance of various operations (slice browsing, editing, saving/loading, etc) can become increasingly slow.

To support a large number of simultaneously displayed segments, we need to find an approach that can support these types of segmentations.

Add shortcuts for slice view navigation

Within the second monitor view, there are 4 views (one "Main" and 3 "Secondary" views).

Desired shortcuts:

• Double click: Move slice intersection in all views to click location (Same a shift + mouse move currently)
• Click in secondary view: Swap secondary view with the main slice view

Automatically populate segmentations with editable list of segments

The list of segments in the module should be automatically populated based on an editable list of segmentations.

User should be able to easily change the default list of segments.

Add module for AutoSaving scenes

To ensure that data is not lost, we need to create a module that can automatically save the scene.

Module will be added to SlicerSandbox.

Add Markup representation for defining planes

Subtask of #4

There is currently no widget in Slicer for placing and modifying an arbitrary plane.
The new plane representation will be implemented within the Markups module.

Add dialogs for case management

Current file IO uses the default Slicer dialog for saving/loading.

Case management could be made simpler by adding options to the NeuroSeg module to manage the cases at a higher level with a predefined file structure (i.e. BIDS: https://bids-specification.readthedocs.io/en/stable/).

Startup:

• A dialog would show the current list of cases in some parent folder.
• User can select a case from the list or create a new one

Save:

• Instead of prompting user to select individual files, automatically save all files in the prescribed file location/structure

Other:

• Provide user the option to clear the current case and load a different one

Initial setup

• Invite collaborators
• Add tasks (tool for adding planes and curves on surfaces, etc.) that we talked about
• Put tasks into Project board (https://github.com/PerkLab/BwhNeuroImagingCollab/projects/1) according to our current understanding of priorities.

Add histogram control to threshold editor effect

By clicking and dragging on the image, the user should be able to create a threshold based on the metrics of the ROI contained within the specified region.

The histogram of the selected region should be contained in the segment editor panel, and users should be able to click and drag on the histogram to further isloate threshold values of interest.

Code here: https://github.com/Sunderlandkyl/Slicer/tree/threshold_histogram

Meeting 9/12/2019

Minutes from meeting:

PNL/Queens Collaboration Skype Call – 11:00am

PNL: Sylvain Bouix (SB), Jarrett Rushmore (JR)

Queens: Kyle (kyle.sunderland (KS)) and Andraas Lasso (AL)

Agenda for collaboration (long-term)

Neurosegmentation Module 

Tablet – needs to upgrade build 

Histogram 

Surface Draw 

Cortical Parcellation 

Topic 1: Surface drawing is the next project.

-specify weights via euclidian minimization between nodes (start here), perhaps with control points

-Similar to what Nikos Makris and Rudolf Pinard worked together to do around a decade ago (probably in C)

-shortest path algorithm is already in in surface VTK

AL – what happens after lines are drawn?

Use lines to cut up surface, needs careful design

Probably provide query using txt file containing rules for landmarks/boundaries/rules; algorithm uses text file input to chop up cortex

Shouldn’t be hard coded, since rules will change

TO DO: Provide example text file (from WMQL) for AL/KS

TOPIC #2: Interoperability between FreeSurfer and Slicer

Free view export is not perfect, there are some transforms.

TO DO: Send examples to Queens

AL: Slicer has importers, may have to update

TOPIC #3: Multiple Segments

Working on improving performance with multiple (I.e., 100s) segments

TOPIC #4: Formats

Segmentation output will ultimately be a bunch of lines, segments, etc

-what is the underlying segment file?

AL: Could store some as arbitrary metadata (e.g., 4D .nrrd)

Could store points and lines in a volume file as custom metadata

If there is a specific format we need, could do that

Would be nice to be able to ‘Save’ within a scene as well as to Export (e.g., to a FreeSurfer specific file)

Next Skype meeting October 10th, 10:30

Improve touch/pen interaction

To improve the ergonomics and speed of the segmentation process, it is important to develop and test segmentation interactions using touchscreen and pen.

• NA-MIC Project (Boston 2019)

These mostly completed, but they need to be integrated into VTK/Slicer.

In the course of using the interactions, we may also come up with additional ideas for improvements.

• VTK merge request (Includes touch, but not pen interactions)
• Slicer pull request
• Video example

Provide access to background info and instructions for selected segment

The right-hand panel in the NeuroSeg module currently contains a terminology widget that is currently a placeholder. It should be updated to have a widget providing background info and instructions for the selected segment.

If this ends up being too cramped (need to test), we may have to move the widget to some other location.

From #2 (comment):

We'd also like to have functionality so that when you add on it, a screen (like a help screen) could come up with links to definitions, movies, papers, instructions on how to segment that thing, etc. We would link this to Mike Halle's system as well so we could get the wiki for that structure.

Surface assisted cortical parcellation

Implement surface assisted cortical parcellation in 3D Slicer.

Surface assisted cortical parcellation manual

1. Load freesurfer surfaces into 3D Slicer (white matter and cortex)
2. Place several points along the sulcus on the white matter surface
3. Connect points using a curve using a pathfinding algorithm (Dijkstra's algorithm)
4. Map points from the white matter surface to the cortex surface
5. Place planes identifying the start/stop locations of the parcellation
6. Perform parcellation of the cortex using the planes and sulci definitions

Video Example

Subtasks:

• #5: Add curve interpolation along mesh surface
• #6: Add Markup representation for defining planes
• #7: Perform cortical parcellation from plane and sulci definition

images not showing up in documentation page

This page is linked from the main readme but the image links are broken:

https://github.com/HOA-2/SlicerNeuroSegmentation/blob/main/docs/General%20Segmentation.md

Add timer for segmentation

Spun off from: #2

If possible, it would be great to get a timer so we can see how long it takes for each segment. Is that possible? We've been using phones to time.

Likely we would track the amount of time that a segmentation spends in the "In Progress" state during segmentation. Only time when the segment is selected in the editor.

Add support for multi-monitor layouts

The current implementation for the multi-monitor widget works, but is not very future-proof.

A better solution would be to support multi-monitor layouts directly in 3D slicer.

Possible approaches:

• Specifying layout XML that supports multiple monitors (Similar to the split parameter)
• Multiple layout managers

Add curve interpolation along mesh surface

Subtask of #4

Given the vertices closest to markup A and B, find the path betwen the two through the sulcus.

Can potentially use:
https://vtk.org/doc/nightly/html/classvtkDijkstraGraphGeodesicPath.html

If the surface mesh is not dense enough to provide a good curve using the algorithm, we can subdivide the mesh along the predicted path to generate a more accurate curve.

Perform cortical parcellation from plane and sulci definition

Subtask of #4

Based on the tracing of the sulci as well as the user defined planes, perform the cortical segmentation of the space between the white/grey matter border and the surface of the cortex.

Send FreeSurfer examples to Queens

As discussed here, FreeSurfer data export and/or import in Slicer is not fully correct. Need to provide example files and description what is not correct and how can be fixed.

Provide example text file (from WMQL) for AL/KS

As discussed here

Add tag/filtering options to qMRMLSegmentsTableView

Tags:

• Allow arbitrary metadata to be stored for each segment
  Ex:
  • Hierarchy
  • State (Not started, In progress, Completed)

Search bar:

• Find a specific segment from the segmentation based on name
• Search for segments with specific tags

Quick filter buttons:

• Buttons to show/hide segments in the view based on current state (Not started, in progress, completed)

New state column in segments table:

• Operates like the flag button in outlook
• Left click toggles state from Not started -> In Progress <-> Completed
• Right click provides option to clear the segment and set state back to "Not started"

---

To facilitate this, it will be necessary to convert the segments table view from a statically populated table to a model/view data structure.