paluchlabucl/deformingmesh3d-plugin

An imagej/fiji plugin for interactive segmentation of 3D surfaces

DM3D

This is being moved. This is the original version that was published. I've moved it to a version 1.0.x to signify feature complete release. Development will continue at Living Technologies

This one will remain at the update site and might receive bug fixes. Future features will be found at a new update site.

ThreeD image segmentation algorithm, for roundish cells.

The latest documentation can be found at:

DM3D Github pages

Roadmap 1.0 Draft

As of version 0.8.0 these features have been added. There are some details that need to be worked out. Especially documentation.

This is the start of a draft for a 1.0 release version. Right now I have quite a few components that are nearly ready but need to be added. The two major features that I want included in a 1.0 release.

installation

The easiest way to install this plugin is through the Fiji update site.

Run Fiji, choose the menu "help", select "update..." after updating there will be a dialog that has "Manage Update Sites".

https://sites.imagej.net/Odinsbane

Once the site has been added, updating fiji should cause it to download and install the deforming mesh plugin which can be found in "plugins"->"PL_Mesh3D"->"Deforming Mesh 3D", and JFilament will also be installed.

Documentation

DM3D Github pages

This includes guides for using the plugin, and javadoc for using the plugin via scripting.

Changes

1.0.0

• Furrow controls in the 3D Canvas window.
• Tracking controls in 3D canvas window
• guessMesh api has more options
• BUG FIX: there was a condition error in the spit mesh algorithm

0.9.8

• Changed volume calculation technique.
• Added a way to create training labels from a labelled image.

0.9.7

• improved slice view synchronization
• added sculpt/select items to slice view
• Limited title length
• Changed the way errors are displayed

0.9.6

• updated scijava pom
• added flatlaf to pom
• created a way to substitute image data
• added a way to remove all tracking information
• added a remote prediction class (experimental)
• removed cursor radius controls, added javascript portal
• improved display for hidpi developing with flatlaf for consistency
• Added run file to javascript console
• Added faq to help menu.

0.9.4

• Improved autotracking accessiblity.
• • linkPossibleTrack() to link the selected track.
• • autotrackAvailableTracks() attempts to link all available in the first frame with the second frame.
• Import from open image has been created, for loading meshes from different image geometries.
• Connection remeshing calculates the expected number of connections differently/better.
• Orient the furrow from the 3D canvas.
• Flip the furrow.
• BUG FIX: empty tracks get removed from the manager. 0.9.2
• 3D viewer has been improved to handle viewing small sections of large images. 0.9.1
• BUG FIX track manager selects correct mesh. 0.9.0
• Selected mesh is now updated in track manager.
• Exposing averaged intensity vs time plot
• exposed frame to frame displacement simple auto tracker.
• Updated mesh file reading to include progress and range checking.

0.8.0

• Select channel from main display.
• Added 3D sculpting - feature preview version.
• mesh track manager and undo/redo follow the state of the ActionManager.
• deformAllMeshes has a limit, can be used in a script.
• restartOffScreenCanvas javascript function to address bug in offscreen canvas.
• scaled units are indicated for connection remesh.
• histogram display improved.
• 3D preview removed for now.
• Meshes can have transparency.

0.7.0

• Creating Binary images algorithm improvement.
• Visualization improvements.

0.6.0

• Feature preview modifier and limited controls to the furrow panel. Debugging for next release.
• created 2d svg/image view and exposed through javascript
• added normals to the data object, that need to be updated.
• light tracks the display.
• objects are lit by a single directional light, and an ambient light.
• lighting can be adjusted in the mesh viewer to find better lighting settings.
• moved the ring controller to be in the controlframe
• exposed mesh sculpting via the furrow3d view input
• BUG FIX slice view 3d scrolling
• made the modification cursor size adjustable.
• ui overhaul
• track manager has select all/ select all before/ and select all after
• track manager modifies names to avoid duplicates.

0.5.0

• moved initializer to a new frame.
• Created a non-3d mode
• BUG FIX names can break mesh loading.
• BUG FIX track manager changes show surface.
• Split mesh and deform partial available from java script.
• Some linear plots.
• Moved manage tracks to a tabbed pane.
• Improved non-3d version to run program for running over x11 forwarded.
• steric mesh uses shorted distance along normal.
• improved redo/undo
• "SegmentationController.selectTrack" was broken.
• Bug fix: BinaryInterceptible was off by 1 for z axis.
• Furrow3D shows the texture.
• Furrow input panel is changing design.
• Histogram input is smaller and adjustabble.
• show/hide furrow.
• Meshes are now selectable from the furrow screen.
• Worked on the import meshes. Created 4 types of import. Checked each on. do and undo seemed to work.
• Removed a bug where removing a track would empty all of it's meshes, breaking undo.
• Added labels to the histogram control.
• Implemented the curvature smoothing algorithm. Not sure if it does anything.
• Channels now sync to the frame when they're added.
• BinaryInterceptible considers edge of image points as edges.
• Added limited transparency control
• Added mesh projections to the Furrow tab.
• Restructuring the furrow tab layout.
• Remove the field 'original_stack'
• Adding a non-energy option.
• Added channel select when selecting an open image plus.

0.3.9

• Added connection remesher to control panel.
• Multi channel support
  • Display multiple volumes as channels
  • Select a single channel from a multi-channel image
• Steric Energy improvements
• Exposed auto-detect meshes through javascript

0.3.8

• Contrast preview dialog matches the clamped values.
• Removed the normalizer weight from controls, added steric neighbors.
• Exposed connection remesher in javascript with min/max lengths.
• Contrast adjustment for displayed channels.

0.3.7

• Added key command actions to 3D display.
• Added help screen to hud to display key commands.
• Added mesh export options for wire-frame and shell meshes.
• Swing javascript terminal has a shortcut execution mode.
• Contrast adjustment tool.
• Selecting open image returns to the current frame for swapping energies.
• Improved 3D volume data handling, now is a DataObject.
• Created a 3D surface for volume textures.
• Improved 3D geometry display performance.
• python meshes have a reader and writer.
• Bug fix: color selection could cause infinite loop.
• Connected components refactoring.
• Connection based remeshing *
• Manual Mesh editing *
• Forces can be displayed as vectors *
• VolumeDataObject is accessible through MeshFrame3D api for displaying multi-channels. *

• only available through javascript.

0.3.6

• javascript available actions improved.
• surface plots available through javascript interface.
• steric forces improved.
• clear mesh from current frame.
• transformations to look at plane.

0.3.5

• Initial window size checks monitor size and attempts to be <=80%
• Mesh initialization zoom keeps cursor location the same.
• ctrl + drag to move mesh initialization views.
• 3 and 1 panel views during initialization
• Create and show a binary representation of currently selected mesh.
• stl writing improved.
• Added a cursor to the initialization window.
• BUG FIX: load meshes now finishes when canceled.
• BUG FIX: restart meshes, undoable.
• load/save parameters
• Initialization sliders are finer controlled.
• Can now switch image by selecting open image.

0.3.4

• status panel/mesh color display
• ctrl z /ctrl-shift z undo/redo respectively
• js terminal has suggestions and completions
• track backwards
• frame number displays 1 off of index. (looks like imagej's 1 index scheme)
• exports to ply file.
• Mesh track manager for tracking arranging meshes through time.
• user preferences are saved

Reference

Chapter 19 - An active contour ImageJ plugin to monitor daughter cell size in 3D during cytokinesis

MB Smith, A Chaigne, EK Paluch

https://doi.org/10.1016/bs.mcb.2016.05.003

deformingmesh3d-plugin's Issues

BUG: Selected mesh stays selected when tracks are cleared.

This BUG affects users when they, load new meshes, use the mesh manager, or start new meshes.

The selected track will be removed from the display, but the track will still be selected. Tracking the mesh will cause it to be added, in it's entirety, to the segmentation.

Volume displayed in the volume viewer doesn't update scale when a new image is opened.

When a new image is open the image can look distorted if it has a different scale than the previous image.

Work-around. Show/hide the volume.

Mesh scale factor

Hi,

Thank you for this awesome plugin to create 3D cell annotations. I want to use existing nuclei segmentations as mesh initialization. To achieve this I load the label images in Python and convert them to meshes with skimage.measure.marching_cubes. Then I shift the obtained meshes such that the origin corresponds to the image center. However, I have to apply an additional scale factor to the obtained vertices to get the meshes to fit.

In code this is what I am doing for each label-mask:

verts, faces, _, _ = marching_cubes(mask, spacing=spacing)
        
# Shift to origin.
verts[:, 0] -= ( labeling.shape[0] // 2 ) * spacing[0]
verts[:, 1] -= ( labeling.shape[1] // 2 ) * spacing[1]
verts[:, 2] -= ( labeling.shape[2] // 2 ) * spacing[2]
# Scale.
verts *= factor

So far I couldn't figure out how this factor is computed. By trial and error I figured out that the following combinations work:

• spacing = (0.6, 0.216, 0.216), factor=0.0225
• spacing = (1.0, 0.25, 0.25), factor=0.0185
• spacing = (0.5, 0.25, 0.25), factor=0.0195
• spacing = (1, 1, 1), factor=0.004875

I tested this by loading the mask image (which has the spacing set) and then loading the mesh. Then I checked by eye if the outlines fit. I guess the re-scale factors are good enough that I can't see it anymore by eye. However, I would not bet that the values are 100% correct.

Could you enlighten me what the correct formula would be to obtain this scale factor?

Thank you and have a nice day!

Control the furrow plan from the 3D canvas.

The commands to move the furrow, center, and split a mesh all exist and just need to be added to the appropriate controls with hot keys.

f - centers the furrow
f+shift - starts the choose rotation
f+ctrl flips the furrow direction
up moves the furrow forward
down moves the furrow backwards

Interactive track adjustment.

Using the T and B keys can be used to track the mesh between frames. By holding down "shift" the mesh in the next frame can be clicked to be selected. "ctrl" will copy the mesh from the current frame to the next frame. "L" will attempt to link automatically to a candidate mesh.

Record snapshots is broken when the volume is showing.

The output image is only the first frame.

Make ctrl-s save the current meshes.

Add an accelerator to save meshes using ctrl-s. Concerns, selecting a file if one hasn't been selected before. Adding some indication that the file is actually being saved.

sphere initialization handles are not scaled properly.

The handles for controlling the size of an initialization sphere are too large for large images.

Colors are linked to names.

Colors are linked to the name of a mesh track. That is ok for autogenerated mesh names, but if a mesh name has been set, it should retain that name. It should also be represented in such places as the status pane, and the mesh track manager.