Comments (5)
It may also be worth looking into Dependant Markups for automatic updating of related markups.
MarkupConstraints now has its own repository; refer to that instead of the old DependantMarkup branch which will no longer be updated. https://github.com/KitwareMedical/SlicerMarkupConstraints
I plan to implement more adaptors (and create a system to register custom adaptors) to allow the constraints system to observe/constrain other nodes than markup control points; for example to observe a spline in its entirety rather than a single control point, or to constrain the position of a model or volume. From what I gather here, that feature would be important for synchronizing camera orientation with other control points so I will create a relevant issue and prioritize that work.
Plane markup has controls to work as a camera pose. Can be used as an interaction widget for placing and modifying camera poses along the flythrough curve.
I believe all markup nodes can enable the interaction widget; for camera position it may be better to use a fiducial markup with a single oriented point. I can confirm the interaction widget functions well with MarkupConstraints.
from vpaw.
Initial survey of modules / extensions that can support our Flythrough requirements.
WIP - will be updating this post more with additional info / discussions (fyi: @jcfr @andinet ):
-
Sequences module
Documentation: https://slicer.readthedocs.io/en/latest/user_guide/modules/sequences.html
Source code: https://github.com/Slicer/Slicer/tree/master/Modules/Loadable/Sequences- Too general? Hard to use.
- Screen Capture is based on Sequences, so perhaps we can extend on it instead or look at how it uses Sequences for lessons / good approach
-
ScreenCapture module
Documentation: https://slicer.readthedocs.io/en/latest/user_guide/modules/screencapture.html
Source code: https://github.com/Slicer/Slicer/tree/master/Modules/Scripted/ScreenCapture- Currently works in tandem with Sequence module (complexity) or 3D rotations
- No live preview or visualization (directly generates video / images)
- We can add FiducialMarkups as additional animation mode (will require camera pos info on point lists)
-
Endoscopy
Documentation: https://slicer.readthedocs.io/en/latest/user_guide/modules/endoscopy.html
Source code: https://github.com/Slicer/Slicer/tree/master/Modules/Scripted/Endoscopy- Very easy to use
- Provides live preview / visualization
- Can create a path using points and flythrough animation
- Cannot attach/modify camera poses (camera always looking along the path).
- We can add camera poses as markups to interpolate.
- Will need to add keyframe interpolation algorithm(s).
-
SlicerMorph Animator
Documentation: https://github.com/SlicerMorph/SlicerMorph/tree/master/Docs/Animator
Source Code: https://github.com/SlicerMorph/SlicerMorph/tree/master/Animator- A basic keyframe-based animation of 3D volumes.
- Supports interpolation of regions of interests (ROI), rotations, and transfer functions (volume property) for volume rendering.
- Output can be either as an image sequence of frames or can be compiled into mp4 format.
- Currently doesn't support following paths / curves but appears to already have keyframe interpolation. It is based on
ScreenCapture
. - Should investigate how easy it would be to add camera path and interpolation using camera-pose keyframes.
-
Volume reslicing
https://discourse.slicer.org/t/can-i-make-igt-volume-reslice-driver-run-automatically-after-an-event/16215 -
CameraPath
Source code: https://github.com/KitwareMedical/Slicer-CameraPath
Documentation: https://www.slicer.org/wiki/Documentation/Labs/FlyThroughNavigation- Based on code review, it might have most of the desired functions already implemented. Existing components:
- vtkMRMLCameraPathStorageNode: MRML node for camera path storage
- vtkMRMLCameraPathNode: MRML node to hold the information about a camera path
- vtkMRMLPointSplineNode: MRML node to hold the information about a 3D spline
- Faced some build errors due to old Slicer template. Might need an update of latest Slicer App template.
- Based on code review, it might have most of the desired functions already implemented. Existing components:
from vpaw.
See last paragraph in comment here: https://discourse.slicer.org/t/support-for-keyframe-based-animation/6885/6
How should we implement vpaw flythrough? Should it be keyframe interpolation of camera poses
along a curve, or features/structures to look-at
as you flythrough.
from vpaw.
Based on our internal discussions and from the discourse discussion thread, we have decided to modify the Endoscopy module to provide full support for a camera flythrough and keyframe interpolation. We can borrow specific implementations from CameraPath as/if needed.
Keyframe Interpolation
(From Discourse)
- Add some pure-VTK camera interpolation functionality, perhaps just one class based on vtkQuaternionInterpolator with some extra code to handle changes in field of view and maybe other parameters. This could go in vtkAddon if itβs C++ or in Endoscopy logic if it could be done in python (Iβd think it could).
User Interface
(From Discourse)
-
We could use the 3D view camera state itself as a way to insert keyframes. Initially we were considering having a flight mode for the 3D view to assist in flying around and placing keyframes, but since we can define the path curve first, it is probably not needed. The user can just fly along the path and stop and insert keyframes wherever necessary.
-
Regarding display of keyframes we can simply use a camera mesh model to indicate the position or use Plane markups
-
Along with camera pos and fov, would it make sense to also interpolate volume property, etc for better local control of visualization?
The Animator module has some code for this and it would be nice to either reuse or generalize this feature. Currently it requires that the transfer functions have the same number of control points at the keyframes. -
We should provide headlights that move with the camera (endoscopy simulation).
Use of Markups
(Discussions with @jcfr)
- Plane markup has controls to work as a camera pose. Can be used as an interaction widget for placing and modifying camera poses along the flythrough curve.
- Explore use of Dependant markups. It may also be worth looking into Dependant Markups for automatic updating of related markups.
- Markup Constraints by @allemangD [David Allemang] will be useful to sync camera position/orientation markups with the path curve and vice-versa:
https://projectweek.na-mic.org/PW37_2022_Virtual/Projects/MarkupConstraints/
Nice To Have
(From Discourse)
- It would also make sense for the Endoscopy module to export to a Sequence, like the Animator does, so that it could be recorded by the ScreenCapture logic. -- Steve Pieper
from vpaw.
Since we want to upgrade endoscopy and contribute it upstream into core Slicer, I have created a parallel issue / enhancement request in Slicer core here:
Slicer/Slicer#6502
cc: @HarryDC @andinet
from vpaw.
Related Issues (20)
- make VPAWModel run segmentation/geometry pipeline on new cases
- Add ImageStack module to VPAW
- Add CMake option to disable building of OpenHaptic support associated with iMSTK
- vpaw debug build fails on g3logger install step HOT 7
- Update application logo
- Create segmentation editor recipe to widen airway model simulating SGS ballooning treatment option HOT 1
- Generate haptics force feedback mechanism for 3D editing models closer to the surface
- Simplify importing a-OCT reconstructed aOCT binary image for simulated surgery
- Investigate using stereoscopic visualization and AR for simulated surgery
- Make VPAWVisual and VPAWModel modules more apparent HOT 2
- visualize centerline and laplace solution isosurfaces HOT 8
- VPAWVisualize does not detect missing patient ID HOT 2
- make VPAWModel run segmentation/geometry pipeline on new cases HOT 1
- DOC: Report more detail for pediatric_airway_atlas failures HOT 3
- DOC: update readme HOT 1
- DOC: Tell user about the run time of the `pediatric_airway_atlas` pipeline
- ENH: VPAW Visualize should show what is available, without an error message
- ENH: Add vpaw_USE_IMSTK as cmake flag HOT 1
- Establish versioning convention HOT 1
- ENH: VPAW Visualize and `qt.QSettings` HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vpaw.