aces/brainbrowser

Web-based visualization tools for neurological data.

BrainBrowser is a JavaScript library allowing for web-based, 2D and 3D visualization of neurological data. It consists of the Surface Viewer, a real-time 3D rendering tool for visualizing surface data, and the Volume Viewer, a slice-by-slice volumetric data analysis tool.

BrainBrowser uses open web-standard technologies such as HTML5, CSS, JavaScript and WebGL. The Surface Viewer uses three.js for 3D rendering.

Demonstrations of available functionality can be found at the BrainBrowser website.

Getting Started

To try out some BrainBrowser sample applications, simply clone this git repo, and make the examples directory available over HTTP. This can be done using nano-server:

  $ git clone https://github.com/aces/brainbrowser.git
  $ npm install -g nano-server
  $ nano-server 5000 brainbrowser/examples

Note that nano-server is recommended because it can send gzip compressed versions of requested files. If a server without this functionality is used, files in brainbrowser/examples/models/ and brainbrowser/examples/color-maps/ will have to be gunzipped.

Once the server is running, navigate to to appropriate address (localhost:5000 in the above example) in your browser and select either of the Surface Viewer or Volume Viewer sample applications.

Surface Viewer

The BrainBrowser Surface Viewer allows users to display and manipulate 3D surface data. The Surface Viewer is invoked by calling BrainBrowser.SurfaceViewer.start() with the id of the HTML element into which the viewer will be inserted, and a callback function to which a viewer object will be passed:

  BrainBrowser.SurfaceViewer.start("viewer-div", function(viewer) {
    // Manipulate viewer object here.
  });

Full documentation of the Surface Viewer API can be found here.

Volume Viewer

The BrainBrowser Volume Viewer allows users to display and manipulate volumetric data in a slice-by-slice manner. The Volume Viewer is invoked by calling BrainBrowser.VolumeViewer.start() with the id of the HTML element into which the viewer will be inserted, and a callback function to which a viewer object will be passed:

  BrainBrowser.VolumeViewer.start("viewer-div", function(viewer) {
    // Manipulate viewer object here.
  });

Full documentation of the Volume Viewer API can be found here.

Contributing

Please see the Contribution Guidelines.

brainbrowser's Issues

Clean up Surface Viewer color blending code.

Not a major issue, but it doesn't need to be split across two separate functions, and it could generally be made easier to follow.

Race condition in Surface Viewer worker preloading.

The preloading of web workers is done through AJAX requests, and currently, if the preloading isn't complete by the time viewer starts, the viewer will throw an exception indicating that its configuration isn't complete.

Switching between functional and structural data in Volume Viewer example.

This will allow for testing of fMRI data handling and would also be a good test of the new dynamic loading features introduced in bf86904.

It might be preferable to have picking occur on the markers.

As currently implemented, markers are ignored for picking. The vertex of the model is what is actually picked and is used to look up an annotation. This can be confusing in terms of the interaction. If you click on an annotation marker but miss the vertex, nothing happens. I think one solution would be to store all annotation information in the marker mesh which could then be used directly through the picking to retrieve the annotation.

Surface Viewer model data object probably shouldn't expose add() and clear()

The add() and clear() methods are used internally, but if they're used directly by a user, they could leave the Surface Viewer in an invalid state (where the models loaded don't match the model data).

New annotation mechanism needs documentation.

To write before the next release.

Volume Viewer: movement in voxel space.

Currently, the Volume Viewer moves one pixel at a time, and this generally maps to one unit in world space. If a voxel is represented by more than one pixel, your have to move through several pixels to get to the next slice. It would probably be preferable to move one voxel at a time.

Volume Viewer example voxel text input isn't working.

It updates when the cursor is moved, but you can't input a position from it.

Document new model data management layer.

Needs to be documented.

Dynamic resizing for Volume Viewer panels.

It should be possible to resize the Volume Viewer panels after volumes are loaded.

Add regions in surface viewer for freesurfer.asc

There is any method to add labels at surface viewer loading freesurfer.asc?
I' m try to use the surface viewer to select a region and take the label that should be compatible with these http://www.slicer.org/slicerWiki/index.php/Slicer3:Freesurfer_labels.
Reading on the documentation I see that is possible to load a freesurfer asc, it will be very useful to have something like the atlas_label also for the this use case.
@PaulMougel

Volume Viewer cursor updating broken.

It seems that when you move the cursor on one panel, the slices on other panels gets updated but the cursor doesn't move. Was probably introduced when I refactored the loading code (07ded5e).

Mouse tracker doesn't follow movement outside of canvas.

The mouse tracker doesn't track movement that occurs outside the canvas. It should.

Simplify configuration

The Surface Viewer currently requires users to redundantly set configuration for built in workers. These workers should simply configure themselves, and the only configuration parameter that should have to be set by a user is the worker_dir. Also, I'd kind of like configuration to be handled by an object that gets and sets parameters through methods. E.g. :

BrainBrowser.config.set("worker_dir", "js/brainbrowser/workers");
BrainBrowser.config.get("model_types.mniobj.worker");

Keyboard controls for time traversal in Volume Viewer.

Would be handy to have a way to move through time using the keyboard.

Centralize event model.

Events are now defined on the viewer objects. I think it would be better to centralize them in the top-level BrainBrowser namespace, so that they can be accessed in non-viewer modules (e.g. BrainBrowser.loading or BrainBrowser.utils).

Write some tests for tree store.

Unlike a lot of the graphical stuff, the tree store should be fairly straightforward to write some unit tests for. A lot of systems depend on it, so I think it would be worthwhile to do so. I'm leaning towards qUnit as the testing framework.

Include a simple node server in the examples directory.

I just noticed that the mongoose server has become a more closed project. It's not as easy to just compile and use as it used to be, so I think the "Getting Started" part of the README is no longer valid. I'll just create a simple node server in the examples directory, and modify the README to use that.

Consolidate data fetching from local files and URLs

The Surface Viewer and Volume Viewer currently each have their own data fetching methods for files and URLs (see src/surface-viewer/loading.js and src/volume-viewer/loader.js). The methods to load color maps could probably be consolidated as well. The best place for the new methods would probably be in BrainBrowser.utils.

Make it possible to load files into the Volume Viewer.

The foundation is there. Just have to extract the networking code from the volume loaders and make it possible to switch in file loading code.

Trigger error events for all errors.

Only some parts of the Surface Viewer are currently triggering them. I'd rather they be more general.

Add usage examples to documentation for methods.

The docular pages don't provide an example of a method call, so I think that would be useful.

Touch controls for Volume Viewer

Would be cool if it were possible to do mobile with the Volume Viewer.

Make the start of rendering in loadVolumes an optional callback.

The Volume Viewer's loadVolumes() always starts rendering when loading is complete. Now that volumes can be removed and reloaded (see bf86904), it would be better if loadVolumes() took an optional complete callback that could start the rendering. That way, it could be used more than once without creating multiple render loops.

JSON loader should be able to load line models.

Currently, it assumes incoming models are polygon-based.

Get rid of caching mechanism in the Volume Viewer viewer object.

Not sure why this was done, but the cached_slices object in the viewer object doesn't cache actual slice objects. It's some strange intermediary object that holds the slice width_space and height_space objects and the image of the slice. Slice caching is already done within the volume objects, and the image can be cached in the appropriate panel object.

Create script to prepare MINC files for use with the Volume Viewer.

The Volume Viewer requires files to be split by the minc tools into header and raw data files prior to use. Create a little node script to make this step easier for users.

Mouse tracker for the Surface Viewer

I've noticed a few places where the mouse has to be captured, and the code is being rewritten each time. It would be nice to have a property of the viewer object that tracks the mouse (like the captureMouse() function does in the Volume Viewer).

Possible issue with coordinate conversion

Louis noticed (and Cecile and I confirmed) that when going through world coordinates in one axis each step is equal for both voxel and world coordinates. Based on the conversion from voxel to world this is probably not expected behaviour.
There is a voxel to world minc tool and some explanation of world coordinates in the following url:
http://www.bic.mni.mcgill.ca/software/minc/minc2_format/node4.html

It might be worthwhile looking at a file that is in stereotaxic space to ensure that voxel coordinates are correct first. For example look at 0,0,0 location to ensure that this is correctly encoded.

Loading Volume Viewer color maps from files.

This makes sense since it's now possible to load volumes from files (bf17c08).

viewer.model_data only holds data for last model loaded.

It should probably hold an array containing data for all the models loaded.

Change signature of Surface Viewer's setTransparency()

Right now it requires you to give a shape name as the first parameter. I think it would be better if it defaulted to changing the transparency of all loaded shapes, and then maybe took an optional second parameter (or an options object with a "shape" property) to set the transparency on a single shape. E.g.

viewer.setTransparency(0.5, {
  "shape": "left_hemisphere"
});

Actually, this type of signature might be good for other methods, like setWireframe().

Surface Viewer configuration documentation is out of date.

Needs to be updated to include separate worker configurations for model types and intensity data types.

Update documentation related to dynamic loading in Volume Viewer

Due to changes in bf86904, some aspects of the documentation are out of sync:

• The Volume Viewer has some new events.
• Sample code in the readme. Possibly sample code elsewhere.

Get rid of jQuery dependency in VolumeViewer.loader

Didn't realize I had left a dependency on jQuery in VolumeViewer.loader. Can fix this with issue #5.

Take Volume Viewer color map loading code out of loadVolumes()

loadVolumes() should ideally only handle the loading of volumes into the viewer.

Document Volume Viewer templating mechanism.

Currently, it isn't very well described anywhere.

Make loading of volumes into Volume Viewer more dynamic.

Currently, the Volume Viewer has to load all volumes when it starts rendering. I'd like to make it possible to load/remove volumes dynamically. This could potentially make it possible to load volume data from files.

Integrate annotator into BrainBrowser

Came up with a rough draft for annotation mechanism for BrainBrowser at a hackathon this weekend. Just need to refine it an integrate into the core.

Document Surface Viewer object model.

The new JSON loader essentially standardizes a Surface Viewer data format. This should be documented somewhere.

Document BrainBrowser tree storage layer.

This should also be documented.

Example server should set proper MIME types.

It's currently sending "text/plain" for everything which is causing annoying console errors.

Load MINC RGB color maps

Look into MINC RGB color map files and how they might be loaded into BrainBrowser.

Volume Viewer distance measurements don't work on Windows

Keyboard events for the control key seem to work differently. It looks like the keydown event is repeated when the key is held, which means the distance measurement anchor will keep getting reset..

Slice series example includes current slice.

When you click one of the "All Slices" buttons in the Volume Viewer example, the current slice shows up as the first image in the series.

Add BrainBrowser to CDNJS

Would it make sense to make BrainBrowser available through cdnjs?

I saw a few examples of using requirejs to load javascript libraries from cdnjs into an ipython notebook and curious if a similar approach would work for brainbrowser. You can always download a release and configure it locally, but cdnjs looks like it might make the process more straight forward.

Volume Viewer synchronization should be based on world coordinates.

I've been told this would be more meaningful than the current setup which syncs based on voxel coordinates.

Surface Viewer instensity_data.createColorArray should be cleaned up.

Method is defined in parse_intensity_data.js. It probably shouldn't be part of the public API. It makes more sense to put it as a private function in surface-viewer/modules/color.js, since that's the only place it's used. It could also probably be cleaned up a bit:

• Use a typed array for the created color array
• Define size of color ahead of time instead of constantly pushing onto the array
• See if the logic can be clarified a bit.

Color map scale shows intermediate values outside min-max range.

Using the threshold slider results in the color map scale canvas displaying intermediate values that are outside the min-max range.

Volume Viewer distance measurements

Create an interface to select two points on a slice and have the viewer indicate the distance in world coordinates between them.