gosling-lang/gosling.js

Grammar of Scalable Linked Interactive Nucleotide Graphics

gosling.js's Issues

Support loading raw data (JSON)

{
   ...,
   data: { type: 'json', values: [{ 'a': 1, 'b': 'c1' }, { 'a': 2, 'b': 'c2' }] }
}

Add a `stack` option?

Can be useful for expressing lollipop plots, pileup tracks, and gene annotation tracks. Using this option, one do not have to transform the data in advance.

We could also consider merging bar and rect since one of the main difference between them is the ability to stack (e.g., using nominal color makes stacked bar charts with bar marks).

Open questions:

• Technically, how to compute the bounding box of individual glyphs, rather than individual marks, and reposition them together?
  • (e.g., in gene annotation tracks, individual genes should move together).
  • We might need to add a id column so that visual marks having same ids can be considered to be belonged to a same glyph.
    • Grouping marks by id will help implementing other user interactions (e.g., selecting a glyph or seeing detailed info with a tooltip).
• stack option would make the implementation much more complicated as we are directly mapping data w/ visual representations so far.

Manage the performance of text elements

Need to correctly show N text elements across tiles:

https://github.com/sehilyi/geminid/blob/3de9a337ac82d3a6ec7d97464acea6852d986025/src/core/mark/text.ts#L83

Currently, text elements sometime disappear incorrectly while panning.

Support vertical tracks

We can use a view prop called orientation (detailed discussions in #242).

We need to think about how to seamlessly support this in circular layouts. For example, it is not straightforward how _l views (one horizontal, and one vertical) should be displayed in circular layouts.

API to merge a compiled HiGlass spec with an existing viewConfig

This can be helpful when one already has a viewConfig but want to add geminid tracks.

For example, we can use this functionality to add a cytogenetic band on the top as an overview in Cistrome Explorer.

To be useful, this should allow...

• determining the position of geminid tracks among the other views
• linking between geminid tracks and existing HiGlass views (e.g., show rect-brush)

Allow turning off dragging/resizing functionalities on brush

Use Case
Sometimes a user may want to only reposition a brush, but it is hard to control when it is too small (e.g., one in an overview cytogenetic band) where it only shows a resizing cursor.

Add options for the capacity in data fetchers

• capacity: limit the number of values to show in the single view
• assembly: select assembly (e.g., 'hg38')

Support reusable templates for tracks

Add a template property in the Track scope.

Many parts of source code can be taken from here: https://github.com/sehilyi/geminid/blob/3593e5c548631f62f2466008573442b092c8a1b3/src/core/geminid.schema.ts#L396

Support a brush in circular tracks

Refer to https://github.com/emeeks/d3.svg.circularbrush

Draft reimplementation of several GenoCAT examples

Support encoding quantitative values to `strokeWidth` and `opacity`

Also, move a background channel to style.

Implement a container app that automatically visualize tracks for given data source urls?

We could get the list of data and data types from the source url (e.g., USCS TrackHubs or HiGlass server) and visualize proper tracks for them.

Similar example of USCS w/ TrackHub:
http://genome.ucsc.edu/cgi-bin/hgTracks?db=hg19&hubUrl=https://hgdownload.soe.ucsc.edu/hubs/birds/hub.txt

Support a pileup track

https://www.ncbi.nlm.nih.gov/dbvar/browse/org/?assm=GCF_000001405.25&studies=nstd167

Show mouse hover effect

Support `BAM` format in the client-side

Zooming in very far in scatterplots makes points disappear sometimes

It seems to be due to an edge case of a shareScaleAcrossTracks function where domain of y-axis contains undefined when there is no marks in a first few tiles. We need to add a test for this to prevent similar bugs.

https://github.com/sehilyi/geminid/blob/cb131366ffbc7d2676e5f0a104db519862c6e7c0/src/higlass-geminid-track/geminid-plugin-track.ts#L125

Example spec:

{
    layout: {type: 'linear', direction: 'horizontal', columnSizes: 800},
    tracks: [
        {
            data: {
                type: 'json',
                values: [{column1: "chr10", column2: 1675492158, column3: 1675492349, column4: "peak5", column5: 8.09199},
                {column1: "chr1", column2: 197354819, column3: 197355017, column4: "peak3", column5: 16.59866},
                {column1: "chr1", column2: 1988820, column3: 1989013, column4: "peak1", column5: 10.37344},
                {column1: "chr1", column2: 206906930, column3: 206907124, column4: "peak4", column5: 14.71685},
                {column1: "chr12", column2: 1994241310, column3: 1994241502, column4: "peak6", column5: 8.86495},
                {column1: "chr14", column2: 2251226715, column3: 2251226907, column4: "peak7", column5: 11.24377},
                {column1: "chr14", column2: 2290828101, column3: 2290828293, column4: "peak8", column5: 11.59765},
                {column1: "chr1", column2: 7334981, column3: 7335172, column4: "peak2", column5: 7.74302},
                {column1: "chr19", column2: 2658902534, column3: 2658902726, column4: "peak9", column5: 10.88678},
                {column1: "chr2", column2: 402178574, column3: 402178768, column4: "peak11", column5: 13.89448},
                {column1: "chr2", column2: 443133037, column3: 443133228, column4: "peak12", column5: 8.07263},
                {column1: "chr2", column2: 281872500, column3: 281873014, column4: "peak10", column5: 13.99521},
                {column1: "chr3", column2: 584620282, column3: 584620721, column4: "peak13", column5: 14.68924},
                {column1: "chr5", column2: 959272251, column3: 959272447, column4: "peak14", column5: 15.45138},
                {column1: "chr9", column2: 1615304825, column3: 1615305022, column4: "peak15", column5: 16.80942},
                {column1: "chr9", column2: 1621778737, column3: 1621778929, column4: "peak16", column5: 11.17822},
                {column1: "chrKI270752.1", column2: "19", column3: "358", column4: "peak17", column5: 5.68613},
                {column1: "chrKI270752.1", column2: "20483", column3: "20810", column4: "peak18", column5: 10.67836},
                {column1: "chrX", column2: 2919283987, column3: 2919284179, column4: "peak19", column5: 11.59765},
                {column1: "chrX", column2: 2920007474, column3: 2920007672, column4: "peak20", column5: 14.53432},
                {column1: "", column5: NaN}],
                chromosomeField: 'column1',
                genomicFields: ['column2', 'column3'],
                quantitativeFields: ['column5']
            },
            tooltip: [
                {field: 'column4', type: 'nominal'}
            ],
            superpose: [
                {mark: "point", size: { value: 3 }, opacity: { value: 1 }},
                {mark: "bar", size: { value: 1 }},
            ],
            color: {value: "gray"},
            x: {
                field: "column2",
                type: "genomic",
                axis: "bottom"
            },
            xe: {
                field: "column3",
                type: "genomic"
            },
            y: {field: "column5", type: "quantitative"}
        }
    ]
};

Support converting other tools' view configs to Gosling specs

• pyGenomeTracks: https://pygenometracks.readthedocs.io/en/latest/content/examples.html
• Circos
• ggBio

Better cache already loaded tiles

When you zoom out and zoom in, and then quickly zoom out again on a track, you can see the white space on the left and right side since the track is taking some time to retrieve and render the tiles that had been already used previously. We could probably reduce this rendering time by properly caching the tiles.

Support selecting an assembly

https://rdrr.io/snippets/

library(RCircos)
data(UCSC.HG38.Human.CytoBandIdeogram)
UCSC.HG38.Human.CytoBandIdeogram

Support showing mouse position

"showMousePosition": true,
"mousePositionColor": "#000000"

Corces et al. 2020 (Nature Genetics)

Figure 4 in Corces et al. 2020 (https://www.nature.com/articles/s41588-020-00721-x)

• Use real data for Logo plot
• When stacking, allow sorting marks based on the size of them (here, height of letters in Logo plot)
• Support showing the blue-color interval
• SNP data
• Allow showing blue rectangular annotations that is not draggable

Also, refer to https://twitter.com/pkerpedjiev/status/1329455281085042690

Replace Travis with GitHub Actions

Refer to https://www.jeffgeerling.com/blog/2020/travis-cis-new-pricing-plan-threw-wrench-my-open-source-works

Limit the total number of visual elements rendered?

Considering the performance, it might be necessary to (1) limit the total number of visual elements that are rendered in the entire views and/or (2) show some kinds of warning messages when there are too many elements to let one know that the rendering performance can be affected.

Show quantitative color legend

Add `title` and `subtitle` in the root-level spec

The title and subtitle should be displayed on top of the tracks.

Allow turning on/off a tileset concept?

For certain visualizations, tileset concept does not apply that well, such as pileup tracks.

Support gradient color

Design inspiring: https://github.com/airbnb/visx

Support custom tooltips

Tooltip Support By Layout and Mark Type

Linear Layout

• rect
• bar
• point
• line
• area
• link
• band
• triangle
• text
• rule

Circular Layout

• rect
• bar
• point
• triangle
• link
• line
• area
• text
• rule

Things To Support

• Show multiple sets of information in a tooltip (i.e., when multiple visual marks are hovered at once). We currently capture all the visual marks that are being hovered (not only the one on the top), but we will need to visually organize such information well in a tooltip.

Add `juxtapose` similar to `superpose`

Genomic visualization often shows different data with the same visualization type (e.g., bar charts showing multiple samples or different genomic regions, like #8). However, defining this multi-view visualization makes the source code long with many duplicated definitions (e.g., parts of bar encoding), making it hard to be managed.

Similar to superpose, we could support juxtapose which helps one to shorten the source code of multiple visualizations with similar encoding:

tracks: [{
   ..., // common Track definition
   juxtapose: [A, B]
}]

is equal to

tracks: [{
   ..., // common Track definition
   A
},
{
   ..., // common Track definition
   B
}]

where A and B are the part of a track definition.

To make the design simple enough, superpose cannot be used inside juxtapose and vice versa, but both can be used together in a track definition:

tracks: [{
   ..., // common Track definition
   juxtapose: [A, B],
   superpose: [C, D]
}]

is semantically identical to

tracks: [{
   ..., // common Track definition
   A,
   superpose: [C, D]
},
{
   ..., // common Track definition
   B,
   superpose: [C, D]
}]

Add an API to compile a HiGlass track

Add an API to compile one track to a HiGlass track in addition to the "compile" function that convert multiple tracks to HiGlass views.

Support matrix view

We need to also discuss how we are going to do this with circular layouts. Given that it uses two genomic axes, it is not very straightforward showing it in a circular one.

On possible option would be to change to a horizontal matrix (#33), instead of not supporting circular layouts at all.

In order to support this, encoding two genomic axes should be supported. This means, we need to use a more generic track, like TiledPixiTrack, as a parent class of a geminid plugin track.

Incorrectly parsed chromosomes are assigned to `chr:1`

The chromosome name that is not parsed correctly (e.g., 12|NW_***), are assigned to the chromosome 1 incorrectly. Those should be entirely removed in the visualizations or correctly parsed (e.g., assign to chr:12?).

Improve rendering performance

Some Initial Thoughts

• Using HiGlass' data structure largely allows us to render visualization w/o too much cost because it pre-aggregates data and send Gosling the manageable amount of data values to display. But, still, improving rendering performance is really important (e.g., using many number of tracks w/ non-higlass data will significantly degrade the performance).
• Performance can be largely improved If we render texture (sprite), instead of individual graphical elements, and stretch them upon zooming in/out until getting new tiles. Some challenges:
  • Not straightforward to apply this concept in circular layouts
  • Some marks (text and circles) would result in ugly or even unreadable appearance
  • We still need to update subset of graphical elements if mark-level visibility options are being used (i.e., visibility: [{..., target: "mark" }])
• Sharing the same texture for identical-looking visual marks would be another option (Example)
  • e.g., use the same texture for the same color bars and just resize them.
• Linking multiple higlass views looks to significantly decrease the zooming/panning performance
• This rendering performance is especially important for dense matrix visualization (#38)
• A constant/related problem that we encounter: #230

Resources

• web worker
• SpriteBatches: http://rbwhitaker.wikidot.com/spritebatch-basics
• https://mattdesl.svbtle.com/drawing-lines-is-hard

Address disconnection between tiles

Rename properties for consistency

Consistent property names would improve the usability of our grammar:

• circularLayout: boolean => layout: 'circular' | 'linear'
• layout => arrangement (this is to make it more consistent to GenoCAT)
  • type: 'circular' | 'linear' currently in the layout should be moved out to the root-level spec (sibling w/ tracks)
• Use either static or zoomable consistently

Better default values for `outerRadius` and `innerRadius`

Changing the layout from linear to circular without specifying outerRadius and innerRadius should work well in properly displaying circular tracks for the given track size.

Support exporting PNG and PDF files

Need to support exporting publication-ready image files.

To support this, a few lines should be added to each visual mark encoding part.

For example, for a point mark:
https://github.com/sehilyi/geminid/blob/3f8e6db2fa80945de95266f4ea70a26387603d53/src/core/mark/point.ts#L68

Support unique URLs for examples

From Qianwen:

Is it possible to get individual url for each example, something like https://sehilyi.github.io/geminid/examples?marks=bar, so that it will be easy to link the example in the doc

Use native `name` option of tracks

{  
    // higlass viewConfig
    "mousePositionColor": "#000000",
    "backgroundColor": "transparent",
    "name": "Local data",
    "fontSize": 12,
    "labelColor": "black",
    "labelPosition": "topLeft",
    "labelBackgroundColor": "#F6F6F6",
    "labelTextOpacity": 0.6,
    "labelLeftMargin": 4,
    "labelRightMargin": 0,
    "labelTopMargin": 2,
    "labelBottomMargin": 0
}

Use a `compactLayout` option to simplify the code

Refer to higlass/higlass#998

Remove padding in text texture for precise height encoding

Converting text graphics into sprite introduces (vertical) padding, which makes height encoding inaccurate in logo plot.

code
https://github.com/sehilyi/geminid/blob/3f8e6db2fa80945de95266f4ea70a26387603d53/src/core/mark/text.ts#L121

screenshot

Editor and docs in a separate repo, eventually

Selection to draw rules across tracks?

Editor app crashes when cannot parse JSON

The part that parse the code for displaying textual description is making the error:

https://github.com/sehilyi/geminid/blob/3593e5c548631f62f2466008573442b092c8a1b3/src/editor/editor.tsx#L115

Support rectangular domain tracks

HiGlass track, named horizontal-2d-rectangle-domains

Need to support the higlass' 2d-rectangle-domains data type.

Support rotated 2D matrix

I do not yet have a good idea to support this in a grammar perspective.

Also, should support a higlass' matrix data type to visualize matrix.

Add tests thoroughly

To make the grammar more stable, we must add tests for each components (grammar, editor, renderer) more thoroughly and collecting sets of good test examples.

• Tests for how each channel is encoded for given mark type and field type (#185)
• Tests for default values of the grammar
• Tests for arrangements of views and tracks
• Tests for <HiGlassComponents/>
  • https://github.com/higlass/higlass/tree/develop/test
  • https://github.com/higlass/higlass-sequence/tree/master/test
• Examples in the editor should be tested to more safely deploy (#97)
• Add tests for circular brush (#111)

Support grids

Show rules for showing the chromosome boundaries

Bezier curves are not clearly defined if too narrow

This seems to happen only in circular layouts.

Allow selection for linking

https://www.ncbi.nlm.nih.gov/dbvar/browse/org/?assm=GCF_000001405.25&studies=nstd167