d3/d3

Bring data to life with SVG, Canvas and HTML. :bar_chart::chart_with_upwards_trend::tada:

D3: Data-Driven Documents

D3 (or D3.js) is a free, open-source JavaScript library for visualizing data. Its low-level approach built on web standards offers unparalleled flexibility in authoring dynamic, data-driven graphics. For more than a decade D3 has powered groundbreaking and award-winning visualizations, become a foundational building block of higher-level chart libraries, and fostered a vibrant community of data practitioners around the world.

_{Daily downloads of D3 · oss-analytics}

Resources

• Documentation
• Examples
• Releases
• Getting help

d3's Issues

Easy way to generate contour maps

I started on this for Protovis: https://github.com/jasondavies/protovis/tree/contours but there's still a bug that I need to fix (try generating a contour map at zero on a grid of zeroes).

I've also written a CONREC implementation here: https://github.com/jasondavies/conrec.js but Paul Bourke suggested simply iterating over the mesh and computing the intersections as computers are much faster now than when CONREC was devised. This approach also works for non-uniform grids. So this is the approach I used in my Protovis branch.

I'd be interested to know if there's a faster way to do it, otherwise I'll continue with this approach and port to D3 when I get some time.

Merge selections; merge transitions.

When implementing transitions, there's often a fair amount of redundancy between actions that happen on enter, update and exit. Currently, it's not easy to specify actions that apply to enter + exit, or enter + update, etc.

For example, say we have a bar chart, where bars are represented with an svg:g element that contains an svg:rect bar and an svg:text label. We might want to support transitions for entering and exiting bars, and also a changing x-scale. An entering bar should fade in from its position using the old scale, while resizing to match the new scale.

Let's break this down into different chunks. First you have the select all the bars. This becomes the default "update" selection:

var bar = vis.selectAll("g.bar")
    .data(data);

(Note that I didn't implement a join function for this example, but typically you would implement one rather than relying on the index for joining data to nodes.) Then we create the entering bars for new values.

var barEnter = bar.enter("svg:g")
    .attr("transform", BAR_TRANSFORM);

barEnter.append("svg:rect")
    .attr("fill", fill)
    .attr("width", BAR_WIDTH_OLD)
    .attr("height", height);

barEnter.append("svg:text")
    .attr("x", TEXT_X_OLD)
    .attr("y", height / 2)
    .attr("fill", "#fff")
    .text(TEXT_TEXT);

The placeholder functions that end in _OLD mean that the values are computed using the old scales. After the entering elements are created, we immediately trigger a transition to the positions with the new scales:

var barEnterTransition = barEnter.transition();

barEnterTransition.select("rect")
    .attr("width", BAR_WIDTH_NEW);

barEnterTransition.select("text")
    .attr("x", TEXT_X_NEW);

At the same time, we also have to deal with updating elements—elements that are neither entering or exiting. First, we update the label:

bar.select("text")
    .text(TEXT_TEXT);

Then we create another transition that computes the bar width and text position using the new scale:

var barTransition = bar.transition()
    .attr("transform", BAR_TRANSFORM);

barTransition.select("rect")
    .attr("width", BAR_WIDTH_NEW);

barTransition.select("text")
    .attr("x", TEXT_X_NEW);

Note that this is almost identical to the barEnterTransition; the only difference is that we don't need to transition the svg:g element's "transform" attribute. Although, we could if we wanted entering bars to fly in from the top or the bottom.

OK, lastly, we need to handle the exiting bars:

var barExit = bar.exit();

var barExitTransition = barExit.transition()
    .each("end", remove);

barExitTransition.select("rect")
    .attr("width", BAR_WIDTH_NEW);

barExitTransition.select("text")
    .attr("x", TEXT_X_NEW);

Again, this is almost identical to the barEnterTransition, except we need to remove the svg:g elements after the transition finishes. It'd be nice if this happened automatically, but currently we don't build any default behavior into the exiting selection.

Cross operator (for data stack)?

The cross operator (as in the splom example) is very useful for recreating something like the Protovis data stack, where data from multiple levels in the hierarchy is available in the property functions.

The implementation is simple, if a bit confusing. Take an array a. To use this array as data, as the y attribute of each element, crossed with the enclosing parent's data as the x attribute:

function cross(a) {
  return function(d) {
    var c = [];
    for (var i = 0, n = a.length; i < n; i++) c.push({x: d, y: a[i]});
    return c;
  };
}

Another possibility might be to pull the parent data out of the current group in the property function (group.parentData). But, we don't currently expose the group index (j) or the group itself in the property function. We could pass these as two additional arguments to the property function, similar to Protovis but limited to the immediate enclosing parent, but I like the current simplicity. The extra arguments were a frequent source of confusion in Protovis.

Yet another possibility is for a parentData() method (or similarly a group() method that returns the current group, changing the node returned by node(), etc.), but that requires keeping state within the selection, which I don't like because of reentrancy issues.

So, I like the idea of the cross operator making the access of the parent data more explicit. But I'm not sure about the efficiency or the semantics.

Format negative numbers with thousands-grouping.

d3.format(",d")(-100000) returns "-,100,000".

Headless unit testing framework.

We should use env.js, now that 1.3 supports Node.

Type error in value interpolation.

For example:

d3.selectAll("p")
  .transition()
    .style("color", function() { return d3.hsl(Math.random() * 360, 1, .5); })

Results in:

d3.js:585 TypeError: 'rgb(145, 255, 0)' is not a valid argument for 'in' (evaluating 'k in a')
d3.js:1539 TypeError: 'undefined' is not an object (evaluating 'ik[tk].call')

Modules.

We need a way to break D3 up into modules so that we can make it extensible without being overburdened. A rough cut:

[core]
  array.js
  blend.js
  call.js
  date.js
  dispatch.js
  ease.js
  end.js (TODO: replace with closure wrapper)
  event.js
  hsl.js
  interpolate.js
  ns.js
  object.js
  range.js (XXX: only used by scale)
  rgb.js
  root.js (TODO: fold into selection.js)
  select.js
  selectAll.js
  selection.js
  start.js (TODO: replace with closure wrapper)
  timer.js
  transition.js
  tween.js

[svg] -> [core]
  arc.js
  area.js
  line.js

[scale] -> [core]
  category.js
  format.js
  linear.js
  log.js
  ordinal.js
  pow.js
  sqrt.js

Sorting nodes.

A sort operator would be nice for reordering nodes.

Easier removal on exit transition.

Currently, it's a bit of a pain to remove elements after an exit transition:

selection.exit()
  .transition()
    …
    .each("end", function() { d3.select(this).remove(); });

I could see this being easier if either the this context was a singleton selection, in which case you could say:

.each("end", function() { this.remove(); })

Or, if there were a unified transition end event, in which case you might say:

.on("end", function() { this.remove(); })

where this refers to selection, rather than a singleton selection. I suppose if we had unified transition events, that trigger individually for staggered transitions (variable delay or duration) or collectively for non-staggered transitions (uniform delay and duration), then we could use the later on("end", …) in either case, which seems nice.

Rename "map" to "datum".

For release 2.8:

• data() would return a one-dimensional array of data for the first group
• data(array) and data(function) would compute the data-join [as today]
• data(null) would throw an error [as today]
• datum is a new alias for map
• datum(function) would evaluate a function for each selected element and set each associated datum [as today]
• datum(null) would clear the bound data
• datum(object) would assign the same data to all selected elements
• datum() would return the bound data for the first non-null node (alias for node().__data__)

For release 3.0:

• rename map to datum, removing alias

Data module?

Should there be a d3.data module to contain the data-related utilities? Currently, that includes: ascending, descending, min, max, keys, values, entries, split, merge, range, format, nest (map, key, sortKeys, sortValues, rollup). There are also some AJAX utilities, which could be in the data module or a separate ajax module: xhr, text, json, html, xml. And csv, but that's already in a separate module.

selection.node

In a similar vein to selection.attr("foo"), we should have a selection.node() method which returns the first node. This is identical to selection[0][0] but works as expected when some of the nodes are null, and makes the code more readable.

Namespaced event listeners.

The on method should allow a namespace, similar to jQuery's bind feature. This allows multiple event listeners to be registered for the same event type under different namespaces, such as "click.foo" and "click.bar".

A new off action should similarly allow de-registration of event listeners.

Namespaced transitions.

We need to give transitions names, so that one transition can easily cancel an existing transition for the current element. Perhaps there's also a way to cancel all transitions of a given name. I'm filling this as a bug because in interaction-driven transitions there is currently no way to cancel the current transition to start a new one.

problem with enter, exit usage

Hi,

Your excellent new documentation led me to try using enter() and exit(), but I seem to be doing something wrong.

I have the most recent version of d3 (git log says feb 2011).

To try to make a simple example, I edited your stack.html example as follows, starting at about line 85 in the original. I didn't see an easy way to add an id to the array of arrays, so I used a summation. I figured that summing the data y values should produce a unique id.

var vis = d3.select("body")
    .append("svg:svg")
    .attr("width", w)
    .attr("height", h + p);

var layers = vis.selectAll("g.layer")
    .data(data, function(d,g,i){
        var sumd = 0;
        d.map(function(val){ sumd += val.y ; });
        return sumd;
    });

layers.enter().append("svg:g")
    .attr("fill", function(d, i) { return color(i / (n - 1)); })
    .attr("class", "layer");

However, while the sums work, the layers object is an array of nulls, and the enter().append
line does nothing, and the subsequent bars lines do nothing either.

I started to dig at the source code, and I can't see what might be going on...but mostly because I don't understand all of the logic flows yet. It does seem like the join function might get called twice, but I was only seeing it called once, so I'm not sure.

My use case is to have stacked bar charts, and to be able to insert and remove series in the stack, as well as to extend or shrink the time scale (x-axis), so being able to use enter and exit would rock. Any help or insight would be appreciated.

Regards,
James

The `select` operator and data inheritance.

Usually, when the select operator is run, each newly-selected node inherits its data from the corresponding node in the original selection. However, this was recently changed so that if the new selection already hd data, it would not inherit data from the original:

if (subnode && !subnode.__data__) subnode.__data__ = node.__data__;

However, I'm not sure this is what we want.

First, what if the data is an array of numbers or booleans (or null, rather than undefined)? This could be very confusing in terms of some of the newly-selected nodes inheriting data and some of them not.

Second, the above implementation prevents the new selection from inheriting the data, even if the original selection has data defined. Perhaps we should check if the data was set explicitly on the original selection, so that it can override what's stored in the DOM's __data__?

Remove event listeners.

Removing a listeners (via .on("event", null)) does not always work. Bug?

Namespaced events for listeners to support multiple listeners would be great.

Trying to Get More Sets into the .matrix for Chord Diagram

Hi,
Started trying this with Circos and got nowhere. This looks easier as I am not a coder. Trying to understand how to get more rows, and or/columns in the matrix.

Tried this and the browser gives me a blank page with no errors in Firebug.

.matrix([
[ 12500, 10300, 10500, 7800 ], //black
[ 16500, 11200, 7300, 9200 ], //green
[ 20600, 10900, 9600, 3200 ], //red
[ 13600, 4200, 1900, 14500 ], //purple
[ 3400, 6100, 1200, 4300 ] //yellow ******* this is new line added
]);

var w = 910,
h = 800,
r0 = Math.min(w, h) * .41,
r1 = r0 * 1.1;

var fill = d3.scale.ordinal()
.domain(d3.range(5)) //color ranges ****** changed from orig (4) to this (5)
.range(["#000000", "#22dd33", "#ff1133", "#3311ff", "#ffd538" ]); ****** added in additional color

But this doesn't work. Guessing I'm missing something else in the code that allows for the addition of more data sets but can't find what that might be.

Anyone know how to handle this? I need to be able to handle many more data sets than the demo has.

Tks,
Jeff

Transition "step" events.

And possibly allowing "text" tweening.

Nested transitions compete for ownership.

Consider the following:

var foo = d3.transition("foo");
foo.selectAll("bar").attr("attr1", "value1");
foo.select("bar.baz").attr("attr2", "value2");

The second select creates a new transition which clobbers the previous selectAll's transition.

Transitions with text(d3.tween(...)) fails.

There's no d3_transform_text_bind implemented yet.

Reselect

Convenience method "reselect()", which reuses the selector and data bindings.

Setting text(null) should clear the text content.

Currently, it sets the text to the string "null" instead. This is inconsistent with attr and style.

Toggling classes.

Proposed: a method classed(name, value) that allows easier manipulation of the "class" attribute. The given value may be specified either as a constant boolean or a function that returns a boolean. A true value will add the class (if not already present); a false value will remove the class (if present).

SVG path utilities.

We should provide some helper libraries for constructing paths in SVG, such as arcs, areas and lines (with spline interpolation).

Immediate transitions if duration is 0?

It would be nice if transitions were applied immediately if the duration were zero (or more generally, if the delay + duration is less than zero). Currently, you have to wait for a setTimeout callback of zero, which typically means a delay of 10ms+ and an extra redraw.

But one difficulty with this approach is that the duration must be set to zero before any operators are defined on the transition. I.e, the order of operations in the transition will matter.

Monotone interpolation for d3.svg.{line,area}.

Not sure yet whether this belongs in the svg or geom module.

A more compact attribute syntax?

I've been using raphael lately and I really like the ability to make a set of attributes, see the example below. Does this functionality exist in d3? Do you think it ever would?

//d3 way to make a simple circle.
p.append("svg:circle")
.attr("cx", 150)
.attr("cy", 140)
.attr("r", 40)
.attr("fill", "#F00")
.attr("stroke", "black");

// The way I would like to do it.
p.append("svg:circle").attr({cx: 250, cy: 240, r: 40, fill: "#F00", stroke:"black"});

Remove shouldn't select the parent node.

The problem is that if you selectAll followed by remove, you end up with redundant copies of the parent node, which is confusing.

Examples broken on main wiki page

All but the first two example img links are broken on http://mbostock.github.com/d3/

Set inner HTML.

An html method, similar to text, but for setting HTML content.

`this` context: Node or singleton selection?

Should the this context of property functions be a Node or a singleton selection, as in d3.select(this)? The singleton selection is convenient in some cases, but it does not provide the complete functionality of the W3C DOM API, and extracting the element is a bit awkward: this[0][0].

Also, there are cases where callbacks are associated with individual nodes (such as "mouseover") and other cases where there are multiple nodes (such as the .call method, and non-staggered transition "end"). Maybe it's only in the latter where we use a selection as the this context.

Also, we could shorten the syntax to d3(this) rather than d3.select(this) to make it easier to convert. And this.node() might be clearer than this[0][0].

Quantile scale. (And quantize?)

A quantile scale would be nice.

Standardize namespace prefix syntax ("svg|…" vs. "svg:…").

CSS and XML use different conventions for namespace prefixes. For example, in XML "svg:g" refers to a g element with the svg namespace prefix; in CSS3 the equivalent is "svg|g". D3 uses both conventions, which leads to confusion. For example:

d3.selectAll("path")
    .data(…)
  .enter("svg:path")
    …

In fact, most current browsers implement the Selectors API Level 1, which does not support namespaces. Level 2 appears the same, strangely.

Still, it might be a good idea to use the same syntax everywhere, although "svg|g" and "xlink|href" is definitely less familiar to folks... And we'd have to have some way of stripping the namespaces from the selectors, as this functionality is not implemented in the standard. Hmm, or not.

Filter operator.

So that we can efficiently reduce the set of selected nodes.

Enter + exit selections are a bit weird.

Currently, when a selection fails to find a match, you can bind it to data and get an enter selection. The entering selection has null node values, but inherits the parent node from the previous selection. This allows you to append nodes to the entering selection, which is convenient, but requires creating a selection first to specify the parent. For example, this adds elements to the root document element:

d3.selectAll("#container div")
    .data([...])
  .enter.append("div")
    .apply();

Whereas this adds elements to the element with the id "container":

d3.select("#container")
  .selectAll("div")
    .data([...])
  .enter.append("div")
    .apply();

Perhaps this is okay, but it seems like structure of the entering selection is a bit weird.

Another problem is that the exit selection doesn't have any associated data. This can make it hard to determine the appropriate transitions values. For instances, if you have a number of tick marks that are displayed, and you change the scale, you might want those old ticks marks to move to their location under the new scale. However, you can't compute the location under the new scale because the exiting selection's data is undefined!

Simplify data join

Specify single data key function, and store result on DOM for later joins?
Enable (for rare cases) application of node key to documents computed outside of d3.
Consolidate to a 3-parameter invocation (e.g., data(data, dataKey, nodeKey) or 1-parameter invocation (e.g., data(data) | data({data:data, dataKey:<...>, etc})).

Arguments and return value for `apply`.

The apply method should probably take as an optional argument an array of nodes as the initial selection, defaulting to [document].

The apply method should similarly return the array of matched nodes. For example,

d3.select("#chart").apply()[0]

should be the same as:

document.getElementById("chart")

Prepend node? Append existing? Insert at given location?

Rename add to append, and introduce a similar method prepend for adding to the front.

Transform inclusion/composition.

There should be a way to define a transform and include it in another transform. Perhaps: a.include(b).

Internet Explorer 6,7,8

I was just looking at how to write a VML renderer for IE but I noticed a few problems:

• querySelector and querySelectorAll aren't supported in IE6 and IE7.
• Array.prototype.slice.call is called with "non-JScript object" causing it to fail in IE8. In the particular case I noticed, it was being called with a document node list (a result from querySelectorAll). I think Array.prototype.slice only works with native Array objects in IE6,7,8.

Personally I'm not bothered about IE support (aside from IE9 of course). This is more of a note to document a couple of things that I found in case they're worth fixing in the future.

Deferred vs. immediate transforms.

Currently, d3 works purely in a "deferred" mode, where you define a transform and then apply it. Nothing gets done until the transform is applied. For example:

d3.selectAll("p").style("color", "red").apply();

One of the gotchas with this approach is that it's easy to forget to apply the transform. The following looks like it should do something, but has no effect:

d3.selectAll("p").style("color", "red");

Another gotcha is that when you apply a transform, you can (currently) only apply the root transform, since that's the only thing that makes sense from a global context. So, if you define a complicated sequence of nested-scope transforms, it only makes sense to apply the transform from the root...

var t0 = d3.select("body");
var t1 = t0.selectAll(...)...
var t2 = t0.select(...).selectAll(...)...
t0.apply();

Calling t1.apply() or t2.apply() currently has the same effect, but looks confusing! The reason we allow calling apply on a nested scope transform is that it avoids the need to pop back to the root transform. For example, this works:

d3.select("body")
    .style("background", "red")
  .selectAll("p")
    .style("color", "green")
    .apply();

The alternative would be something like this:

d3.select("body")
    .style("background", "red")
  .selectAll("p")
    .style("color", "green")
    .pop // and potentially more .pop.pop
    .apply();

Or like this:

d3.select("body")
    .style("background", "red")
  .selectAll("p")
    .style("color", "green")
    .root
    .apply();

Anyway, I feel like being able to call apply on a scoped transform and have it automagically perform the root transform is likely more confusing than its worth. On the other hand, it might be nice to take an arbitrary transform and apply it to a specific scope. And I suppose that scope could default to the single-element node list [document].

So, in addition to the semantics of apply, there's also the question of whether we should provide an immediate-mode form of transforms, where the actions are executed immediately as they are defined. With this approach, you might create a deferred transform explicitly:

var tx = d3.transform();

tx.select("body")
    .style("background", "red")
  .selectAll("p")
    .style("color", "green");

tx.apply();

It's a little bit more verbose for the deferred mode, but it does make things a bit more explicit. And it makes the immediate mode less verbose, since you can omit the apply:

d3.selectAll("p").style("color", "red");

Although, I also think you might want to specify a context in this case, such as:

d3([input]).style("color", "red");

Or possibly:

d3.select(input).style("color", "red");

(This would allow d3.selectAll to take an array, whilst d3.select would only take a single argument.)

This might would have the same effect as saying:

d3.transform().style("color", "red").apply([input]);

Animation sequence support

Add abstraction to simplify staged animations? The abstraction would take a set of transitions (wrapped in function instances) and then oversee their sequential execution. It should enable easy parameterization of stage durations, be cancelable, etc...

Two potential cases:

1. Independent transitions: no element-level chaining. Specify function list and list of durations (or offsets) and run each stage accordingly.
2. Dependent transitions: transitions are chained per-element (e.g., .each("end", next()), allowing staggering to propagate. The transition wrapper function might take a "next" parameter in addition to a duration to avoid hard-wiring the next transition.

Tree traversal.

More tree traversal methods are needed. Potentially:

• parent - select parent node.
• previous - select previous sibling.
• next - select next sibling.
• firstChild - select first child.
• lastChild - select last child.
• children - select all (immediate) children.

Some convenience filtering method might also be useful:

• first - filter selection to the first node.
• last - filter selection to the last node.
• slice(start[, end]) - filter selection to the given subset by index.

Zoom behaviour not working in Firefox

The new zoom behaviour is very nice :-) I'm already using it in http://www.jasondavies.com/poincare-disc/ - but the scroll-to-zoom behaviour doesn't seem to work in Firefox 4.0b11. The example in examples/zoom-pan/zoom-pan-transform.html also doesn't work. (Drag-to-pan works fine).

svgweb support

I've been playing around this afternoon to try and get the streamgraph example working with limited success. The following code will create the root svg element and won't throw any errors, but when it comes to the vis.selectAll("path").data(data0) portion it craps out. In normal mode in chrome calling this will return a bunch of svg path elements. When I switch on svgweb the root flash element is created but is ultimately blank. Calling vis.selectAll("path").data(data0) just returns an empty array.

I think this has gotten beyond the point where I can hope to just tweak a few things and get it working. The potential for working with d3 and svgweb seems high though, since the "level of abstraction" between the rendered output and the language constructs is so minimal. Is it a priority to support svgweb at all?

    function createRoot(width, height, callback){
        if(!window.mmd3counter){
            window.mmd3counter = 0
        }
        window.mmd3counter++
        var id = 'mmd3-'+window.mmd3counter;
        var svg = document.createElementNS(d3.ns.prefix['svg'], 'svg');
        svg.setAttribute('width', width);
        svg.setAttribute('height', height);
        svg.setAttribute('id', id);
        svg.addEventListener('SVGLoad', function(evt) {
          callback(d3.select('#'+id));
        }, false);
        if (window.svgweb) {
          window.svgweb.appendChild(svg, document.body);
        } else {
          document.body.appendChild(svg);
        }
    }
    
    var n = 20, // number of layers
        m = 200; // number of samples per layer
    var s = stream_layers(n, m);
    var data0 = d3.layout.stack().offset("wiggle")(s);
    var data1 = d3.layout.stack().offset("wiggle")(stream_layers(n, m));
    var color = d3.interpolateRgb("#aad", "#556");
    var w = 960,
        h = 500,
        mx = m - 1,
        my = d3.max(data0.concat(data1), function(d) { return d3.max(d, function(d) { return d.y0 + d.y; }); });
    var area = d3.svg.area()
        .x(function(d) { return d.x * w / mx; })
        .y0(function(d) { return h - d.y0 * h / my; })
        .y1(function(d) { return h - (d.y + d.y0) * h / my; });
    window.onsvgload = function() {
                console.log('onsvgload');
              // do stuff now

              console.log('vis')
              createRoot(w, h, function(vis){
                  console.log('path');
                  console.log(vis, data0);
                  vis.selectAll("path")
                       .data(data0)
                     .enter().append("svg:path")
                       .attr("fill", function(d) { console.log('d', d); return color(Math.random()); })
                       .attr("d", area);
                console.log('done with path');
                   window.addEventListener("keypress", transition, false);

                   function transition() {
                     d3.selectAll("path")
                         .data(function() {
                           var d = data1;
                           data1 = data0;
                           return data0 = d;
                         })
                       .transition()
                         .delay(100)
                         .duration(1500)
                         .attr("d", area);
                   }                  
              });            
    }

Tick labels.

Better ticks.

We need date ticks, at a minimum. But, it would also be nice to have a ticks algorithm that considers text metrics, so that we can avoid placing overlapping labels.

Fast singleton/empty selection implementations.

For single-node and empty selections, this could greatly improve performance!

Transition sub-selection should support delay, duration etc.

Currently the transition per-instance parameters can only be specified on the root-level selection of the transition. If there is a sub-selection, it does not have delay or duration method defined so it is not possible to customize these transition parameters.