Comments (21)
@mbostock Any more recent thoughts in this area? I need some line smoothing for a time series of continuous data in my application, and would love to contribute something.
from d3-shape.
I think it might be a good idea to do smoothing as a pre-processing step in data space rather than doing it internally as a curve:
- Data space seems like a more natural fit when specifying kernel bandwidth.
- You might want to compute a smooth fit once before interactively redrawing (e.g., pan & zoom).
- You might want to access points in the computed smooth fit for display (e.g., hover tooltip).
I’m imagining something that has a x and y accessors and other configuration (e.g., kernel and bandwidth) and then computes the smooth curve, generating an array of [[x1, y1], [x2, y2], …] points in data space. Those can then be passed to a line generator, likely with an x and y accessor that apply scales to transform to screen space.
(I’m slightly tempted to have it return [[x1, x2, x3, …], [y1, y2, y3, …]] because that’s a more more efficient representation, but it would make implementing the accessors a little bit more awkward, so it feels like premature optimization.)
from d3-shape.
Here’s a simple moving average implementation:
https://observablehq.com/@d3/moving-average
from d3-shape.
A kernel regression is another possibility, as in this kernel density estimation example. That implementation is O(n^2), but I suspect you could make it O(n lg n), or even O(n) if you assume that data is already sorted on x, and then implement a single-pass, fixed-window incremental algorithm.
from d3-shape.
LOESS is another option.
from d3-shape.
And here’s a simple test case:
With a smooth curve in R:
from d3-shape.
Yes! When I first saw this visualization I thought of trying to implement a gaussian KDE for use with D3.
from d3-shape.
I couldn’t resist taking a crack at it. Here’s the slow O(n^2) implementation of kernel smoothing:
Things that would be nice:
- Make it faster than O(n^2) by tracking a fixed window of input points (in x ± bandwidth).
- Support kernels besides Epanechnikov.
- Remove the scale factor 0.75 from the default Epanechnikov kernel, since it has no effect.
- Maybe pick a default bandwidth of 20 (fast)? Or Silverman’s rule of thumb (slow; global)?
- Maybe pick a default precision of 10 (distance in pixels between computed spline control points)?
- Remove dependency on d3_scale.
from d3-shape.
I’m going back to working on API documentation if anyone else wants to make a better implementation…
from d3-shape.
Doing a little research, here's a list of common kernel functions
- Uniform
- Triangular
- Epanechnikov
- Quartic
- Triweight
- Tricube
- Gaussian
- Cosine
- Logistic
- Silverman
Also, here's a formula for bandwidth estimation using "Silverman's rule of thumb".
from d3-shape.
See also science.js, which implements bandwidth estimators and various kernels.
from d3-shape.
My fit above, by the way, is the sum of two gaussian PDFs, plus the sum of the same gaussians for x-24 and x+24, all raised to a power, for no good reason other than that it looked about right. I'd love to know a better way than eyeballing it and then applying general curve fitting to guess the components of things that look like the sum of gaussians.
from d3-shape.
See also ggplot2’s stat_smooth, which uses LOESS and GAM (generalized additive models). And ggplot2 has a geom_smooth which can smooth using a ribbon, showing an upper and lower bound on the smoothed model. That might be a nice curve type for an area shape.
from d3-shape.
Here's the Loess + D3 example from science.js by @jasondavies . I made a bl.ock to study it:
It looks like the API expects loess(xValuesArray, yValuesArray)
and returns an array of smoothed Y values.
from d3-shape.
Since the kernel operates in pixel space, I bet it would be reasonable to just have stupid defaults for the bandwidth (±20px) and precision (control points every 10px). Though certainly it would be nice if Silverman’s rule of thumb were easy to implement if you wanted a better default value, albeit at the cost of a global computation.
from d3-shape.
Here's an example that adds a LOESS curve to your tweet example using science.js:
I made this to try to understand the science.js LOESS implementation. Here are a few things I learned:
- It was necessary to sort the data by time. I thought it would be sorted by time already (based on the order from the CSV file), but somehow it doesn't work without first sorting by time.
- The
loess
function accepts arrays of values in data space, and surprisingly it works with dates. - The
loess
function returns an arrays of smoothed Y values that is the same length as the original arrays. I would have hoped that the length of the returned array is configurable, but it looks like in this implementation it is not.
from d3-shape.
+1 on the idea that this should be a data space operation.
from d3-shape.
Probably should create a d3-smooth module for smoothing functions.
from d3-shape.
Hope you don't mind me commenting but I found my way to this after looking at the issues on science.js.
I'm sure things have moved on since December, but I implemented a kernel smoothing function for use with D3 a while back. It cuts off the calculation for each point once the calculated weight goes below a defined threshold (with relation to the weight at the point that is being smoothed).
- It looks like it scales linearly but there is a trade-off in accuracy. So for the twitter data in question it’s about an order of magnitude quicker (depending on the bandwidth) but gives a line that can be up to 0.03% different.
- It assumes that data is sorted on the x-coordinate and hence that the calculation can be cut off.
- It assumes that the kernel function will always decrease beyond the cut-off point.
I'm fairly new to JavaScript so I'm sure that it’s not the best implementation. Anyway, I've created a gist if it’s of any interest (or use).
from d3-shape.
d3-regression implements loess
https://github.com/HarryStevens/d3-regression
from d3-shape.
This feels more appropriate to do in data space rather than geometrically, so I don’t think d3-shape is the right place for this. Still interested in seeing progress here, though.
from d3-shape.
Related Issues (20)
- Arc - Inconsistent error notification when invoked via fluent function calls vs invoking with data HOT 1
- Dynamic smoothing interpolation HOT 2
- Possible typing issue or a missing example (How to write a custom accessor?) HOT 1
- lineGenerator fail
- es5 to es6+ syntax? HOT 1
- The pie arguments problem HOT 1
- d3.arc startAngle and endAngle are switched when startAngle > endAngle HOT 8
- the BumpRadial curve only works with two-point lines
- Needs connection to me not fast enough!
- How does Catmull Rom work with a Bézier curve? HOT 2
- The types for Arc are incorrect HOT 1
- Unexpected arcs when corner radius is nonzero HOT 1
- curveBumpX creates a circle in Chrome if applied to a straight line. HOT 2
- How to add pointer effect to the shape? HOT 1
- running tests with d3-shape HOT 1
- Configurable precision for generated path data HOT 1
- startPadAngle and endPadAngle HOT 4
- Curve Natural gets outside the svg HOT 1
- add optional angle radius parameter in curbeBezier `curveStep` curves HOT 4
- option for arc's curve precision
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from d3-shape.