Time-of-day scale?

[d3]

Time-of-day scale?

[mbostock]

Yes, the way I typically solve this is I just pick an arbitrary day (typically January 1, 1900 due to the default behavior of d3-time-format), and then map all of my dates to that day. The simplest way of doing that (in D3 3.x) is something like this:

var parseTime = d3.time.format.utc("%H:%M").parse;
var midnight = parseTime("00:00"); // "Mon, 01 Jan 1900 00:00:00 GMT"

Then, make a time scale for the day:

var x = d3.time.scale.utc()
    .domain([midnight, d3.time.day.utc.offset(midnight, 1)])
    .range([0, width]);

The result:

I’m not sure it’d be possible for a scale to do this for you automatically, since it’s more of a data-processing step to pick a consistent representation for time-of-day (given that JavaScript only provides a natural representation for absolute time). I suppose you could standardize those arbitrary intervals, though, and then maybe do some modulus arithmetic so that any date passed to the scale outside the standard interval was converted. At any rate, the current solution doesn’t feel terrible.

There’s an interesting (and common!) problem with this particular use case, which is that by aggregating the data in UTC, you blur the local pattern slightly since the local time zone offset changed over the course of data collection. That can only be fixed by changing how the raw data was aggregated, though, since at this point the counts have already been assigned to the wrong bins.

[curran]

Thank you for this explanation and example. It makes total sense. Indeed, using JavaScript Dates to represent relative time does not feel terrible. I'm learning lots of things from studying this example, like passing negative values to tickSize, and how to measure text. Also, that's a great point about using local times before aggregation to show the pattern more clearly.

I noticed that your normalization does not account for the aggregation over 521 Twitter users. The Y axis shows the number of tweets per hour summed over that whole group, but if you divide by 521 when assigning d.rate, then the Y axis would be the average tweets per hour per user.

As a side note, the Vega-Lite project has relative time units in its roadmap.

Closing the issue, as I think the fundamental answer here is "it’s more of a data-processing step to pick a consistent representation for time-of-day (given that JavaScript only provides a natural representation for absolute time)". Thanks again.

[mbostock]

I noticed that your normalization does not account for the aggregation over 521 Twitter users.

Right. I’m plotting the rate of Eric’s feed (i.e., the number of tweets he needs to read every hour to not miss out). You could divide by the number of followers to estimate their average tweet rate (i.e., how often one of his followers typically tweets), but I doubt that this rate follows a normal distribution so I don’t think the average would tell you much. It’s more likely it follows a power law, with some users tweeting far more frequently than others and being responsible for a disproportionate part of his feed.

[curran]

Haha what a great article on "Fear of Missing out"! I never heard that term before.

I see what you mean. Interesting insight that tweet rates probably follow a power law. Thanks for clarifying.

[e-n-f]

@mbostock Yes, not too far from a power law, although the people at the top aren't as prolific as you'd expect, and there are a bunch of people credited with one tweet that I've never even heard of, so I don't know what they're doing in my API responses.

There's probably a better way to model it but it's not jumping out at me.

[mbostock]

Neat, Eric. Thanks for sharing!

[curran]

Nice! I wonder if those one-tweet people are from those "Promoted" tweets that Twitter injects into everyone's feed.