wadefagen / 91-divoc Goto Github PK

Source code for 91-DIVOC

License: GNU General Public License v3.0

HTML 21.19% Ruby 0.32% CSS 2.46% JavaScript 68.57% Python 7.47%

91-divoc's Introduction

Hi, nice to have you here! 🧡💙

I'm Wade Fagen-Ulmschneider – I am faculty in the Siebel School of Computing and Data Science in the Grainger College of Engineering at the University of Illinois (UIUC). My passion is at the intersection of systems, data, and creating tools to nerd out with both systems and data – particularly through interactive tools and data visualizations. In my work, I work to educate, encourage, and inspire you to change the world. See https://waf.cs.illinois.edu/ for more about my role at UIUC.

Projects

I absolutely love code and there is something ✨ magical ✨ about being able to create something that can impact the world with just a keyboard! There's a few major projects I have worked on in the past few years:

My Massively Open Online Courses on Coursera wadefagen/coursera: "Object-Oriented Data Structures in C++" 100k students, "Accelerated Computer Science Fundamentals Specialization" 40k students, "Ordered Data Structures" 20k students, and "Unordered Data Structures" 10k students.
My 91-DIVOC project wadefagen/91-divoc, which provided data-forward visualizations of COVID-19 data, used by governors of multiple states during the COVID-19 pandemic.
University of Illinois projects and datasets wadefagen/datasets including the GPA visualizations, UIUC LinkedIn Banner Generator, among others.

2024: The Year of FULL SEND

I never want to stop creating, and each year I choose a theme to define the year. This year, my theme is to FULL SEND. Some projects that I've already started working on:

Mastery Platform, to re-imagine how homework and learning can be done at The University of Illinois and beyond.
Data Science Clicker, because a great student response system should be simple.
visualized, to get back to my visualization roots!

Come by and say hello and lets nerd out! :)

91-divoc's People

Contributors

Stargazers

Watchers

Forkers

binary-eater covidvis karenerobinson titaniumbones davidmalcolm windwardapps stevebissett gavinr kenuiuc vineet-sinha battyone bunnymatic bbodenmiller tobes knowlesdominique allenmiller simoncozens mar1965

91-divoc's Issues

please add critical and death grap by Country, normalized by country population

Feature Request: Logistic Curve Trend line

Since you have got an exponential curve trend line, perhaps it would be interesting to attempt to fit the data onto an logistic curve as this is the real model by which infection follows.

Normalize the 'start point' too!

Great work! Thank you!

When normalizing by Country, could you also 'normalize' the 100-cases starting point? This would probably get rid of a lot of the vertical shift between the curves. Even though it sais so at the bottom of the chart, the data does not look as if it was normalized in that way.

91-DIVOC/pages/covid-visualization/index.html

Line 149 in f780d12

<h4>Same Data -- Normalized by Population</h4>

Add previous days growth value to doubling trend lines / moving these to a legend.

Ie:

Doubling every two weeks (0.05).

Doubling every week (0.10)

Alternatively, have these listed in a legend on the right, or at the bottom to reduce an already highly dense graph.

Feature request: smoothing for per day numbers

Either add a selection for smoothing, or add a smoothed line to the selected country's display.

This is especially important for the "per day" statistics as delays / inefficiencies in the stats flow to central publishers means sawtooth graphs: low number one day followed by high number the next. All artefacts of the stats collection process not the "real" situation.

Graphs do not reflect select

Thanks for this great visualization - I have a small usability issue

When the page is refreshed graphs and the select options do not match. FF75, linux

Please either force the selects to the default or use the values in the select to draw the graphs.

mouseover effects

When I mouseover a data point, I get a tooltip. Thanks.

It also would be nice if all of the points for the state would be highlighted on mouseover.

Clicking should change the selection.

Animate for states bug: Uncaught TypeError: Cannot read property 'cases' of undefined

Animation doesn't work on New Confirmed COVID-19 Cases per Day by US States/Territories

At http://91-divoc.com/pages/covid-visualization/ when trying to animate the New Confirmed COVID-19 Cases per Day by US States/Territories, console throws error: Uncaught TypeError: Cannot read property 'cases' of undefined

Happens for every state but some examples:

http://91-divoc.com/pages/covid-visualization/?chart=states&highlight=Minnesota&show=us-states&y=both&scale=linear&data=cases-daily-7&data-source=jhu&xaxis=right#states

http://91-divoc.com/pages/covid-visualization/?chart=states&highlight=Arkansas&show=us-states&y=both&scale=linear&data=cases-daily-7&data-source=jhu&xaxis=right#states

Highlight==None results in broken graph

On this page: http://91-divoc.com/pages/covid-visualization/

It looks like the y-axis goes to a default very small scale so all the data is clamped to the top edge of the graph. This is resolved by setting the y-axis manually, but it defaults to "Scale to Highlight and Cur. Max".

I suggest that if the y-axis is set to Scale to Highlight and Cur. Max and the user sets the highlight to None, you should automatically change the y-axis to Fixed.

Switzerland is missing from population-normalized graph

In the third graph "COVID-19 Cases by Country, normalized by country population", Switzerland is missing from the country-highlight dropdown

Please add South Africa to population normalized graph

Please add South Africa to population normalized graph!

Our lockdown start tomorrow midnight.

Per state graphs for countries other than the US

Seeing that time_series_covid19_confirmed_global.csv (and deaths and recovered) include per state data for a number of countries outside the US, would you please consider adding a "country" dropdown to the "by US States/Territories per Day" graph where you can select one of the countries that has per state data?

By 2016 election results per county

In addition to the cases by governor partisan affiliation I would like to see how a county voted in 2016 vs infection rate. An eyeball here in WA says that there is a strong correlation between the red side of the state and the most effected counties.

#01: indicate when stay at home orders, wear mask orders, etc. went in to effect

Some states appear equal up until a certain point when normalized. Would be cool to see when various distancing measures went in to effect to try and draw some basic correlations.

Why was the growth rate changed back and forth?

1.35 was somewhat useful, but then it was changed to "doubles every 2 days, 3 days, 1 week, 2 weeks, etc." -- This was VERY useful, but now it is back to % daily increase. Was this an oversight or is there something I am missing about why perhaps the other "models" didn't work? I found them absolutely gamechanging for conversation and debate with the "timeframe to double" overlays.

Total Active Cases are missing in some countries

Some countries are missing Total Active Cases updates after their numbers dropped down to ~ 100.

Not all countries showing up in Normalized by population charts

Several countries (India for example) missing in the charts normalized by country population.

menu item for Deaths vs year ago in addition to Deaths

Two candidates for addition to the Data: popup
Deaths vs year ago
unreported deaths %

https://www.economist.com/graphic-detail/2020/04/03/covid-19s-death-toll-appears-higher-than-official-figures-suggest

Y-axis scale to highlight, takes into account out-of-window data

Scale to highlight takes into account all data points of the highlighted states. But, when using right-align, some of the data may be out of the window.

This means that it's hard to see current variations for data which values have decreased a lot since their global maximum. For instance:

This issue will be increasingly important as more state/countries squash their epidemy (hopefully).

This problem also impacts visualization of test positivity, since some states had very noisy data in the early stage of the epidemics. The best example is probably California:
https://91-divoc.com/pages/covid-visualization/?chart=states&highlight=California&show=highlight-only&y=highlight&scale=linear&data=testPositivity-daily-7&xaxis=right-4wk#states

This could also be mitigated if the user could select the bound of the y-axis themselves.

As always, thanks for making such a great tool available for free!

Mortality rate: plot new deaths with 2 weeks lag from new confirmed cases

The idea is to plot a ratio of deaths against the new confirmed cases with a 2 weeks lag, or whatever the current known average "from infection until death" time span is. Basically, to show mortality rates among those who get infected on each particular day.

The expected behavior is that there will be an almost horizontal line (probably only when smoothed), potentially with a "hill" where there are too many people sick at once and a health system has no more capacity to handle this surge.

I read claims that because of different countries counting deaths differently (some even suspect hiding the actual number of deaths in some non-democratic countries), we see differences in numbers. While this graph would not reveal the reasons for differences, it can potentially show surges in mortality rates, indicating that health systems' capacities have been exceeded.

Japan shows US

On the COVID-19 Cases by Country, normalized by country population graph choosing Japan shows data for US

Edit: So it is about not having data for the specific country

Suggestion for how to explore data

According to some fairly solid work published in the Lancet, the overall fatality rate, under best conditions is about 1.4%, and it takes about 18 days from onset of symptoms to a fatal outcome. So, if we know current death count, we could go backwards by some offset of an estimate of mean days between test and death (18 at best case, likely a good bit less), and then extrapolate two things - first, if I have say 9267 dead now, then assuming hospitals are not completely a mess, it took about 662,000 cases to produce that outcome. If we then actually had 662,000 cases in the US 18 days ago, then we knew about 6374 cases on day 13, so we had about a 1:100 underestimate. Say we then go up to 10 days ago, we knew about around 65k cases, meaning we had an underestimate of around 90% at that point.

Don't know if any of this will make it into graphs, just something to perhaps poke at. Might be nice to add a page for experimental graphs, since the front page is getting a lot of attention.

feature request: new cases as a percentage of tests

It would be useful to have an option to see the number of new cases as a percentage of tests. This could be shown on a daily basis, and with a 7 day average

Y-axis scale on linear graphs

Thanks for this project! The linear graphs show most of the data squeezed down near the bottom. Is it possible to rescale the Y-axis? I suspect the 35% line may be the culprit.

"New Deaths, 1 week average" actually averages last 7 data points

If data is missing for a day, or it's below the cutoff, the average does not update, which makes it look as if the weekly average isn't dropping. See eg Washington normalized for the few days leading till 2020-04-23.

Request Option to Normalize on Metric per Population per AREA

We talk a lot about normalizing on population, but how about normalizing on population DENSITY?

It would look the same as normalized on population but simply divide the metric/population by the area of the geographic region for another dropdown or pair of charts. Population density plays a huge role in the ability to spread so I think it could be a useful/interesting perspective on the pandemic.

Could you add days since lock down/stay at home orders (or some measure of preventative action)? Or maybe color code or allow a hinge in a fit line?

Feature request: adding EU to the graph normalized by population

I like that EU was included in the first graph. That's a meaningful supra-national entity, and a good comparison with the United States. Unfortunately, they don't seem to be included in the 3rd graph (normalized by population).

Their population, according to wikipedia is 447,206,135

California New Deaths/Day is missing from "COVID-19 Cases by US States/Territories, normalized by population"

In the chart titled "COVID-19 Cases by US States/Territories, normalized by population"

"California" is missing from the dropdown list, if I try to choose "New Deaths/Day"

California comes back for other items

Feature request: display at least 2 significant digits (when hovering on data point)

When I monitor the deaths in the European Union, normalized by population, it is hard to see if it's going up or down. The average has been 0.2 for the last few weeks. But that could be anything from 0.151 to 0.249. So, a constant number could hide the fact that deaths halved or doubled in the last few weeks.

The more successful a country, the more noticeable the problem: for Japan, the graph says 0 deaths/1m inhabitant even though it's non-zero (closer to 0.001).

#02: normalize data based on population

Similar to normalized data on #02, would be great to have normalized data on #02.

secondary axis using "additional data" mixes datasets using national + state

This is kind of complicated so I'll keep the text description short and let the screenshots do the explaining.

I think there's a bug in how...

secondary axis data is mis-labeled, in this instance
incorrectly labeled secondary series seems wrong or maybe normalized using primary series' datum?
...while comparing both cases, and deaths, for the US as a whole, plus Arizona

Parameters for this setup:
http://91-divoc.com/pages/covid-visualization/?chart=countries-normalized&highlight=United%20States&show=highlight-only&y=both&scale=linear&data=deaths-daily-7&data-source=jhu&xaxis=right-8wk&extra=Arizona&extraData=cases-daily-7&extraDataScale=separately#countries-normalized

USA&AZ: [new deaths; new cases] (normalized; 7d avg)

The individual charts for deaths and cases are as follows:

USA&AZ: new deaths (normalized; 7d avg)

USA&AZ: new cases (normalized; 7d avg)

I created a Canadian Version - issue not an issue

I'm too far removed to create a fork of this project but wanted to inform of my "fork".

With a few modifications I have created a Canadian version of this web page, displaying provinces instead of states.

I have put it up on Github Pages:
https://esklarski.github.io/canada.covid19/

and my code is here:
https://github.com/esklarski/canada.covid19

Thanks for sharing your code!

Active case count for the USA has been stalled since March 22nd

All future entries for the USA have a 0 in that column!

Feature request: Add date to data point tooltip

It would be nice to add the actual date in the tooltip. This would help correlate points on the curve to real-world events.

Observed behavior: tooltip says "Day 17"
Requested behavior: tooltip also says "March 23" or whatever the date should be.

(The website is very cool and provides information that I couldn't find anywhere else.)

I suggest that instead of 35%, the chart should say 1.35x

Much clearer.

Line color changes during animation

During the animate function, the legend color seems to stay with the highest-ranked data point instead of staying consistent across the simulation.

This makes it difficult to track a specific line through the animation.

Feature Request: Normalize by 100,000 instead of 1,000,000

For state level graphs numerical stability would be better if normalized by 'per 100,000' rather than by '1e6'

Daily Test Positivity 1-week average shows the average of the last 7 percentages, not the true average positivity rate

Being in Arkansas, I noticed the 7-day test positivity averages seemed too high compared to the daily total tests and positive results. It appears that the state sometimes reports low numbers of tests with high or even 100% positive results, and this is skewing the 7-day average. It appears that the chart is averaging the last 7 percentages, regardless of the numbers of tests, so the days with 100% positive results on a lower number of tests is pulling the average far higher. The 7-day average of of this ratio should instead sum the number of positive tests over the last 7 days and divide by the total number of tests over the last 7 days.

Add some data smoothing to new cases/day

It seems like new cases/day is very volatile with lots of ups and downs, I am guessing this is probably because the labs might be reducing staff/testing capacity on some days of the week. It might look more useful if it was "new cases/week" or avg. daily cases (for previous weeks) and new cases/day for the current week.

Add Country/State Filters

Could you add a filter to limit the amount of countries being shown. I would suggest by a few large groups:

For countries

by continent
OECD

I am dubious about the quality of data coming out of most non-OECD countries.

And maybe some aggregates for countries and states

Feature request: allow adding sub-national regions to nations

Subject sort of says it all. It might be useful to compare (with the same start date thresholds ) a State to a county, for example. The alternative would be able to add countries together (like what does the EU look like compared to the US, for example).

Thanks for a great visualization tool!

Scatterplots vs. population

How about some scatterplots (cases, deaths) vs. population of each reporting unit (country, state) so we can know when normalizing by population becomes more useful?

See https://twitter.com/jburnmurdoch/status/1244380095164420101?s=20

Feature request: Plot percentage change per day per country

Instead of absolute numbers or population normalized values, it'd be useful to see percentage change values for each day.

The vertical bar might show the range -20% to 50%, and one might notice some countries are grouping around 2-3% daily change while other countries might be doing better or worse.

Thanks for this super useful tool.

Trend lines are too noisy to be useful

The new trend line feature is great and very helpful, but... not very trustworthy at all.

It looks like it's implemented by drawing a line on the log chart between the current value at day N and the value at day N-7, ignoring all the data points in between.

That's... kinda just wrong. It ends up being very sensitive to the sampling error at both of those points, which for some data sets is really high (c.f. France's occasional burps where they seem to batch 2-3 days into one day of reporting).

What you need to be doing is fitting a line to those 8 data points. Just a dumb least squares fit of a line to the log chart would be infinitely better than what we have.

It's pretty clear that this is March, 27 in US date format.

Soon images will be showing something like "Updated 4/3/2020" - this will be interpreted as April, 3 by US folks, and 4th of March by every other nation, English speaking or not.

Best solution here is to switch to ISO date format:

https://www.w3.org/QA/Tips/iso-date

https://www.ctrl.blog/entry/date-formats-web.html