Investigate perf issues with auto_viz

[jupyter-incubator]

Investigate perf issues with auto_viz

[msftristew]

Marking this as a bug because Alejandro says he has seen performance issues with auto viz.

[msftristew]

IMO 2500 is a good baseline but it seems to me that that's not all that many rows at all. I don't think a graphing mechanism that can only deal with a few hundred rows is very useful. I'll investigate this this week.

Two possibilities:

1. We're doing something wrong with datasets of this size. That's possible, and we can fix it.
2. Plotly isn't good at datasets of that size. This is the tougher situation but if we can identify the cause, we can make a contribution.

[msftristew]

In my brief testing I've done today, autoviz isn't suitable even for datasets of 100 rows. I will have a look at the code.

[msftristew]

I did some testing on basic plotly graphs in the browser locally (without the magics or anything) and all of the graphs were reasonably performant even when dealing with thousands of rows except the pie graph, which gets super slow with more than a couple hundred rows. This is not really a big deal since you shouldn't have a pie graph with that many rows anyway (there are only 360 degrees in a circle). On our end I think it makes the most sense to detect when a pie graph would have too many rows and throw an error when that happens ("Your result set has more than 200 rows; please reduce the size of your dataset for better visualizations".) When I push the change to fix the way labels and datasets are being computed for pie graphs, this will be alleviated partially.

What this means on our end is that if it takes a very long time to render line graphs and bar graphs (etc.) that are more than a few hundred rows large, it's our fault, because plotly can handle it.

[aggFTW]

So after some investigation, it seems like Table representation and Pie graphs have perf issues.

Plotly Table takes ~42 seconds to render 60 rows or so. I believe the bottle neck is on the JS side of it. Opened plotly/plotly.py#383 in Plotly

Then, qgrid is not compatible with Plotly (see plotly/plotly.js#90), so the only alternative with good perf is default pandas df html representation.

[aggFTW]

There's performance issues with a large amount of slices for pie graphs.
* 1500 rows crash the browser.
* 500 rows take ~15 s.
* 100 rows is almost automatic.

I'll go with (configurable) 100 rows.