Deeper pandas integration

[hnykda]

Deeper pandas integration

[chriddyp]

Here's a new library by @santosjorge for dealing with this: cufflinks: Productivity Tools for Plotly + Pandas.

There's a great notebook here: http://nbviewer.ipython.org/gist/santosjorge/cfaaf43b40db19d6127a

Looks great!

[chriddyp]

+1! Deeper integration with Pandas will be awesome.

Right now, we just handle Pandas Series (see here: https://github.com/plotly/python-api/blob/master/plotly/utils.py#L71-L78)

And to start, we can add more Pandas data types to that encoder, like DatetimeIndex (convert to datetime with .to_pydatetime()).

I imagine we could also wrap up some nice helper functions into the plotly.tools module, like plotly.tools.pandas2plotly(), that would take an intelligent guess of what kind of graph to make. So, instead of:

import plotly.plotly as py
from plotly.graph_objs import Data, Scatter
import numpy as np
import pandas as pd

N=10
ts = pd.Series(np.random.randn(N), index=date_range('1/1/2000', periods=N))
df = pd.DataFrame(np.random.randn(10, 4), index=ts.index, columns=list('ABCD'))

scatter_figure = Data(
    [
        Scatter(x=df.index, y=df[series_name], name=series_name)
        for series_name in df
    ])

py.plot(scatter_figure)

we might have something like:

import plotly.plotly as py
from plotly.tools import pandas2plotly
import numpy as np
import pandas as pd

N=10
ts = pd.Series(np.random.randn(N), index=date_range('1/1/2000', periods=N))
df = pd.DataFrame(np.random.randn(10, 4), index=ts.index, columns=list('ABCD'))

scatter_figure = pandas2plotly(df)

py.plot(scatter_figure)

with other chart types supported with something like:

figure = pandas2plotly(df, type='box')

There is nothing in the way of providing the plotly versions of the wonderful pandas plotting tools like scatter_matrix and andrews_curves.

figure = plotly.tools.pandas.scatter_matrix(df)

The neat thing about the pandas tools is that they would just convert the dataframes into plotly graph_objs, making it pretty easy and familiar to customize the style of figure.

[hnykda]

This just sounds great! If I could kiss your mind...

You basically described all the functionality I would expected. Pandas has some basic plot functions, but they return matplotlib.axes.AxesSubplot object, so it is not so easy to use pyplot.plot_mpl(figure) to convert them to plotly.

If I think of something interesting, I will add it here.

[chriddyp]

Python's ggplot is great with dataframes, too:

Plotly approach:

import pandas as pd
import numpy as np

cols = ["x", "y"]
ndf = pd.DataFrame(np.random.rand(200,2), columns=cols)
ndf["var"] = ndf.index > 100

# Plotly approach
import plotly.plotly as py
from plotly.graph_objs import *
py.sign_in("DanielHnyk", "30ojpz2ff1")

df0 = ndf.iloc[:100,:]
df1 = ndf.iloc[100:,:]

trace0 = Scatter(x = df0["x"], y = df0["y"], mode="markers")
trace1 = Scatter(x = df1["x"], y = df1["y"], mode="markers")
data = Data([trace0, trace1])

py.plot(data)

ggplot approach

from ggplot import * 
g = ggplot(aes(x="x", y="y", color="var"), data=ndf) + geom_jitter()
g

Our matplotlib convert should be able to convert plots made with ggplot, so I don't think that we should put much effort to re-inventing that wheel.

[hughesadam87]

Farence,

As it stands with the fig, axes problem, a simple workaround is to create a figure and axes with subplots and pass the ax in, then everything works. Annoying, but lots matplotlib-compatible libraries tend to pass around axes object only. Also, at any given time, you can do plt.gcf() [get current figure].

So my first solution that works fine is:

fig, ax = plt.subplots(1,1)

df.plot(ax=ax)
iplot_mpl(fig)

Alternatively, this one should work too (unteseted)

df.plot()
fig = plt.gcf()
iplot_mpl(fig)

We have been doing a lot of pandas/plottly cross over in our lab. In truth, I don't think there would be any benefit to deeper plotly/pandas integration, unless you want streaming plots, or other plots that are much more complex than simple iplot_mpl() wrapping. I came to this conclusion the hard way after building a small plotly API for a library that's based on pandas, realizing it could be completely circumvented using iplot_mpl() and the support proposed in #28. Of course it's up to the plotly developers, but if they supported these pandas conversions, then that's one more API to keep up, which seems to go against plotly's goal of being a single nexus for integrated visuals. (I'm not a plotly developer, so this is just my 2 cents)

[hnykda]

Understandable. Thank you for answer

[msund]

From someone who has been using Plotly with pandas. @theengineear & @etpinard.

"For us, it’d be great to rather have something like the pandas2plotly converter as outlined by @chriddyp. I’m planning on using it for default plots or plots requested by the user controllable only via a simple options dictionary forwarded from the user’s data submission.

This dict would be defined as expected by the keyword arguments of df.plot; see here for how I’ve solved this so far using Pandas default plot function and plotly’s plot_mpl. This strategy would need a bit of fiddling with the layout, though, to make it look nice in plotly which I’d like to avoid. I’m hoping a pandas2plotly converter could help with this."

[hughesadam87]

Awesome, thanks for sharing

On Mon, Jan 12, 2015 at 6:33 PM, chriddyp [email protected] wrote:

Here's a new library by @santosjorge https://github.com/santosjorge for
dealing with this: cufflinks: Productivity Tools for Plotly + Pandas
https://github.com/santosjorge/cufflinks.

There's a great notebook here:
http://nbviewer.ipython.org/gist/santosjorge/cfaaf43b40db19d6127a

Looks great!

—
Reply to this email directly or view it on GitHub
#18 (comment).

Adam Hughes
Physics Ph.D Candidate
George Washington University

[jonmmease]

Closing this for now. plotly.py does support DateTimeIndex values now, and the cufflinks project has become an established way to build plotly plots from pandas data frames. Feel free to open a new issue if anyone would like to discuss additional pandas integration ideas!