nteract/dx

Data Explorer for Python

the interactive computing suite for you

nteract is an open-source organization committed to creating fantastic interactive computing experiences that allow people to collaborate with ease.

We build SDKs, applications, and libraries that help you and your team make the most of interactive (particularly Jupyter) notebooks and REPLs.

To learn more about the nteract open source organization and the rest of our projects, please visit our website.

What's in this repo?

This repo is a monorepo. It contains the code for the nteract core SDK and nteract's desktop and web applications. It also contains the documentation for the SDK and the applications. Here's a quick guide to the contents of the monorepo.

Folder	Description
applications/desktop	Source code for the nteract desktop application. The desktop application is a cross-platform app built using Electron.
applications/jupyter-extension	Source code the nteract Jupyter extension. This extension can be installed alongside Jupyter classic and JupyterLab in your Jupyter deployments or personal Jupyter server.
packages	JavaScript packages that are part of the nteract core SDK.
changelogs	Changelogs for each release of the nteract core SDK and applications.

How do I contribute to this repo?

If you are interested in contributing to nteract, please read the contribution guidelines for information on how to set up your nteract repo for development, how to write tests and validate changes, how to update documentation, and how to submit your code changes for review on GitHub.

How do I use the nteract core SDK?

If you are a developer who wants to build an nteract-based notebook application, check out the following documentation resources for more info.

Link	What's in it?
docs.nteract.io	This page contains our how-to and tutorial style documentation. Get started learning about the nteract core SDK here.
packages.nteract.io	This page contains the API documentation for packages in our core SDK. Bookmark this and use it as a reference when building your nteract-based UI.
components.nteract.io	This page contains the documentation for our suite of composable React components. It contains code samples that you can reference when building your nteract-based UI.

Our documentation is living. We are always making changes and adding more content. If you have feedback about the documentation, please open an issue in this repo. If you are interested in submitting a change to our documentation page, please review the contribution guidelines and submit a pull request.

How do I use the nteract desktop application?

To get started with the nteract desktop app, head over to the nteract homepage to download the application for your operating system.

Once you've download the app, head over to our documentation page for tutorials and guides on using the app for your data analysis and science workflows.

Supporting nteract

nteract is a non-profit open-source organization fiscally sponsored by NumFOCUS. If you are interested in supporting development on nteract, please consider making a recurring donation.

Development on nteract is also supported by the following organizations.

dx's Issues

Doesn't Run in MyBinder?

Aside from the missing pandas dependency, which is easily rectified, the demo notebook doesn't display an explorable dataframe in MyBinder, it just displays a flat pandas df as normal. Any particular reason why it shouldn't work there?

Dependency update pandas/pydantic/SQLAlchemy

• Support request for pandas > 2, pydantic > 2 & SQLAlchemy > 2

Create a README

Include:

• Screenshot(s)!
• Installation instructions
• Development setup instructions

Option to pass initial metadata

Related to nteract#4377, it would be nice to be able to pass in initial metadata, something like this:

dx(df, metadata={
  'view': 'scatter',
  'selectedMetrics': [
    'Economy (GDP per Capita)',
    'Family',
    'Health (Life Expectancy)',
    'Freedom'
  ],
  'chart': {
    'metric1': 'Happiness Score',
    'metric2': 'Economy (GDP per Capita)',
    'metric3': 'Freedom',
    'dim1': 'Region',
    'dim2': 'Region'
  }
})

Run in Jupyterlab

Blasphemy! Cats and dogs living together under one roof!

The nteract Data Explorer is brilliant! It's the demo I show folks all the time when I talk about the future of notebooks about what will be achievable with interactive data prep in notebook environments.

Unfortunately we're a Jupyterlab shop. It'd be great to be able to be a member of the data explorer community.

Clarify in README dataframe support

Probably should be explicit that this is currently Pandas dataframe support.

Option to trim other MIME types

Toggling on the pandas option display.html.table_schema only adds a new MIME type alongside the text/plain and text/html representations of similar size. This is a problem because people tend to output large amounts of data for use with the Data Explorer, so a significant amount of space is potentially used up by the other MIME types unnecessarily. It would be nice to have an optional parameter with the dx() function to trim the other MIME types to optimize space.

Example:

dx(df, trim_output=True)

This would remove the text/plain and text/html representations from the resulting MIME bundle.

The only downside I can think of is that if someone wants to view the notebook in another UI that does not support the Data Explorer, nothing can be shown because there are no other representations in the output. While it might be good to have the option default True, the user could at least have the escape hatch of setting it to False in case preserving the other representations is important to them (though unlikely since seeing hundreds or thousands of rows is probably not what they want).

clean up schema files from data-explorer

We should update the data-explorer schema files to use for code gen of python models.

convert to file later

dx/dx/codegen.py

Lines 8 to 13 in ebbae18

	# TODO convert to file later
	json_schema: str = """{
	"$schema": "http://json-schema.org/draft-07/schema#",
	"title": "Person",
	"type": "object",
	"properties": {

---

This issue was generated by todo based on a TODO comment in ebbae18 when #24 was merged. cc @willingc.

Option to configure max_rows

It would be a nice addition to allow passing a max_rows parameter.

dx(df, max_rows=None)

This would essentially be a convenience mechanism so the user can avoid doing the following.

with pd.option_context('display.max_rows', None):
  dx(df)

The default value could potentially be None, but I think it's better to have users explicitly set that so they are more likely to realize the ramifications.

Prep 0.3 release

Rely on Vega/Vega-Lite for visualizations to generate outputs portably

Vega and Vega-Lite are a set of visualization grammars interactive data viz designed by the UW Interactive Data Lab. They're the underlying language and grammar that power Python's Altair and R's ggviz (the interactive successor to ggplot2).

Vega-Lite defines a chart by pointing it at a dataset and declaratively expressing how it should be plotted and interacted with in JSON. It then compiles down to D3. But because it's expressed in this json-based grammar, it's highly portable. So something done in Python Altair is portable to ggviz or just renderable on its own.

Here's where dx comes in: imagine a data scientist finds the plot they're looking for from the data explorer. How do they save it? How do they preserve the visualization as part of the notebook? How can colleagues see the specifications that went into the plot? How can they manipulate it with code? Vega-based outputs could fill this gap, preserving the steps a user did in a GUI exploration, even as the underlying data changes.

I get that this is an absurd ask, but I feel like it's worth a shot. Preserving these explorations in outputed notebooks is a worthy cause.