Add converters for tabular data

[jupyterlab]

Add converters for tabular data

[saulshanabrook]

@Nestak2 You have to set pandas.set_option('display.html.table_schema', True) so that it outputs the JSON, like in this examples.

[Nestak2]

@saulshanabrook I have a question, that looks related to your post - When using jupyterlab-data-explorer, how can I convert a pandas dataframe to json in the notebook, so that I can use the different graphic options of nteract's data-explorer?
As an example, see here the example jupyterlab-data-explorer notebook, where you have the different graphical options in the red rectangle.
On the otherhand, this graphical options are not available for plain dataframes (example in my personal notebook). How can I make them available?

[Nestak2]

@saulshanabrook Thank you very much, I didn't know the purpose of this line, now the graphical features are there!

[saulshanabrook]

Great! Glad it's working for you. I have added this to the usage docs to hopefully make it more clear in the future: #135

[westurner]

FWIW,

• tablib https://tablib.readthedocs.io/en/stable/ supports a bunch of formats: 'cli, csv, dbf, df (DataFrame), html, jira, json, latex, ods, rst, tsv, xls, xlsx, yaml'
• Tabulate does [HTML, LaTeX, *] tables from lists of lists, lists of dicts, etc. (without pandas)
  https://github.com/astanin/python-tabulate . Hoping for these to land in a release soon:
  • Jupyter support: astanin/python-tabulate#27
  • astanin/python-tabulate#26
• odo
  https://github.com/blaze/odo (2018) does conversion between very many formats:

  Odo migrates data using network of small data conversion functions between type pairs. That network is below:
  odo conversions

  Each node is a container type (like pandas.DataFrame or sqlalchemy.Table) and each directed edge is a function that transforms or appends one container into or onto another. We annotate these functions/edges with relative costs.

  This network approach allows odo to select the shortest path between any two types (thank you networkx). For performance reasons these functions often leverage non-Pythonic systems like NumPy arrays or native CSV->SQL loading functions. Odo is not dependent on only Python iterators.

• Ibis
  https://docs.ibis-project.org/backends.html
  • Impala, BigQuery, HDFS, Spark, SQLAlchemy, Pandas
• blazingsql
  https://github.com/BlazingDB/blazingsql is really fast. It reads into the GPU from CSV, TSV, JSON, Apache Parquet, Apache ORC, Apache Hive, GDF (GPU Dataframe), S3, GCS, Apache HDFS:
  https://docs.blazingdb.com/docs

  BlazingSQL is a GPU accelerated SQL engine built on top of the RAPIDS ecosystem. RAPIDS is based on the Apache Arrow columnar memory format, and cuDF is a GPU DataFrame library for loading, joining, aggregating, filtering, and otherwise manipulating data.

  BlazingSQL is a SQL interface for cuDF, with various features to support large scale data science workflows and enterprise datasets.

• notes re: "A dataframe protocol for the PyData ecosystem"
  https://discuss.ossdata.org/t/a-dataframe-protocol-for-the-pydata-ecosystem/267/9

CSVW would be ideal for tabular data (with Linked Data metadata about the dataset and each column). More about this here: "Linked Data formats, tools, challenges, opportunities; CSVW, schema.org/Dataset, schema.org/ScholarlyArticle" https://discuss.ossdata.org/t/linked-data-formats-tools-challenges-opportunities-csvw-schema-org-dataset-schema-org-scholarlyarticle/160