datahaskell/type-providers

Type providers for Haskell: plans and shared interfaces

Type providers for Haskell

Plans

The most pressing needs are currently:

1. Write XML type provider using XML Schema - Michal is willing to mentor a student.
2. Support more flat types in Frames and json-autotype.
3. Make SQL type provider.

Long term goals:

• Type provider for Servant types for whole WebAPIs
• Automatic detection of filetype, and parsing it using your favourite Haskell library.

You may apply for Google Summer of Code projects. We are also looking for other ways of mentoring students to help us with this project.

Shared interfaces

Anthony Cowley of Frames and Michal Gajda of json-autotype agreed to join the recognition of flat types, so that it can be shared between Frames , json-autotype and other type-providers.

XML

XML Schema is published for most popular XML formats. For those that have RelaxNG schema, we can use converter. So I recommend starting by using TagSoup to parse XML Schema.

It can be done by beginner-to-intermediate level Haskell programmer. Level of task complexity is similar to this. (No advanced type classes needed, just IO, and XML parsing monad.) No need to bother if code to convert XML Schema to Haskell will be ugly at first - we can use our own tool to generate a new one :-).

References:

• Lazy version of TagSoup
• XML to SQL conversion
• Example XML Schemas
• Office OpenXML are probably one of the largest
• XML-ish and XPath-ish XML parsing library
• UUXML

• there was another effort to provide XML-Haskell data binding I am contacting authors about their UUXML attempt, but it seems that is defunct as of now (We contacted the authors - waiting for response.)

SQL

Ideally we could use Selda and generate record types for each table to start with. We just need a interoperable SQL library that allows to query table types (like ODBC).

Usage of meta-type-provider prototype

$ type-these input/*

Then see Haskell modules made to read your data in input/ directory.

Currently need to add new providers to type-it script. [WIP:]This script will be generated automagically.

Package API

Before we make .cabal accept type-provider target, we can use fixed path:

1. Compile executable named ./type-provider.
2. Register package by sending to [email protected]
3. Get PR to accept the type provider to build as type-providers dependency.
4. [WIP] We make type-providers.haskell.org to upload data on type providers.
5. [WIP] Container that has type providers preinstalled.

Type provider executable

Type provider executable should accept the following arguments:

• -o filename - output module name
• -m modulename - output Haskell module name
• inputname - filepath to the file or directory matching the pattern, we read the input file as-is readInput :: FilePath -> IO InputType

First thing that type provider does should be to detect whether file matches the pattern.

Question: do we also need a library option to detect if file might be handled? (If so, contact me.)

Standard exit codes:

• 0 - success, file handled by type provider, subdirectories pruned
• 2 - file unhandled

Proposal for .cabal extension

Instead of building normal executable, we would mark it type-provider:

type-provider ExcelTypeProvider.hs
  mime-types: text/plain
  glob: *.xlsx

It can either have a list of glob patterns for matching file and directory names, or have mime-types for matching MIME type (warning: MIME type matching may work differently on different operating systems!).

type-providers's Issues

Type provider for Servant types for whole WebAPIs

SQL type provider

Related: part of @gagandeepb 's GSoC work ( https://github.com/gagandeepb/Frames-beam )

Cabal type provider extension

Proposal for .cabal extension

Instead of building normal executable, we would mark it type-provider:

type-provider ExcelTypeProvider.hs
  mime-types: text/plain
  glob: *.xlsx

It can either have a list of glob patterns for matching file and directory names, or have mime-types for matching MIME type (warning: MIME type matching may work differently on different operating systems!).

XML type provider using XML Schema

Support more flat types in Frames and json-autotype

(Could you elaborate on this one @mgajda ?)