dhall-lang/dhall-lang

Maintainable configuration files

Dhall is a programmable configuration language optimized for maintainability.

You can think of Dhall as: JSON + functions + types + imports

Note that while Dhall is programmable, Dhall is not Turing-complete. Many of Dhall's features take advantage of this restriction to provide stronger safety guarantees and more powerful tooling.

You can try the language live in your browser by visiting the official website:

• https://dhall-lang.org

Getting started

The easiest way to get started experimenting with Dhall is to install the dhall-to-json and/or dhall-to-yaml executables, which enable you to generate JSON and YAML, respectively, on the command line. Platform- and runtime-specific installation instructions can be found in the Dhall documentation.

For other ways to integrate Dhall in your project, read:

• How to integrate Dhall

Tutorials

For a short introduction, read:

• Learn Dhall in Y minutes

To learn more about core language features, read:

• Core language features

For an even longer hands-on tutorial, read:

• Getting started: Generate JSON or YAML

... and for an even longer tutorial, read:

• Language Tour

Finally, we have a cheatsheet for a very condensed overview and quick lookup:

• Dhall Cheatsheet

What is this repository?

The Dhall configuration language has multiple implementations so that Dhall configuration files can be understood natively by several programming languages. You can find the latest list of the language bindings and their respective repositories here:

• How to integrate Dhall - Language support

This repository contains language-independent functionality, such as:

• The grammar and formal semantics

  Dhall is a formally-specified language standard, and language bindings follow the specification in order to ensure portability of Dhall configuration files across language bindings.

• The standard test suite

  This repository contains a test suite that language bindings can use to check compliance against the standard.

• The Prelude

  One Dhall package named the Prelude is versioned with and distributed alongside the language standard. This package contains general-purpose utilities.

• Shared infrastructure for the Dhall ecosystem

  Several services support Dhall developers and this repository contains a NixOps specification of that infrastructure that automatically deploys changes merged to that configuration.

Development status

The current version and versioning policy is detailed in the Versioning document, and you can see the latest changes in the Changelog.

The Dhall configuration language slowly evolves in response to user feedback and if you would like to participate in the language evolution process then you should read:

• CONTRIBUTING.md - How do I change the language

Design philosophy

Programming languages are all about design tradeoffs and the Dhall language uses the following guiding principles (in order of descending priority) that help navigate those tradeoffs:

• Polish

  The language should delight users. Error messages should be fantastic, execution should be snappy, documentation should be excellent, and everything should "just work".

• Simplicity

  When in doubt, cut it out. Every configuration language needs bindings to multiple programming languages, and the more complex the configuration language the more difficult to create new bindings. Let the host language that you bind to compensate for any missing features from Dhall.

• Beginner-friendliness

  Dhall needs to be a language that anybody can learn in a day and debug with little to no assistance from others. Otherwise people can't recommend Dhall to their team with confidence.

• Robustness

  A configuration language needs to be rock solid. The last thing a person wants to debug is their configuration file. The language should never hang or crash. Ever.

• Consistency

  There should only be one way to do something. Users should be able to instantly discern whether or not something is possible within the Dhall language or not.

The Dhall configuration language is also designed to negate many of the common objections to programmable configuration files, such as:

"Config files shouldn't be Turing complete"

Dhall is not Turing-complete. Evaluation always terminates, no exceptions

"Configuration languages become unreadable due to abstraction and indirection"

Every Dhall configuration file can be reduced to a normal form which eliminates all abstraction and indirection

"Users will go crazy with syntax and user-defined constructs"

Dhall is a very minimal programming language. For example: you cannot even compare strings for equality. The language also forbids many other common operations in order to force users to keep things simple

Name

The language is named after a Dustman from the game Planescape: Torment who belongs to a faction obsessed with death (termination). The fountain pen in the logo is the modern analog of Dhall's quill.

The name rhymes with "tall"/"call"/"hall" (i.e. "dɔl" for a US speaker or "dɔːl" for a UK speaker using the International Phonetic Alphabet).

Contributors

Code Contributors

This project exists thanks to all the people who contribute. [Contribute].

Financial Contributors

Become a financial contributor and help us sustain our community. [Contribute]

Individuals

Organizations

Support this project with your organization. Your logo will show up here with a link to your website. [Contribute]

dhall-lang's Issues

Remove support for `@`

One of the lesser-known features of Dhall's import system that it inherited from Morte is that you can import a directory if that directory has a file named @ (Similar to how default.nix works for Nix). However, as part of standardizing the import semantics I would like to remove support for this feature because it considerably complicates the semantics (for example: computing paths relative to an @ file) and also deteriorates error messages (for example, if an import fails it has to fall back to also trying ./@ which can lead to confusing errors).

I'm mainly opening this issue to broadcast that I will likely remove this feature as a breaking change. Feel free to object if you are particularly attached to this feature.

Dhall interpreter - Continuing after Ctrl + D

Right now, the interpreter quits after Ctrl + D. It would be nice if it wouldn't quit and keep continuing.

Arbitrary string record field identifiers

I'd like to be able to use a wider set of characters to compose record field names. My use case is that I'd like to interface with a system that uses YAML for configuration, and which expects keys that sometimes have colons in them. So for example, I'd like to have...

{ "something::complex" = 123 }

-> dhall-to-yaml ->

something::complex: 123

Yaml allows any arbitrary value as a key (strings, lists, etc...), but for my use case, all I need is to be able to use colons.

I want to make bindings for Swift

How do I start? :)

Syntax highlighting on GitHub

It would be nice if .dhall files (like this one) were syntax highlighted on GitHub. The instructions for adding a language don't seem too bad. I do wonder if Dhall is popular enough yet to be added.

Import path as absolute path to file

What do you think about as FilePath? It would be really nice to be able to write configuration files that encode a path existence check. Obviously this can be done in my library if I have Text fields, but then my users are forced to write string literals, when Dhall actually has syntax for file paths.

HTTPS for dhall-lang.org

http://dhall-lang.org works but https://dhall-lang.org doesn't. Could it?

Improve discoverability of Dhall code

The inspiration for this issue is feedback from the recent Dhall survey:

Some way to find functions would be immensely helpful. The way to currently find functions is by word of mouth.

If somebody is looking for a way to contribute this would be a great project

Date and time types and syntax

Would it be possible to add date and time of day, and date-time as first class values with their own literals syntax? We would probably want to think about time zones, while we are at it?

I am thinking of something like 2018-01-31 or so. The use case is that we are evaluating dhall for use as a configuration langue to specify when to schedule jobs. (Think like a cron on steroids.) Ability to specify time differences would also be handy.

Combine 'compilers' and interpreter?

I wonder if it would be more discoverable if there were just one dhall command. If run without options, it is the interpreter, and if run with dhall --to-json it compiles to JSON. WDYT?

There is no normalization rule for forall's that do not name their argument

https://github.com/dhall-lang/dhall-lang/blob/master/standard/semantics.md#functions lacks the following rule

A₀ → A₁   B₀ → B₁
─────────────────
A₀ → B₀ ⇥ A₁ → B₁

I note that in the informal syntax, we should consider A → B is short-hand for ∀(_ : A) → B)`, but there are plenty of other cases in the semantics where this distinction is actually made clear in the rules.

What is the precision of Double?

The grammar suggests infinite precision, but dhall-haskell just uses a (Haskell) Double. Which is right? Should dhall-haskell use Scientific instead?

Grammar confusion.

Reading the Dhall grammar, the following line stood out to me

text-literal = (double-quote-literal / single-quote-literal) / whitespace

Is that second / supposed to be there? I.e., is a text-literal really meant to be any of double-quote-literal or single-quote-literal or whitespace? Given the weirdness of that, and the apparently needless parentheses, I'm not so sure this was intentional…

Standard pretty-printer for dhall

There should be a single canonical format for dhall code, and there should be a tool that converts dhall files to be in that format, ala gofmt.

Reason: no one should have to care about formatting of configuration files.

Comment on the tutorial

I just went through the tutorial for the first time, haven't used Dhall "for real" yet. Following a few things I wish the tutorial included:

• A recommendation for how to order type parameters in functions of several arguments. For example something like "type parameters are usually introduced as late as possible in the list of arguments of a function, on the left of the first argument that needs the type".
• A comment on how Dhall restricts general recursion to guarantee termination. What can and cannot be done with Dhall functions?. For extra point, written in a style that most readers would understand, trying to avoid complex CS lingo.
• Example of implementing a non-trivial function. For example, how to write replicate, that would require introducing build, etc.
• Brief explanation of why is Dhall a good configuration language. Why non turing complete (many people will interpret that as a negative thing initially)? Why is having a "language" for config usefull? Why functions? And we can even have comments!

Hope this helps.

Allow combining record types

In dhall-to-cabal, we have a bunch of types that are all almost identical:

• https://github.com/dhall-lang/dhall-to-cabal/blob/master/dhall/types/Benchmark.dhall
• https://github.com/dhall-lang/dhall-to-cabal/blob/master/dhall/types/Executable.dhall
• https://github.com/dhall-lang/dhall-to-cabal/blob/master/dhall/types/TestSuite.dhall

They all have some common BuildInfo fields, and then add one or two extra fields. If I try and factor out BuildInfo, I get

Error: You can only combine records

./BuildInfo.dhall  ⫽ { main-is : Text, scope : ./Scope.dhall  }

dhall/defaults/../types/Executable.dhall:1:1

Does it have to be this way?

Improve nix install docs for projects that depend on dhall-haskell

Directions in the getting started wiki page say to install dhall-json like this:

nix-env --install --attr dhall-json

Running that in a local clone of dhall-lang/dhall-json resulted in the following error:

Setup: Encountered missing dependencies:
dhall >=1.9.0 && <1.10

Can the docs be improved to avoid this problem? It would help those who aren't yet familiar with nix and just want to get things working. The directions in various dhall-* projects should be reviewed too.

One potential solution is to add a symlink in ~/.nix-defexpr:

ln -s $DHALL_HASKELL_CLONE_DIR/release ~/.nix-defexpr/dhall

Recursive data types

I suspect being not turing complete, the following is not possible;

Is it possible to represent a recursive type in dhall?
e.g. a Node union that references itself:

Node.dhall:

< Plain : Text
| Heading : { level : Natural
            , heading : Text }
| Listing : List Node
>

akin to

data Node
  = Plain Text
  | Heading { level :: Int, heading :: Text }
  | Listing : [Node]

Recursive imports are not allowed, but maybe I’m missing something.

`Text` vs. `as Text`

I've just found out that I can do let value = "x" in "value = ${value}".
But I can not do let value = "x" in (./template as Text)
where template contains "value = ${value}".

The first results in x; the second in ${value}.

This would be really nice to have for dhall-to-text as it would allow
separating the template from the driver.

double-literal should finish with whitespace

In the standard grammar, integer-literal and natural-literal all end with whitespace, but double-literal doesn't. This appears necessary to validly parse https://raw.githubusercontent.com/dhall-lang/dhall-haskell/master/tests/parser/double.dhall, as the last element ends with whitespace.

Import path resolution affected by record shape

After learning of #103 I tried to make a minimal proof of concept showing how imports-as-types can reduce some of the type duplication; however, I found a strange behavior I can't explain.

Record with two elements (broken)

$ cat SimpleType
Integer

$ cat RecordTypeBroken 
{ a : ./SimpleType, b : Text }

$ cat SampleExprBroken
{ a = 5, b = "Hello world" } : ./RecordTypeBroken

$ dhall <<< './SampleExprBroken'

↳ ./SampleExprBroken 
  ↳ ./RecordTypeBroken 
    ↳ ./SimpleType, 

Error: Missing file

Same record with one element (works fine)

$ cat RecordType
{ a : ./SimpleType }

$ cat SampleExpr
{ a = 5 } : ./RecordType

$ dhall <<< './SampleExpr'
{ a : Integer }

{ a = 5 }

Version info

$ dhall --version
1.6.0

Better Syntax for Lists, Records (and Unions)

There are 3 different syntax styles for list like constructs:

1. Separator
2. Terminator
3. Enumerator

Here a quick overview

Separator (e.g. JavaScript)

Single line:

const colors = ["Red", "Green", "Blue"]

Multi line:

const colors = [
  "Red", 
  "Green", 
  "Blue"
]

This feels very natural for single line expressions, but has drawbacks for multi line expressions:

1. Items can't be easily rearranged when the last element is involved
2. Adding an element changes 2 lines (diffs get harder to read & understand)

Other acceptable characters for this style (in single line form) are | or / and if necessary also ;.
In multi line form this really sucks.

Terminator (e.g. OCaml)

Single line:

let colors = ["Red"; "Green"; "Blue";]

Multi line:

let colors = [
  "Red";
  "Green";
  "Blue";
]

This looks a little weird for single line expressions, but excels for multi line expressions.
Making it also mandatory for the single line version has the advantage that
it's not possible to forget to add the last ; when converting a single line to a multi line expression.

A special case of this syntax is when the newline character is used as terminator.
(e.g. in CoffeeScript)

colors = [
  "Red"
  "Green"
  "Blue"
]

Other acceptable characters for this style are . and , (but less so).

Enumerator (e.g. YAML)

Single line (is not valid in YAML, but could be in another language):

colors: - Red - Green - Blue

colors:
  - Red
  - Green
  - Blue

Needs a little getting used to in single line form, but is actually not too bad.
Multi line is great, as it feels very natural to use and is quite readable.
Other good characters for this are * or + (* not so much, as it is vertically aligned to the top in many font faces)

Now, what does Haskell / Dhall do?

It uses a broken version of the Enumerator style (broken, because first element can't be easily changed / rearranged) with the worst possible character (worst, because , is totally skewed to the bottom).

I really hope you can reconsider this design choice!
At the moment it is just awful.

My favorite version would be ; or , in the terminator style.
(I think . is too dangerous, as it could be misunderstood as a decimal point or vice versa,
but I haven't thought this through yet.)

colors = ["Red"; "Green"; "Blue";]

color = {
  "Red": 50;
  "Green": 60;
  "Blue": 70;
}

-- And Unions could look like this:
size = | Small | Medium | Large

What do you think?

The formatter is not pretty

In some cases, the formatter gives ouput which I feel isn't that great for readability. I consider this to be readable:

\(union : < Left : Natural 
          | Right : Bool >) ->
  let handlers = { 
    Left = Natural/even, 
    Right = \(b : Bool) -> b 
  }
  in merge handlers union : Bool

But the formatter currently gives this:

  λ(union : < Left : Natural | Right : Bool >)
→     let handlers = { Left = Natural/even, Right = λ(b : Bool) → b }
  
  in  merge handlers union : Bool

I feel that I can get used to the formatter syntax over time, but I still think it can be improved. The whitespace at the beginning of the first line should be removed IMO.

Imports relative to the file that's importing

Take the following files:

• a/b containing the Dhall expression ./c
• a/c containing the Dhall expression True.

I then run

$ dhall < a/b

and would expect to see True. Instead, I actually get:

↳ ./c 

Error: Missing file c

I find this very surprising, and find Nix's import semantics much more sensible. Is there still time to rethink this?

Support compiletime-resolvable let bindings in // and /\ ?

I'm working on https://github.com/arianvp/dhall-kubernetes
and my code generator currently generates the following code which works:

  λ ( model
    : { metadata : ./io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta.dhall 
      , spec     : Optional ./io.k8s.api.apps.v1.DeploymentSpec.dhall 
      , status   : Optional ./io.k8s.api.apps.v1.DeploymentStatus.dhall 
      }
    )
→ model ⫽ { apiVersion = "apps/v1", kind = "Deployment" }

But refactoring to make the generated code more readable gives a compiler error:

let
  ModelSpec =  {
    metadata : (./io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta.dhall) ,
    spec : (Optional (./io.k8s.api.apps.v1.DeploymentSpec.dhall)) ,
    status : (Optional (./io.k8s.api.apps.v1.DeploymentStatus.dhall)) ,
  }
in
  \(model : ModelSpec) ->  model // { apiVersion = "apps/v1", kind = "Deployment" }

Use "dhall --explain" for detailed errors

ModelSpec : Type
model : ModelSpec

Error: You can only combine records

model // { apiVersion = "apps/v1", kind = "Deployment" }

(stdin):8:28

Capability to override default values with provided via ENV

I was wondering if there's a way to provide default values for undefined (or possibly blank) environment variable values. E.g. say I have a default service URL value of "http://some.svc/", but I want to be able to override it with a SERVICE_URL env variable.

Looking at the Haskell implementation I figure there's an exception thrown as soon as lookupEnv fails, which seemingly eliminates possibilities of handling it within Dhall domain.

If this is a part of the language standard, I propose changing it in some way at some point to accommodate for the use case.

reference current record?

I was trying to build a record of functions and couldn’t figure out how to reference other fields in the same record, so I started chaining them as below:

      let foo = { g = λ(a : Natural) → λ(b : Natural) → a + b }
  in  let fooo = foo ⫽ { h = λ(a : Natural) → foo.g a +2 }
  in  fooo ⫽ { i = fooo.g (fooo.h +4) }

so that the final record has g, h, and i in it.

Simplify if x then y else y to y?

Is it worth having the normalization step

l ⇥ x   r ⇥ x
─────────────────────── 
if t₀ then l else r ⇥ x

?

Cannot import Optional

I found the following quite surprising:

[nix-shell:~/work/dhall-to-cabal]$ dhall <<< '[] : Optional Natural'
Optional Natural

[] : Optional Natural

[nix-shell:~/work/dhall-to-cabal]$ echo 'Optional Natural' > foo

[nix-shell:~/work/dhall-to-cabal]$ dhall <<< './foo'
Type

Optional Natural

[nix-shell:~/work/dhall-to-cabal]$ dhall <<< '[] : ./foo'

Error: Invalid input

(stdin):1:2: error: expected: expression,
    whitespace
[] : ./foo

Is this intended?

I actually ran into this in dhall-to-cabal, where I wrote:

      , library     =
          [ { build-dependencies =
                [ common-deps.base
                , common-deps.Cabal
                , common-deps.dhall
                , common-deps.text
                , common-deps.bytestring
                , { bounds = majorVersion [ +0, +5 ], package = "containers" }
                , { bounds = majorVersion [ +0, +12 ], package = "vector" }
                , { bounds = majorVersion [ +1, +7 ], package = "trifecta" }
                , { bounds = majorVersion [ +0, +3 ], package = "text-format" }
                ]
            , exposed-modules    = [ "Distribution.Package.Dhall" ]
            , hs-source-dirs     = [ "lib" ]
            , name               = [] : Optional Text
            , other-modules      = [] : List Text
            }
          ] : ./dhall/types/Library

And got

...
... with this type or kind:                                                     
                                                                                
↳ Optional { build-dependencies : List { bounds : VersionRange, package : Text }, exposed-modules : List Text, hs-source-dirs : List Text, name : Optional Text, other-modules : List Text }                                                                
                                                                                
... but the inferred type or kind of the expression is actually:                
                                                                                
↳ List { build-dependencies : List { bounds : VersionRange, package : Text }, exposed-modules : List Text, hs-source-dirs : List Text, name : Optional Text, other-modules : List Text }

That is, it expected Optional but for some reason got infered as List - even though the import is an Optional type. Library contains:

Optional
{ build-dependencies : List { bounds : VersionRange, package : Text }
, exposed-modules    : List Text
, hs-source-dirs     : List Text
, name               : Optional Text
, other-modules      : List Text
}

Enum Types

When writing dhall configs, I find myself using this pattern to specify enums:

Type: { aType: Text }
Type/A: { aType: "a" }
Type/B: { aType: "b" }

and then when typing dhall code, I want a record to be of type Type which Type/A and Type/B are instances of.

But this feels a bit cludge-y to just specify enum types. Are there plans of having enum types in dhall, maybe an alternative to the Union types where I can do:

< Enum: "a" | "b" >

Support period in record identifiers

Like #64 but for .. I am considering using dhall to generate kafka connect json but most of the keys have periods in e.g. "table.name.format", "connection.url". Is it reasonable to whitelist period as well?

Thanks

Semantics contradicts itself on type of True and False

https://github.com/dhall-lang/dhall-lang/blob/master/standard/semantics.md states:

True and False have type Bool:

───────────────
Γ ⊢ True : Type

────────────────
Γ ⊢ False : Type

But shouldn't this be True : Bool and False : Bool?

README suggestions

Hi! I've wanted something like dhall for a long time. My primary use case is configuring other applications, like grafana (check out https://github.com/weaveworks/grafanalib for how I did it instead) & Kubernetes.

I think the README currently doesn't help as much as it might.

Big ideas:

• Your primary audience is people writing new programs in Haskell who want to use a new config format. I think you should change the audience to be people who are sick of writing repetitive config files in YAML & JSON and want a sane way of doing that
• README needs to start from user problems, rather than dhall solutions
• README must be a pitch to convince people to investigate further

Here are some detailed suggestion:

• the first two lines are great
• don't lead with "Features", instead lead with problems that dhall solves, e.g.
  • "Need a safe, concise way to generate repetitive YAML or JSON? With dhall you can turn the repetition into function definitions, and trust the type system to catch errors before they hurt users."
  • "Adding a new config file to your app? Using dhall-lang as the format means"
  • I find the Q&A format easier to write, but the idea is starting with problems that your end users are facing or ways that you can make their life better, rather than how clever dhall is
• The README repeatedly uses "I". That's cool, but say who "I" is.
• Please please move the core documentation off of hackage. Browsing the implementation's library documentation to learn how to use something makes it feel like its only for insiders. This goes double when the implementation language is Haskell.
• "I recommend the following progression" — I think this really undersells dhall
  • There's a large devops audience who could benefit, for whom "If your language does not yet support Dhall" just sounds like they are being grudgingly allowed to use this software
• Replace the overview with a "getting started" guide
  • Start with a single self-contained, concrete, representative example that's not too complicated and shows a couple of features worth highlighting
  • Move the REPL stuff to a fully separate "Learning the language" section
  • No one wants to evaluate expressions in a REPL. They evaluate expressions in a REPL because they are trying to do something else.
  • Move all the details about language bindings to their respective documentation systems. A colleague told me "dhall is a Haskell thing", based on skimming this.
  • Don't even mention the word "compliers". Instead have headings like "Converting dhall to JSON", "Converting dhall to YAML", "Converting dhall to Nix"

I hope this helps. Take or leave this any or all of this advice as you will. I'd love to send a PR, but I don't have the time and there's not much point unless you agree in principle with the direction I'd like to take.

Issues using List/map + dhall-format

Probably doing something stupid here, but given the following:

in  let envToStr = λ(e : { name : Text, val : Text }) → e.name ++ "=" ++ e.val

in  let List/map =
          https://ipfs.io/ipfs/QmQ8w5PLcsNz56dMvRtq54vbuPe9cNnCCUXAQp6xLc6Ccx/Prelude/List/map 

in  let toEnv =
            λ(es : List { name : Text, val : Text })
          → List/map { name : Text, val : Text } Text envToStr es

then dhall-format converts List/map on the last line to List /map, and I get Error: Missing file. dhall-format doesn't munge List/head in the same way, which is odd.

If I disable dhall-format then I get the same error, though.

Using dhall 1.8 from stackage nightly 2017-11-25, on MacOS.

Include rail-road grammar diagrams to compliment the ABNF

GrammKit can generate so called "rail-road" diagrams from some grammars. Dhall.js uses PEG.js for its parser, which is a supported input to GrammKit. The result is https://ocharles.org.uk/dhall-grammar.html. We can probably tidy this up, but I think it would be a great addition to the public facing documentation and standards.

Standardization: Grammar Specification

I'm interested in helping standardize dhall. A good place to start would be a grammar specification. While referencing the tutorial I started working on a PEG:

Type <- 'Bool' / 'Natural' / 'Integer' / 'Double'
      / 'List' Type / 'Optional' Type
      / '{}' / '{' (TypeBinding ',')* TypeBinding '}'
      / '<' (TypeBinding '|')+ TypeBinding '>'
      / (Type / '∀' '(' TypeBinding ')') '→' Type
// TODO: add string type

Expr <- Bool / Natural / Integer / Double
      / Optional / List / Record / Field
      / 'if' Expr 'then' Expr 'else' Expr
      / 'let' (TypeBinding / Ident) '=' Expr 'in' Expr
      / Func / Ident / Union / String
      / Expr Expr / Expr BinOp Expr / BIF
Ident <- TODO
Bool <- 'False' / 'True'
Natural <- '+' [0-9]+
Integer <- '-'? [0-9]+
Double <- '-'? [0-9]* '.' [0-9]+
Optional <- '[' ']' / '[' Expr ']'                   // Type checker needs ability to conditionally promote Optional to List
List <- '[' (Expr ',')+ Expr ']'
Record <- '{=}' / '{' (Binding ',')* Binding '}'
Binding <- Ident '=' Expr
TypeBinding <- Ident ':' Expr
Field <- '.' Ident
Func <- ('\' / 'λ') '(' TypeBinding ')' ('->' / '→') Expr
Union <- '<' ((TypeBinding / Binding) '|')+ (TypeBinding / Binding) '>'
// TODO: Fix Union to include exactly one Binding.

String <- TODO
BinOp <- '||' / '&&' / '==' / '!=' / '+' / '*' / '++' / '#' / '//' / '⫽' / '/\' / '∧'
BIF <- 'Natural/' ('isZero' / 'even' / 'odd') Expr /
       'Natural/fold' Expr Type Expr Expr /
       'Natural/build' Expr Expr /
       'List/fold' Type Expr Type Expr Expr /
       'List/' ('build' / 'length' / 'head' / 'last' / 'indexed' / 'reverse') Type Expr /
       'Optional/fold' Type Expr Type Expr Expr

Note this is completely hand-done, incomplete, and unverified. I'd like to see if there's interest in this approach before spending more time on it.

Some aspects that aren't yet addressed include:

• whitespace rules
• imports
• using for headers
• strings (double-quoted and multi-line quoted)
• strict grammar rule for Union
• merge for unions
• Ident rule for legal identifiers
• parenthesis grouping rules for Exprs and Types

Question: How do I write eliminators for "private" types?

In dhall-to-cabal I learnt the really useful trick of encoding a "private" type by a function containing the type and arguments for each possible constructor:

forall ( foo : Type ) -> forall ( fromText : Text -> Foo ) -> forall ( fromInteger : Integer -> Foo ) -> Foo

In another project, I'd like my users to pull out information from Foo, but keep Foo's implementation private. I'd like to offer an accessor into Foo, blah : ./Foo -> Text. Continuing the above example, this might do some arbitrary IO in my interpret that uses the fromText or fromInteger values appropriately.

Can this be done? If so, how do I write blah? I could do it with a custom type checking context, but I'm wondering if this is also possible in pristine Dhall.

Expose dhall as javascript library via ghcjs WIP

I plan on playing a bit with compiling the compiler to JS, depending on the free-time I have.

A first (trivial) success:

On current nixpkgs master, nix-build -A haskell.packages.ghcjsHEAD.dhall gives us node binaries that basically work (you have to split the shebang line of the binary manually, then it works).

philip@katara ~/nixpkgs (current-build)> ~/tmp/dhall
{ foo = "bar" }   
{ foo : Text }

{ foo = "bar" }
philip@katara ~/nixpkgs (current-build)> file ~/tmp/dhall
/home/philip/tmp/dhall: a /nix/store/gbladqk26m3mhmaj147lfv5rpsiilfxb-nodejs-slim-6.11.5/bin/node script, UTF-8 Unicode text executable, with very long lines
philip@katara ~/nixpkgs (current-build)> head /nix/store/z8kb2pgilq0083van0faqghkj21p2m2q-dhall-1.8.0/bin/dhall
#!/nix/store/gbladqk26m3mhmaj147lfv5rpsiilfxb-nodejs-slim-6.11.5/bin/node var h$currentThread = null;
var h$stack = null;
var h$sp = 0;
var h$initStatic = [];
var h$staticThunks = {};
var h$staticThunksArr = [];
var h$regs = [];
var h$r1 = 0;
var h$r2 = 0;
var h$r3 = 0;
philip@katara ~/nixpkgs (current-build)> 

Container management integrations

Three separate users responded to the survey that they were using Dhall for managing containers using either Kubernetes or Docker Swarm

• WHAT: Generating docker swarm and other configs from common templates with
  specific variables inserted safely.

  WHY: JSON and yaml suck, want to know about errors at compile time.

• WHAT: Kubernetes

  WHY: Schema validation. Kubernetes is known for blowing up in your face at
  runtime by doing evaluation and parsing at the same time

• WHAT: configuration - use haskell integration and spit out json/yaml on
  some projects (to kubernetes configs for instance)

  WHY: I love pure fp and dealing with configs wastes a lot of my time and
  is often error prone

I'm opening this issue to note that there might be value in providing a more specific
Dhall integration with either Kubernetes or Docker Swarm for managing large numbers
of containers with repetitive configurations.

Set data type?

Purcel uses the type

{ package : Text
, modules : List Module
, dependencies : List Module
}

for packages. Using List in this context is okay-ish, but a dependency list is really a collection of unique dependencies, a Set.
I haven’t thought about this a lot, but stuff like this might be a good reason to support sets in dhall, along
with the normal (axiomatic?) set operators.

One problem I can think about is that Set/union needs to check for equality to merge two sets and might be a pain to implement.

Can we drop the mandatory type annotations on Optional and List?

Title says it all, these feel like a bit of a wart in what is otherwise a lovely format to work in.

Lack of unification prevents true separation of data and type "schema"

The type inference algorithm goes in a single direction / isn't unification, which makes the interesting use case of data/schema separation less ergonomic/useful.

Case in point:

[nix-shell:~/daedalus/installers]$ dhall
{ a = ["a"] } : { a : Optional Text }

Use "dhall --explain" for detailed errors

Error: Expression doesn't match annotation

{ a = ["a"] } : { a : Optional Text }

(stdin):1:1

..but:

[nix-shell:~/daedalus/installers]$ dhall
{ a = ["a"] : Optional Text } : { a : Optional Text }
{ a : Optional Text }

{ a = [ "a" ] : Optional Text }

Optionally present fields: is it possible?

I am prototyping something with Dhall and it looks very nice except for one part: I cannot figure out the way to express optionally-present fields.

For example, an API dictates that the data type should look like:

{ cpus: Optional Double
, memoryMb: Optional Double
, diskMb: Optional Double
, numPorts: Optional Integer
}

Most of the time there is no need to configure all of these fields, so I'd like to be able to use this type as:

{ cpus = 1 } : ./Resources

Unfortunately, it doesn't seem to be possible, and the only way to use this type that I have found would be to write something like:

{ cpus = ([1] : Optional Double)
, memoryMb = ([] : Optional Double)
, diskMb = ([] : Optional Double)
, numPorts = ([] : Optional Integer)
} : ./Resources

which is not a great user experience IMO, especially when the configuration is a bit more complicated/nested.

Is there a way to express optional/optionally present fields in Dhall somehow different and simpler for users?

Another question: I have found that dhall-json generates the following JSON from the example above:

{
  "numPorts": null,
  "diskMb": null,
  "cpus": null,
  "memoryMb": null
}

Is there a way to not render null fields at all, or is it left to post-processing?

Semantics has no mention of interpolation

It simply says that text literals are in normal form. Should I be parsing interpolation as text literals with concatenation?

can’t partially-apply built-in merge function

I was stymied by the error

expected: "(",
    ".", built-in expression,
    double literal, import,
    integer literal, label,
    list literal, natural literal,
    record type or literal,
    text literal,
    union type or literal,
    whitespace

for a while before I realized I had to do λ(x : Foo) → merge { ... } x instead of just merge { ... }. I’m not sure if this is true of all built-in functions, because others that I’m likely to attempt this with (e.g., List/fold) have a parameter order that makes partial application difficult.

Let binding pattern matching to unpack records

The ability to unpack a record through a let-binding is tremendously powerful for building a lightweight module system. I could envision doing something like:

let { map, filter, span } = http://prelude.dhall/List
in ...

Right now, my only option is to import each function individually. Not only is that more verbose, it makes it much harder to ensure consistency. While the above functions are separate, one could imagine an import where we want to make sure that both function comes from the same "version" of a library, and the only make sense (operationally) if we're using the same version.

Native integration with C/Rust: libdhall?

https://github.com/abiggerhammer/hammer is a parser combinator library written in C with bindings to multiple languages (C++/Python/Ruby/Perl/Go/PHP/.NET).

Building the dhall parser in hammer might give a good starting point to support these languages?

Issues with user-defined types

It's (sort of) possible to create a new type:

% dhall <<< 'let Day : Type = Natural in Day'
Type

Natural

It doesn't appear possible to do anything with said type, though:

% dhall <<< 'let Day : Type = Natural in \(d : Day) -> d'

Use "dhall --explain" for detailed errors

Error: Unbound variable

Day

(stdin):1:35

It seems like Day should be defined, leaving aside the issue of how one would create a value of type Day in the first place. (It doesn't appear to simply be a synonym for Natural, as evidenced by

% dhall <<< 'let Day : Type = Natural in \(d : Day) -> d + +1'

Use "dhall --explain" for detailed errors

Day : Type
d : Day

Error: ❰+❱ only works on ❰Natural❱s

d + +1

(stdin):1:43

)

should `identifier` also have trailing whitespace?

The following

λ(a : Type) → λ(a : Type) → a@1\n

Doesn't seem parsable by the ABNF. We finish parsing an identifier, but hit a new line - but apparently nothing follows an identifier.

Allow deep combining without failure

Currently, we have two options to combine records:

The (//) operator combines the fields of both records, preferring fields from the right record if they share fields in common

and

The (/) operator also lets you combine records, but behaves differently if the records share fields in common. The operator combines shared fields recursively if they are both records but fails with a type error if either shared field is not a record.

Would it be possible to add a 3rd option to "combine fields recursively if they are both records, preferring non-record fields from the right record"?

In my use case I want to have some "default" value (deeply nested type) and combine it with a user-provided record so that user doesn't have to specify all the fields but the combined record would form a valid type.