dialohq/stork

An OCaml JSON migrator for ATD

Stork

A JSON migrator CLI for ATD

Installation

Using Esy

Add this to your esy.json or package.json file:

  ...
  {
    "stork": "*"
  },
  "resolutions": {
    "stork": "dialohq/stork#<COMMIT_SHA>"
  }
  ...

Usage

stork gen -o 'dir/file' [ --rescript ]

Will generate dir/file.ml and dir/file.mli. Use --rescript to generate ReScript files.

TODO: explain whole workflow.

Contributing

Take a look at our Contributing Guide. `

stork's Issues

Generating Reason File for web

In reference to the --rescript flag. Is it possible to have a --reason flag targeting web as well?

Publish stork to opam and npm

Feedback

The npm install -g doesnt work. Looks like its not published on npm.

Building from source is throwing an error on trying to build @opam/[email protected]

esy start
info building @opam/fmt@opam:0.8.9@e0843a5b
info building @reason-native/[email protected]@d41d8cd9
info building @reason-native/[email protected]@d41d8cd9
info building @opam/opam-state@opam:2.1.0~beta4@d7643898
info building @opam/ppx_string_interpolation@opam:1.0.1@790c9919
error: build failed with exit code: 1
  build log:
    # esy-build-package: building: @opam/fmt@opam:0.8.9

I saw the note about rely and runtest.

You can probably fix that with:

  "resolutions": {
    "refmterr": "jfrolich/reason-native:refmterr.json#08085a4048974b06aaf3aab918b677b39dea3d18",
    "@esy-ocaml/reason": "EduardoRFS/reason:reason.json#35aa4df3de0daa60bdc1133dcf97855decac48f7"
  }

I cant seem to figure out the fmt issue just yet or I would fix it for you.

Details

.... installing @opam/dune-release@opam:1.5.0@f91b0e                                                    info installing: done                             
info building @opam/fmt@opam:0.8.9@e0843a5b
error: build failed with exit code: 1
  build log:
    # esy-build-package: building: @opam/fmt@opam:0.8.9
    # esy-build-package: pwd: /Users/mando/.esy/3/b/opam__s__fmt-opam__c__0.8.9-1e341c19
    # esy-build-package: running: 'ocaml' 'pkg/pkg.ml' 'build' '--dev-pkg' 'false' '--with-base-unix' 'true' '--with-cmdliner' 'true'
    ocamlfind ocamldep -package seq -package stdlib-shims -modules src/fmt.ml > src/fmt.ml.depends
    + ocamlfind ocamldep -package seq -package stdlib-shims -modules src/fmt.ml > src/fmt.ml.depends
    ocamlfind: Package `seq' not found
    Command exited with code 2.
    pkg.ml: [ERROR] cmd ['ocamlbuild' '-use-ocamlfind' '-classic-display' '-j' '4' '-tag' 'debug'
         '-build-dir' '_build' 'opam' 'pkg/META' 'CHANGES.md' 'LICENSE.md'
         'README.md' 'src/fmt.a' 'src/fmt.cmxs' 'src/fmt.cmxa' 'src/fmt.cma'
         'src/fmt.cmx' 'src/fmt.cmi' 'src/fmt.mli' 'src/fmt_tty.a'
         'src/fmt_tty.cmxs' 'src/fmt_tty.cmxa' 'src/fmt_tty.cma'
         'src/fmt_tty.cmx' 'src/fmt_tty.cmi' 'src/fmt_tty.mli' 'src/fmt_cli.a'
         'src/fmt_cli.cmxs' 'src/fmt_cli.cmxa' 'src/fmt_cli.cma'
         'src/fmt_cli.cmx' 'src/fmt_cli.cmi' 'src/fmt_cli.mli' 'src/fmt_top.a'
         'src/fmt_top.cmxs' 'src/fmt_top.cmxa' 'src/fmt_top.cma'
         'src/fmt_top.cmx' 'src/fmt_tty_top_init.ml']: exited with 10
    error: command failed: 'ocaml' 'pkg/pkg.ml' 'build' '--dev-pkg' 'false' '--with-base-unix' 'true' '--with-cmdliner' 'true' (exited with 1)
    esy-build-package: exiting with errors above...
    
  building @opam/fmt@opam:0.8.9
esy: exiting due to errors above
[email protected] /Users/mando/Github/stork > 

Very much looking forward to seeing what you have done here.

Best

Nominal equality of unmodified types

Currently the types for all generated versions are nominally different, even if they are structurally the same.

Let's say that we have two atd files:

type person = {
  name: string;
  surname: string
}

type employee = {
  person: person;
  age: int
}

and

type person = {
  name: string;
  surname: string
}

type employee = {
  person: person;
  age: int;
+ title: string
}

The type person stays the same between both files (it's not even transitively modified). Currently this generates two types Version1.person and Version2.person. They are structurally the same but nominally different. Behind the scenes %identity is used to transform one to the other.

This has two cons:

1. Less important: the implementation of stork doesn't leverage OCaml type checker as a sort of proof mechanism
2. More important: When you write migrators there's a lot of repetition for nested fields (i.e. you have to run migrate on them even if they didn't change). e.g.:

    let convert_employee converter doc (from: OldVersion.employee): NewVersion.employee = {
      person = converter.convert_person converter doc from.person;
      age = from.age;
      title = "IC"
   }

instead of

    let convert_employee converter doc (from: OldVersion.employee): NewVersion.employee = {
      person = from.person;
      age = from.age;
      title = "IC"
   }

Allow strings for version fields

It's not always convenient to use plain ints for versions.

run generated files through fmt

Any default settings would be better than no formatting. Although a real win would be to check for a local .ocamlformat file.

Git based workflow for .atd files

The git workflow should probably be first class (files workflow being a secondary citizen). I was thinking about something like:

stork gen 1:34fdsd3 2:fasfsdf 3:abc123 4 -o ./prefix/to/file <atdfilepath>

So the syntax is <version_number>:<commit_hash> and for the last one it's just <version_number>

Where the last one is for the current file.

Set the version field automatically during serialization

Currently the version field needs to be set explicitly which is a source of potential bugs (as experienced in our product :) )

move test from rely to dune cram tests

Make tests less boilerplate-y

When I was reviewing #31 I noticed that the tests are extremely boilerplate-y without a need. They seem to be duplicated between tests, too. It'd be good to reduce this clutter because otherwise it's super hard to make sense of them.

Generate single *.ml(i) files for all versions