prayerslayer/js.spec

clojure.spec for Javascript

js.spec

clojure.spec for Javascript

There is clojure.spec, a part of Clojure's core library that is intended to help with specification, testing and error messages. I recommend you to read the linked rationale, you will get the gist even without knowledge of Clojure. In any case here's my best attempt of a summary:

In a dynamic language like Javascript or Clojure it is common to represent information with data. For instance, you could encode the score of a soccer match with a list of two integers [0, 1]. (In a statically typed language like Java the idiomatic way would be an instance of a SoccerScore class.) This data is passed around between modules of your code or sent to external systems, yet the knowledge about what this list of integers stands for is not available anywhere. Maybe you described it in the project's documentation, which is likely outdated. Most probably it is implicitly assumed in your code (var goals_scored = score[0] + score[1];). If the semantics change (e.g. list contains also teams), your code breaks.

One way to mitigate this is by static analysis tools (Flow) or typed languages (Typescript), but they only get you so far. For instance they don't work at runtime (duh!) and offer limited expressiveness. (They also need more tooling, but that's another story.) So what you're left with is manual parsing and checking, which is repetitive and half-assed at worst (if (!Array.isArray(score)) ...), and non-standard at best, ie. there is no way for users of your function to know where exactly their provided input failed your specification.

Another issue is that schemas get more popular in the Javascript community (JSON Schema, React.PropTypes...), yet there is no uniform way to define them. Error messages differ from system to system. You can't port your schemas, but need to define them multiple times in different syntax.

js.spec tries to solve those problems.

Usage

For a quick usage example see example.js. Otherwise please refer to the documentation.

Documentation

https://prayerslayer.gitbooks.io/js-spec/content/

Installation

npm install js.spec

This will install the current release candidate, as there is no 1.0 yet.

If you want to include it into a web page directly via script, please use the UMD build like this:

<script src="./js.spec.bundle.js" charset="utf-8"></script>
<script type="text/javascript">
  const {spec, valid} = window['js.spec']
  const foo = spec.map("foo", {
    name: spec.string
  })
  alert(valid(foo, {name: 'hugo'}))
</script>

Implementation Status

The 0.0.x published versions should be seen as developer previews. Code may or may not work. API may or may not witness breaking changes.

• Specs
  • ✅ Primitives
  • ✅ Map
  • ✅ Collection
  • Combination
    • ✅ And
    • ✅ Or
  • ✅ Tuple
  • ✅ Nilable
  • ✅ Enum
• ✅ Spec Registry Removed after discussion in #21
• 🚧 Spec Regexes (cat, alt, *, ?, +)
• 😰 Generator Functions
• 😫 Function Specs (Not even sure if it's possible the way it works in clojure.spec)

Why not use Clojurescript?

If you already thought about using CLJS, go ahead. clojure.spec is already available there. However if you meant to pull in CLJS as a dependency: clojure.spec is macro-heavy. Macros exist only at compile-time, so all the "functions" (they are macros) are gone.

License

MIT, see LICENSE

js.spec's Issues

Protect against misnamed specs

Report of a possible UX issue and yes, it bit me hard 😄 🦁

Say I have a js.spec map:

const S = require("js.spec");
...
const persistedEvent = S.spec.map("PersistedEvent", {
  name: S.string,
  domain: S.string,
  aggregate: S.string,
  id: S.string,
  timestamp: S.number,
  data: S.object
});

On assert, I get a bit of a strange error:

Error: PersistedEvent: isObject failed for undefined at [].

Because I am not using S.spec, all the predicates are undefined and everything breaks.

It would be great to detect undefined predicates and somehow warn.

Enum specification

There should be a way to define "1 of n discrete values".

spec.alt and spec.cat have no documentation

js-spec.online has no docs for these two regex methods. They are exported from spec so they should be documented.

Add some sugar for chai testing

Hello folks!

I had this idea for quite a bit and I was wondering what you think about it.

Reading here I can see that we could basically add chai assertions around the spec validation, something like:

chai.use(function (_chai, _) {
  _chai.Assertion.addMethod('conform', function (msg) {
    _.flag(this, 'message', msg);
  });
});

expect(a).to.conform(spec);

Some inspiration is here: https://github.com/chaijs/chai-json-schema/blob/master/index.js#L125

It can be a separate module maybe, what do you think?

Warn about undefined predicates

Logging here what happened in https://github.com/arichiardi/js-spec-chai.
When mistakely using predicates that resolve in undefined, warn the user.

For instance this should be detected and avoided:

import * as s from 'js-spec'
console.log(s.string)
// => undefined
console.log(s.spec.string)
// => Function

Peace ☮️

Please review Typescript definitions for js.spec

I am preparing a PR for DefinitelyTyped to add a TypeScript definition for js.spec. I plan on maintaining this definition over time.

Before I submit a PR I would appreciate a review: https://github.com/mattbishop/DefinitelyTyped/tree/add-js-spec/types/js.spec

Alternately the type definition file can be part of this project too.

Better string predicates

There should be built-in predicates for

• regexes
• emails
• URLs
• ????

Custom lodash build

Use custom lodash build to avoid loading the same parts in different lodash.x modules for predicates.

Expose spec.finite as docs state

http://js-spec.online specifies a finite predicate yet this predicate is not exported.

Do we need built-in predicates?

Thanks for your hard work in getting this off the ground so far! Looks great!

Out of curiosity, is there any reason you feel like this library needs built-in predicates? It seems to me that the basic building blocks of clojure.spec are specs, regexs (cat, alt, *, etc), data generation and function specs. I know you mentioned JS not having a standard library of predicates like clojure, but I'd argue that that is a good reason to let people bring (or build) their own. The most interesting predicates that make spec valuable beyond a type system are not ones that an easily expressed by a single predicate anyway. Think positive integer, number between, array with a certain length, etc etc.

API Documentation

There is currently no way to know available specs besides what's in examples.

`oneOf` not exported

The docs refer to a oneOf spec, but the exported S.spec does not export this function from enum.js

Registry aliases?

I'm working on a PR for js.spec and am short some unit testing coverage. Istanbul showed me a branch in registry.get() that is not tested:

export function get(id) {
  const s = REGISTRY[id]
  if (typeof s !== 'symbol') {
    return s
  }
  return get(s)  // <-- here is the untested branch
}

My interpretation of this code is that one could define an alias symbol for another Spec's symbol, and then use either symbol to fetch the defined spec. I have a test that looks like this:

    it("defines an alias to another defined spec and gets the aliased spec back", () => {
      const id = Symbol()
      const spec = {}
      define(id, spec)
      const alias = Symbol("alias")
      define(alias, id)
      const actual = get(alias)
      expect(actual).to.equal(spec)
    })

This test fails with:

TypeError: Cannot create property 'name' on symbol 'Symbol()'
    at define (lib/registry.js:19:3)
    at Context.<anonymous> (test/registry.test.js:20:7)

The problem is the second define(alias, id) call; define() assumes that spec is not a symbol and sets the name property.

Am I misinterpreting the purpose of Symbol values in Registry? How would I even define such a thing?

Regex operators for sequences

From clojure.spec, but we need to rename some as ?, + and * are not valid in javascript identifiers.

• cat
• alt
• ? => qm? atMostOne?
• + => plus? atLeastOne?
• * => star? any?

So that we could

import {define, spec} from 'js.spec'

define("ingredient", spec.cat(
  "quantity", spec.number,
  "unit", spec.string))

define("odds-then-maybe-even", spec.cat(
  "odds", spec.plus(spec.odd),
  "even", spec.star(spec.even)))

Check to what extent we can leverage pamatcher.

Integer failure error can be improved

Hello again, just logging here a potential improvement of the error message for s.spec.number.

I have:

const school = s.spec.map("schoolSpec", {
   city: s.string
});
const friend = s.spec.map("friendSpec", {
   name: s.spec.string,
   age: s.spec.number,
   school
});

and (using the new js-spec-chai:

it("test", () => {
  const obj = {
    name: "andrea",
    age: "18",
    school: {
      city: "Turin",
    }
  };
  obj.should.conform(friend);
})

I receive AssertionError: friendSpec → i: i failed for 18 at [age]. which could be improved to AssertionError: friendSpec → age: s.spec.number failed for 18 at [age]. for instance.

Name mangle is performed on internal predicates

Hello again!

I have another problem with name mangling. With the current uglified version, when I get an error I see:

Error: PersistedEvent: a failed for undefined at [].

If I comment out the plugin I see the correct:

Error: PersistedEvent: isObject failed for undefined at [].

I am trying to understand why it is renaming all the things and if I can avoid it.

Thoughts and suggestions are more than welcome!

Simple use case failing in node.

https://runkit.com/crunchnode/spec-testing

Am I doing something wrong?

Javascript spec type not discoverable from Typescript?

Hello again @prayerslayer!

I am using js.spec from Typescript and, unless I am doing something silly, it looks like the instanceof operator is not returning true for spec passed in from modules. Don't ask me why 😢

Investigation has become, I will write some more things here as I discover them. Not a JS dev here, so it might be something very obvious that I wish I can quickly discover.

The part that fails in the js.spec code is toSpec, which throws all sorts of errors even if the input objects seem correct when I print them.

Have you considered using an explicit key, maybe using Symbol which does not interfere with the "normal" ones, for type information?

Are custom predicates supported?

It is not clear from the documentation whether custom predicates are supported and if so, how to create/use them.

Add assert() that throws an Error

Clojure's spec has s/assert that validates and then throws an Exception at runtime with explain's output. I had to write something similar:

export function validateSpec(data, aSpec) {
        if (!spec.valid(data, aSpec)) {
            throw new Error(spec.explain(data, aSpec));
        }
}

allow spec.map to work with an empty map

spec.map("sth", {})

throws the error Cannot use Keys spec without keys.

Well, it's the same as spec.object

But it would be nice to do something like this:
spec.map("sth", {[optional]: {a: spec.string}})

So I think there should just be no checking for empty maps, it's ok when they are empty.

standalone browser script

Is there a bundled script for js.spec that can run independently in the browser?

spec.map not working as described

running the example from the documentation

const person = spec.map("person", {
  email: spec.string
  })

// Error: Cannot use Map spec with shape person

I couldnt find anything in the code that would be causing that but after checking this line I figured that somehow the name was being passed as the shape object.

and voila!

spec.map({email: spec.string, [spec.optional]: {}}, "foo")
t {
  options:
   { requiredKeys: [ 'email', 'undefined' ],
     requiredSpecs: { email: [Function: p], undefined: {} },
     optionalKeys: [],
     optionalSpecs: {} },
  name: '' }

sadly, I dont know why or where is the name being swapped with the shape.

For the record I am using:

js.spec: ^1.0.0-1
node: v6.9.4

Get error message `Error: Predicate condition of spec taskParams is null or undefined`

const {spec, valid} = require('js.spec');

const dateRange = spec.map("dateRange", {
  startsFrom: spec.int,
  endsBefore: spec.int,
});

const taskParams = spec.map("taskParams", {
  timezone: spec.string,
  dateRange,
  condition: spec.options,
});

/*
const schema = spec.map("schema", {
  taskId     : spec.string,
  reportType : spec.int,
  callbackUrl: spec.string,
  taskParams,
});
*/

console.log('valid(dateRange, {startsFrom:5, endsBefore 6})',
  valid(dateRange, {startsFrom: 5, endsBefore: 6}));
console.log('valid(dateRange, {})', valid(dateRange, {}));
console.log('valid(dateRange, {startsFrom:"5", endsBefore 6})',
  valid(dateRange, {startsFrom: "5", endsBefore: 6}));
console.log('valid(taskParams, {timezone: "tw", dateRange: {startsFrom:5, endsBefore: 6}, condition: {}})',
  valid(taskParams, {timezone: "tw", dateRange: {startsFrom:5, endsBefore: 6}, condition: {}}));
console.log('valid(taskParams, {timezone: "tw", dateRange: {startsFrom:5, endsBefore: 6}, condition: 1})',
  valid(taskParams, {timezone: "tw", dateRange: {startsFrom:5, endsBefore: 6}, condition: 1}));
console.log('valid(taskParams, {timezone: 2, dateRange: {startsFrom:5, endsBefore: 6}, condition: 1})',
  valid(taskParams, {timezone: 2, dateRange: {startsFrom:5, endsBefore: 6}, condition: 1}));

Error: Predicate condition of spec taskParams is null or undefined, probably that's not your intention.
    at u (/home/ubuntu/workspace/i2w/ce-backend-scheduler-report-generator-marketplace/node_modules/js.spec/dist/js.spec.js:1:883)
    at t.undefinedPredicateWarning (/home/ubuntu/workspace/i2w/ce-backend-scheduler-report-generator-marketplace/node_modules/js.spec/dist/js.spec.js:1:2450)
    at Object.t.default [as map] (/home/ubuntu/workspace/i2w/ce-backend-scheduler-report-generator-marketplace/node_modules/js.spec/dist/js.spec.js:1:26731)
    at Object.<anonymous> (/home/ubuntu/workspace/i2w/ce-backend-scheduler-report-generator-marketplace/spec.js:13:25)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Function.Module.runMain (module.js:676:10)

Publishing and dist

Hello @prayerslayer,

I was publishing a new version of js.spec with np and it seems like the dist/js.spec.js file is included in .gitignore now and therefore is not published. This is one of the reasons why I don't see the changes when I import in node, I guess node uses dist/js.spec.js which is never updated, but I might of course be wrong.

Before changing things I decided to open an issue and ask, so that I don't break things unexpectedly.

Which tool do you use for publishing and is there a reason behind excluding dist/js.spec.js?

Thanks as usual!

Nilable spec

Currently it's a predicate, but that doesn't work as you can't have nilable maps then.

Code documentation

It's really bad.

Registry

In #19 and #20 some confusion arose around the purpose of the registry. Currently it has the following responsibilities, which may or may not be useful:

Store available specs for easier access

So you can do this:

define("string", p.string)
conform("string", "foo")

Instead of always having to do it like this:

const string = p.string
conform(string, "foo")

However the underlying question is probably how different modules share their specs or provide them to "userspace." Should they define all their things to store them in the registry? (I don't think it's possible with npm as, if I recall correctly, you can end up with depdendency A using js.spec v1 and dependency B with js.spec v1.1, effectively using different registry instances) Also, they still would need to share they IDs to look up in the registry.

// module A
export const specA = Symbol("string")
define(specA, p.string)

// module B
import {specA} from 'a'
conform(specA, "foo")

So maybe overall it's better to use regular es6 modules for transporting specs directly.

// module A
export const specA = p.string

// module B
import {specA} from 'a'
conform(specA, "foo")

Which would make the registry obsolete.

Name the specs

The one other reason why the registry exists is that, since you give a semantically useful ID to define, it can auto-name the specs, which is super important for error messages. Generic error messages are not helpful, especially when you consider two syntactically equal specs that have different semantics. (It's in the morning, I don't have a good example ready.)

Here I would just make the name part of a spec's data, it's cleaner anyways.

Compose spec

One of the great features of clojure.spec is the ability of merging specs, because they are essentially data (ok macros are still in the way, but nothing is perfect).

At the moment js.spec does not do that, but it would be great if we could do something like:

const aSpec = S.spec.map("a spec", {
  a: S.spec.object,
  b: S.spec.object,
});

const composedSpec = S.merge(aSpec , {
  c: S.spec.object
}

Ok the example does not make too much sense, but we should be composing and reusing specs.

...I have had a look at the code and this is actually a big shift because of the use of class...still think is worth doing it...maybe the S.merge function can just merge the fields...

Port to Deno?

Have you considered porting to Deno? Deno just hit 1.0; and I think this would be a well received tool. Thoughts?

Improve explainStr message

Hello @prayerslayer! First of all thanks for the port!

I am playing with it from TypeScript and have the following spec:

/**
 * Return true if there are no spaces
 */
export function hasNoSpaces(s: string): boolean {
  const regex = new RegExp(/\s+/);
  return !regex.test(s);
}

export const AggregateSegmentSpec = S.spec.and("s_aggregateSegment", S.spec.string, hasNoSpaces);

When it fails the message is:

"s_aggregateSegment → hasNoSpaces: hasNoSpaces failed for undefined at []."

Which is a bit cryptic and probably has a bug in it (see undefined).

I'll try to have a look but can't promise anything 😄

Docs for spec.or() don't match actual usage

The docs for "or" look like this:

or (name, ...specs) <-- Looks like you can pass in a bunch of specs like and()

Data must conform to at least one provided spec. The order in which they are validated is not defined. conform returns matched branches along with input data.

const big_or_even = spec.or("big or even", {
  'even': spec.even,
  'big': nr => nr > 1000
})```

Perhaps or should be:

or(name, {key1: spec, key2: spec...})

Handle predicates

All the functions like isInteger, isString etc. are not part of clojure.spec as they are available in clojure.core. JS does not have a good standard library, so we need to keep them. Bonus: We control the naming.

• wrap lodash helpers in own function for consistent naming

• export in different part of api, say spec.predicates? mainly to avoid name clashes with specs.

map spec is too strict with object definition

I'm using map() to spec classes that have fields but I am getting rejected because it eventually uses lodash's isPlainObject() does not like my class instance. I tracked it down to some deep inspection of the value's prototype constructor therein that is rejecting my instance.

I'd like to suggest that we change to use lodash isObjectLike() instead. It is simpler and rejects functions as objects. It will accept arrays as objects, and if that is a problem then adding in a test for isArray could allow the object spec to be strict in that regard.

Cannot use map() with only-optional keys

I expect I could spec an object that has only optional keys. Here is a failing unit test:

  describe("map()", () => {
    it("should allow optional-only keys map()", () => {
      const mapSpec = S.spec.map("optional keys", {
        [S.symbol.optional]: {
          banana: S.spec.string
        }
      });

      S.assert(mapSpec, {});
    });
  });

Value generation from specs

This is one of the more interesting features of clojure.spec. Are methods like exercise on the roadmap?

On a related note, what are your thoughts on testcheck-js and how it compares to js.spec?

registry.js typo

https://github.com/prayerslayer/js.spec/blob/master/lib/registry.js#L14

Symobl should be spelled Symbol

Data generation based on the defined spec.

I am planning to use js.spec for one of my projects. Is there a way to generate data based on the defined spec?

Assertions with promises

Hello @prayerslayer!
I am opening this PR in order to see how you feel about adding wrappers around assert (and potentially others) that return a Promise.

I know I could do it on my side (and I am 😄), and it would be as simple as:

/**
 * Fullfill or reject the Promise against the spec.
 */
export function assertAsPromise (spec, value) {
  return new Promise((f, r) => {
    S.assert(spec, data);
    f(data);
  });
};

but I was wondering whether its place should be here (in util.js maybe).

Let me know what you think and I can contribute this back!

Eliminate webpack build

Given js.spec is a node module, nobody is going to <script> it into a browser directly. Instead they will use their own packaging toolchain to do so like browserfy or webpack. Js.spec doesn't need to create a webpacked build.

Not doing so will also increase the understandability of stack traces and the project in general.

False positive or feature?

Hello @prayerslayer,

I was playing again with js.spec but I think I am either getting things wrong or found a problem.

I have the following spec:

export const DD = S.spec.map("DD", {
  name: S.string,
});

export const AD = S.spec.map("AD", {
  name: S.string,
  domain: DD
});

export const AI = S.spec.map("AI", {
  id: S.string,
  definition: AD
});

When I try S.assert(spec.AI, aggId) where aggId is:

{
  "id": "-",
  "definition": {
    "domain": {
      "name": "my-domain-name"
    },
    "name": "order and prices"
  }
}

I get:

Error: AI → AD → Keys(AD): predicate failed for order and prices at [definition, name].
AI → AD → Keys(AD): predicate failed for undefined at [definition, undefined].
AI → AD → Keys(AD): predicate failed for [object Object] at [definition, domain].

Am I doing something wrong?

spec.nilable() doesn't work in map()

I am expecting to use nilable to test the value (or lack thereof) of a field in an object with map().

Here is a unit test:

    const nilableSpec = S.spec.nilable("not required", S.spec.string);

    it("works with values", () => {
      S.assert(nilableSpec, "matt");
      S.assert(nilableSpec, undefined);
    });

    it("does not work in map()", () => {
      const mapSpec = S.spec.map("nilable", {
        banana: nilableSpec
      });

      S.assert(mapSpec, {});
    });
  });```

What I get is `Error: nilable → Keys(nilable): predicate failed for undefined at [banana].` 

It looks like it is failing because there's no `banana` field: https://github.com/prayerslayer/js.spec/blob/master/lib/spec/map.js#L11

Function specification idea

I can think of an AST transform that could instrument functions. Here's some ideas:

• Define specs in separate files/modules (with .spec extension, similar to .d.ts in TypeScript)
• Find a way to map spec modules to JS modules
  • /* @spec path/to/spec.spec */ in JS files?
  • import from "path/to/spec.spec";
• Match specs with function by name?
• Use Webpack loader and Babel AST transform to load specs and instrument JS code

self-referential specs

Maybe I'm missing something really obvious, but how do you make a spec refer to itself?

const folder = spec.map("folder", {
  subFolders: spec.collection("subFolders", folder), // <-- doesn't work, obviously
});