wework/speccy

Well Spectually 🤓 Enforce quality rules on your OpenAPI 3.0.x specifications.

speccy's Issues

Component responses resolving external files in an odd fashion

Description

When referencing external files within components it seems as though the reference requires a director e.g. ./. In the example repo if you remove the ./ on https://github.com/catharsisjelly/speccy-test/blob/316c892b70c6c9d50d5d99aca07c8232f74be3d3/api.yml#L87 then the linting does not pass. However note that the reference https://github.com/catharsisjelly/speccy-test/blob/316c892b70c6c9d50d5d99aca07c8232f74be3d3/api.yml#L70 does not need this ./.

I think this is related somehow to the original problem I had where external references did not seem to be finding other external references. There is an example of this included in the repo, just uncomment line 30.

Context

This will allow people building API's to split their resources into smaller and more manageable chunks

Your environment

Linux Mint running Node 8.10.0 lts

Example in the following repo

https://github.com/catharsisjelly/speccy-test

Don't output diagnostics while parsing schema

Detailed description

Speccy commands output diagnostics about the files that are being parsed:

GET file1.yaml
GET file2.yaml
GET file3.yaml

This is not very useful in most cases, and creates noise in the output of the command.

Context

It's particularly bad for the resolve command, because it results in YAML that can't be parsed without manual editing.

Possible implementation

Ideally Speccy would either not print the list of files being parsed, or have a verbose option that prints this information to stderr.

Your environment

Node 10, Ubuntu 18.04.

Callbacks fail parsing before linting

When using the current version of speccy v0.8.2, callbacks no longer pass schema validation. This is blocking me from upgrading and using some of the new features.

This worked in 0.7.3, I'm not sure if it's this library or another dependency. I tried the official example to make sure it wasn't something wrong my spec.

working

npm install [email protected]
speccy lint https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v3.0/callback-example.yaml -s info-contact -s operation-operationId -s operation-tags -s operation-summary-or-description -s openapi-tags

Specification is valid, with 0 lint errors

not working

npm install [email protected]
speccy lint https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v3.0/callback-example.yaml -s info-contact -s operation-operationId -s operation-tags -s operation-summary-or-description -s openapi-tags

Specification schema is invalid.

#/paths/~1streams/post/callbacks/onData/post

Bug: Speccy doesn't catch duplicate keys

Try duplicating a path in https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v3.0/petstore.yaml and running speccy lint. It won't catch the duplicate key.

Feature: allow speccy to watch and resolve

Detailed description

Many cli tools allow you to -w watch changes to a specified file and re-run execution for the differences.

I propose adding a -w flag to speccy allowing similar behaviour.

speccy resolve -w -v -j myspec.yml -o ~/elsewhere/projects/specs/myapi.yml

Context

Proposed documentation workflow is roughly:

1. Define a spec + schema
2. 'resolve' the spec into an output asset. Publish (somewhere)
3. External docs (static site, using redoc to inline API specs)

This workflow would allow a smoother edit-and-preview workflow for the above toolchain.

Current toolchain uses speccy 0.7.2+

Site does not include path-keys-no-trailing-slash

Detailed description

Speccy is detecting a path key with a trailing slash. It is linking to https://speccy.io/rules/#path-keys-no-trailing-slash which does not exist.

feature: Lint multiple OAS files with glob syntax

Detailed description

The user should be able to provide a glob as the file parameter and have the linter run on each of the files returned from the glob command.

Not sure if this is a bug or enhancement for if a glob is given to the linter, it will lint the first file returned from the command.

Given a package structure like this:

v1/
    openapi.yaml
v2/
    openapi.yaml
package.json

The linter should be able to lint each file when given the command: speccy lint **/openapi.yaml.
Currently, only v1/openapi.yaml is linted.

Context

We have folders where we separate versions of our API. We'd like to be able to lint each specification without hardcoding our versions in our scripts.

component referencing doesn't work across different files

Detailed description

I found the following bug when defining components in a separate file, they don't show up with speccy.

openapi.yaml:

---
openapi: 3.0.0
info:
  version: 1.0.0
  title: OpenAPI Ref Example

components:
  schemas:
    myobject:
      type: object
      properties:
        a:
          description: property a
          type: string
        b:
          type: number

paths:
  /a:
    $ref: './a.yaml'

  /b:
    post:
      summary: b
      parameters:
        - $ref: './parameters.yaml#/foo'
      responses:
        '405':
          description: Invalid input

  /c:
    $ref: './c.yaml'

c.yaml:

post:
  summary: c
  parameters:
    - $ref: './parameters.yaml#/foo'
  requestBody:
    required: true
    content:
      application/json:
        schema:
          $ref: '#/components/schemas/myobject'
  responses:
    '405':
      description: Invalid input

Rendered result:

If I move the definition of c inline (without the $ref: './c.yaml' ) then it's shown correctly.

Your environment

• Node Version: v9.7.1
• Operating system and version (e.g. Ubuntu 16.04, Windows 7): OSX el capitan (10.11.6)

Document how to create custom lint rules

Hello All,
Currently I am validating json file which is in openapi3.0 format. Speccy has predefined rules in it. But I want to create my own rule like we have speccy builtIn rules for validating my json file against it.
If possible need easy steps for creating rule. Don't want to do code much.

Thanks

speccy lint --skip not skipping 0.8.1

When using the current version of speccy 0.8.1, the --skip (-s) option for linting doesn't seem to do anything.

Previously, this would ignore lint rules by rule id.

Working

npm install -g [email protected]
speccy lint https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v3.0/petstore.yaml --skip openapi-tags --skip info-contact
Specification is valid, with 0 lint errors

Not Working

npm install -g [email protected]
speccy lint https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v3.0/petstore.yaml --skip openapi-tags --skip info-contact
Finished resolution!
Specification contains lint errors: 2
#/info  R: info-contact  D: info object should contain contact object
<...snip...>
#/  R: openapi-tags  D: openapi object should have non-empty tags array
<...snip>

TypeError when dealing with schema externalDocs URL

Given the following schema:

openapi: 3.0.1

info:
  title: An API that Speccy hates
  description: Speccy doesn't like my UUID schema.
  contact:
    name: Iain McGinniss
    email: [email protected]
  version: 1.0.0

tags:
- name: demanded-by-speccy
  description: Speccy demands that I have tags defined. Ergo, a tag.

components:
  schemas:
    uuid:
      description: An RFC 4122 formatted UUID.
      type: string
      pattern: '^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$'
      externalDocs:
        description:
          >
          RFC 4122 - A Universally Unique IDentifier (UUID) URN Namespace
        url: https://tools.ietf.org/html/rfc4122.html#section-4.1

paths:
  /api/uuid:
    get:
      operationId: listUuids
      description: Provides a list of UUIDs, for reasons.
      tags:
      - demanded-by-speccy
      responses:
        200:
          description: A list of UUIDs.
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/uuid'

Speccy dies during linting with the following (simplified) stack trace:

TypeError: Cannot read property 'length' of undefined
    at validateUrl (.../node_modules/speccy/lib/validate.js:55:21)
    at .../node_modules/speccy/lib/validate.js:289:2
    at Function._throws (.../node_modules/speccy/node_modules/should/cjs/should.js:705:5)
    at Function.assert.doesNotThrow (.../node_modules/speccy/node_modules/should/cjs/should.js:758:11)
    at checkSubSchema (.../node_modules/speccy/lib/validate.js:288:16)
    at walkSchema (.../node_modules/speccy/lib/walkSchema.js:32:5)
    at checkSchema (.../node_modules/speccy/lib/validate.js:302:5)
    at validateSync (.../node_modules/speccy/lib/validate.js:1087:13)
    at .../node_modules/speccy/lib/validate.js:1250:9
    at Generator.next (<anonymous>)

If the "externalDocs" declaration for the uuid schema is removed, the spec passes with 0 lint errors.

Environment

Include as many relevant details about the environment you experienced the bug in and how to reproduce it.

• Node Version: v9.11.1
• OS Version: macOS High Sierra 10.13.14

How to write rules for parameters?

There's some existing rulesets in place for pathItem, but they don't seem to expand past summary and description on those paths. How can I go about writing a ruleset for enforcement on a parameter, such as requiring it has summary?

speccy expects non-empty tags array while it's not a required parameter in the OpenAPI spec

While trying to lint an OpenAPI 3.0.1 spec without the tags array an error is being thrown as speccy treats tags asa required parameter

However, the OpenAPI spec 3.0.1 defines it as an optional parameter

It should be a warning instead if including tags objects array is considered best practice.

Security requirement with scopes fails schema validation

When using the security object to apply a securityScheme to a resource+method, validation fails when the required scopes array is non-empty. It should be allowed to be non-empty when the securityScheme.type is openIdConnect, or oauth2.

From the OAS 3.0.1 - security requirement

Each name MUST correspond to a security scheme which is declared in the Security Schemes under the Components Object. If the security scheme is of type "oauth2" or "openIdConnect", then the value is a list of scope names required for the execution. For other security scheme types, the array MUST be empty.

Detailed description

Example gist and error, this seems like a bug, but I haven't dug in to see what does schema validation.

speccy lint https://gist.githubusercontent.com/jblazek/4305176861e270aeaa31f32140d46f91/raw/b86617b19939d903c6cfdb3418ecd608226c67fb/openIdConnect.yaml

Specification schema is invalid.
#/paths/~1foo/get/security
expected Array [ 'scope1' ] to be empty

Context

I have a need to restrict access to resources with specific API scopes, and can't do so in the spec currently and lint it with speccy. Other tools I've used to validate like swagger-cli and swagger-ui allow the schema validation as the OAS spec defines.

Your environment

• Speccy version: 0.8.2
• Node Version: 8.7.0
• Operating system and version (e.g. Ubuntu 16.04, Windows 7): Windows 10

short-summary seems very broken

#/paths/~1locales~1/get  R: short-summary  D: summary should be short (description can be long)
expected 'List locales' to have property length of 20 (got 12)

Detailed description

Bug, looks like the expected length logic is not max length, but expects ==.

Separate linting and file loading concerns

Currently, the linter is doing file IO to load the rules as well as performing the linting.

It would be easier to test the linter if the lint function accepted an argument with a set of rules rather than a specific ruleset name that gets internally loaded as a file.

This should help with adding custom rule files, since you could validate the contents of the rules in the loader.

lint --json-schema option does not work

Detailed description

The command line parsing code isn't handling the --json-schema option. I debugged the code a bit and saw that the Config object is empty after the call to config.init(cmd) on line 68 in lint.js.

$ node_modules/.bin/speccy lint -j v2api.yaml 
XXX { config: Config {} }
Usage: speccy <command>

Options:
  -V, --version                    output the version number
  -c, --config [configFile]        config file (containing JSON/YAML). See README for potential values.
  -h, --help                       output usage information

Commands:
  lint [options] <file-or-url>     ensure specs are not just valid OpenAPI, but lint against specified rules
  resolve [options] <file-or-url>  pull in external $ref files to create one mega-file
  serve [options] <file-or-url>    view specifications in beautiful human readable documentation

Your environment

• Node Version: 10.10.0
• Operating system and version (e.g. Ubuntu 16.04, Windows 7): Ubuntu 18.04

Serve does not work with $ref

Either we need to internally resolve files, or make the whole folder available for relative paths.

Detailed description

When running serve on a $ref-filled schema it fails to load the target. Instead of shows the same index.html so we dont even get a 404, just a invalid response error.

For a workaround, resolve the doc before you serve it:

speccy resolve docs/openapi.yml && speccy serve resolved.yaml

Possible implementation

Probably could just update the serve.js command to resolver.resolve(options).then(function(){ // something, which I probably could have done instead of opening this issue, but I'd rather do it properly later and write some darn tests.

Silently fails to resolve $ref which does not exist

Noticed that speccy was giving a valid response on a spec which was $ref'ing a non-existent file.

Detailed description

When linting with the --verbose switch, $ref that points to a missing file shows something like this:

... everything is fine up here...
Creating initial clone of data at #/paths/~1foos/post/responses/400/content/application~1json/schema/properties/result
Skipping ref from other source /Users/psturgeon/src/rails-template/docs/components/schemas/foo.json
GET /Users/psturgeon/src/rails-template/docs/components/schemas/foo.json
{ Error: ENOENT: no such file or directory, open '/Users/psturgeon/src/rails-template/docs/components/schemas/foo.json'
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/Users/psturgeon/src/rails-template/docs/components/schemas/foo.json' }
Finished resolution!
Specification is valid, with 0 lint errors

That needs to fail much harder.

Your environment

• Node Version: v8.11.1
• Operating system: OSX
• Speccy: master

readme doesn't explain "resolve" command

Detailed description

There is no description of the "resolve" command in the readme file. What does resolve do? What options does it take? From a Phil Sturgeon post on APIs You Won't Hate, I gather that resolve "pulls in all the $ref’s to create one mega-file." That's the function that I'm interested in, and it would be nice if the readme said at least a little bit about it to save me the time of downloading it and trying it just to find out. I'm sure it's great, but a description would be nice.

speccy lint fails: UnhandledPromiseRejectionWarning: ReferenceError: message is not defined

Detailed description

Seems to be a bug in the speccy code - message is not always defined, so can cause crash when codes accesses it.

Context

Would be helpful if speccy provided useful error message rather than crashing.

$ speccy lint --skip openapi-tags --skip operation-tags openapi.yaml
Specification schema is invalid.
(node:12112) UnhandledPromiseRejectionWarning: ReferenceError: message is not defined
    at err.errors.map.error (myproject/node_modules/speccy/lint.js:50:31)
    at Array.map (<anonymous>)
    at readableJsonSchemaMessages (myproject/node_modules/speccy/lint.js:39:23)
    at formatSchemaError (myproject/node_modules/speccy/lint.js:30:32)
    at validator.validate (myproject/node_modules/speccy/lint.js:97:28)
    at myproject/node_modules/speccy/lib/validator.js:1274:9
    at <anonymous>
(node:12112) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:12112) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
✨  Done in 0.93s.

Appears to be thrown by this line:

speccy/lint.js

Line 50 in 63cb0ce

return `${dataPath} ${message}`;

Possible implementation

Not sure yet.

Your environment

Include as many relevant details about the environment you experienced the bug in and how to reproduce it.

• Node Version: 8.11.1
• Operating system and version (e.g. Ubuntu 16.04, Windows 7): OS X

Resolving JSON schema failing

I'm seeing a bug when resolving a YAML spec that references a JSON schema. The resolved JSON schema ends up embedded into the spec as a single stringified string value and not as a nested object. It's affecting both theresolve and lint modes of Speccy.

I've found the issue is due to a syntax error in the JSON schema file. Speccy ignores the error and resolves the reference as a string instead of as a nested object.

Detailed description

I have openapi.yaml file like this:

openapi: 3.0.0
info: 
  title: Title
  description: Description
  version: 1.0.0
servers:
  - url: https://host
    description: Description
paths:
  /example:
    get: 
      description: Description
      operationId: getExample
      responses:
        '200':
          description: Example response
          content:
            application/json:
              schema:
                $ref: example-schema.json

The example-schema.json looks like the following. I've created a syntax error in the JSON by removing the , between the type field and the properties field.

{
  "type": "object"
  "properties": {
    "example": {
      "type": "string"
    }
  }
}

I've ran the resolve mode with the following command:

node_modules/.bin/speccy resolve openapi.yaml -o resolved.yaml

The resolved.yaml file outputted by Speccy looks like the following. The schema has been embedded in a string rather than an error being raised by Speccy.

openapi: 3.0.0
info:
  title: Title
  description: Description
  version: 1.0.0
servers:
  - url: 'https://host'
    description: Description
paths:
  /example:
    get:
      description: Description
      operationId: getExample
      responses:
        '200':
          description: Example response
          content:
            application/json:
              schema: "{\n  \"type\": \"object\"\n  \"properties\": {\n    \"example\": {\n      \"type\": \"string\"\n    }\n  }\n}\n"

Context

The lint mode was failing with a messaging saying it was not expecting the schema to be a string. The error it printed out said it expected the schema to be an object and not a string

It was difficult to track down the root cause. It would be much quick to diagnose for future people if Speccy raised an error for the root cause which was that referenced JSON file contained a syntax error.

Possible implementation

Raise an error when a referenced JSON or YAML file contains a syntax error.

Your environment

• Mac
• Node v9.10.0

speccy fails on external (http) references

Detailed description

Either this is a bug or I'm doing something wrong. I'm running speccy lint on a file which references a remotely-hosted file and speccy seems to be treating it like a local file and complaining that it doesn't exist.

Context

I'm developing an API and if there's an error in an API I use, I want to pass the third party API's error to the client. So in my API, I want to just say "errors will be of the type described here:" and refer to a third party spec hosted remotely. (Given that this issue appears unreported, perhaps this is not a common use case? How are other people dealing with these situations?)

Possible implementation

Unsure w/o diving into the code.

Your environment

• Node Version: 8.11.1
• Operating system and version (e.g. Ubuntu 16.04, Windows 7): OS X
• Speccy version: 0.7.1

To reproduce:

1. Download the simple petstore example spec.

2. Change the error ref to refer to an externally hosted file, for example:

--- petstore.yaml.orig	2018-05-30 13:16:26.000000000 -0700
+++ petstore.yaml	2018-05-30 13:16:33.000000000 -0700
@@ -52,7 +52,7 @@
           content:
             application/json:
               schema:
-                $ref: "#/components/schemas/Error"
+                $ref: "http://raw.githubusercontent.com/stripe/openapi/master/openapi/spec3.yaml"
   /pets/{petId}:
     get:
       summary: Info for a specific pet

(The actual ref in the above example would have #components/schemas/error appended but this simpler example fails too.)

3. run npx speccy lint --skip openapi-tags --skip operation-tags --skip info-contact petstore.yaml --verbose

(Note, skipping a few lint rules above just because the example doesn't include tags or contact details, but error occurs either way.)

You'd expect to have it declared valid, but this error is reported:

GET /project/node_modules/speccy/rules/default.json
Found 17 rules in default: parameter-description,parameter-name-regex,operation-operationId,operation-summary-or-description,operation-tags,path-keys-no-trailing-slash,server-trailing-slash,openapi-tags,openapi-tags-alphabetical,reference-no-other-properties,pathItem-summary-or-description,example-value-or-externalValue,reference-components-regex,no-script-tags-in-markdown,info-contact,license-apimatic-bug,no-eval-in-descriptions
GET petstore.yaml
GET http://raw.githubusercontent.com/stripe/openapi/master/openapi/spec3.yaml 
Error: Invalid $ref, pointing to a missing or inaccessable file: ENOENT: no such file or directory, open 'http://raw.githubusercontent.com/stripe/openapi/master/openapi/spec3.yaml'
    at readFileAsync.then.then.catch.err (/project/node_modules/speccy/lib/resolver.js:230:21)
    at <anonymous>
Invalid $ref, pointing to a missing or inaccessable file: ENOENT: no such file or directory, open 'http://raw.githubusercontent.com/stripe/openapi/master/openapi/spec3.yaml'

"Would You Like To Know More?" Links

Detailed description

Right now the errors are not always clear to folks who aren't super into OpenAPI specifications, so we need to provide links to more information.

Context

Both @rachelsison and @iainmcgin have requested this feature this week. 😅

Possible implementation

An easy step forward would be linking to the anchor that relates to the object being linted, so if we're saying "openapi" is missing the "tags" object, we can link to https://swagger.io/specification/#oasObject, which describes what the tags object should be and offers a link to learn more.

I'd also like to get a mini site up on speccy.io, which will explain each of the built in suggestions in more detail. Why do you need a tags object? Do I really need to write descriptions for parameters?

Unsupported types fail without reporting why

Detailed description

If an unsupported type is used in a json file (other than Data Types), the linter will solely output err is not defined.

It does report properly in a yaml file

To Replicate

1. Create a json file for some portion of your documentation.
2. Instead of a supported datatype, put foo.
3. Run the linter
4. Profit

Show specific line error of customer rules

hi there, I'm working with openapi3 validation job
I validate our own swagger(very big api file) with this tool. but it just showed
#/paths/~1me~1manager/patch/parameters/0

so confused where I can find the error detail info.

Skip is ignored

Detailed description

I believe that speccy is ignoring speccy.yaml and the --config option. I can skip warnings with --skip, but not by putting them in speccy.yaml.

$ node_modules/.bin/speccy --config ./speccy.yaml lint v2api.yaml |& grep -c parameter-description
204
$ node_modules/.bin/speccy --skip parameter-description lint v2api.yaml |& grep -c parameter-description
0
$ cat speccy.yaml
lint:
  skip:
  - parameter-description

Your environment

• Node Version: 10.10
• Operating system and version (e.g. Ubuntu 16.04, Windows 7): Ubuntu 18.04

Is it possible to validate response schema with speccy tool.

Hello All,
Currently I am able to validate my API specification which is in OpenAPI 3.0 format with default and custom rules. But wanted to validate response too with speccy tool. For example in my specification I have schema's under responses which return data, so I want to validate the schema's with data.
How to do with speccy tool any thoughts?

Different exit codes for different errors

Detailed description

Right now, it looks like Speccy always exits with 1, regardless of whether a validation error or a lint error occurs. I think it could be handy to have it exit with 1 when there are only validation errors, with 2 if there are only lint errors, and with 3 if there are both.

Context

I'm planning to use this in CI and am unsure yet if I will respect all lint errors, particularly the one about the contact object.
I would use this to make comparisons with the exit code, to make the CI fail exactly when I want it to.

How can it benefit other users? Other users with old, valid definitions that don't pass linting would be able to start using this tool in CI, and slowly make their schema valid

Possible implementation

Store one integer (0 or 1) for each problem type, multiply one of them by 2, then sum

Your environment

Not relevant I think.

Reference error when referencing parameter array

I'm trying to reference an array of parameter objects and I get a reference error when I run it through the linter.

Detailed description

In my main openapi.yaml file, I have a path that is defined as follows:

  /datasets/{datasets}/parts:
    get:
      tags:
      - Datasets
      operationId: datasets.parts
      summary: Get Dataset Parts
      description: Get parts for a given group of datasets.
      parameters:
        $ref: parameters/parts.yaml
      responses:
        '200':
          description: Parts Response
          content:
            application/json:
              schema:
                $ref: schemas/parts.index.json
        '401':
          $ref: '#/components/responses/UnauthorizedError'

parameters/parts.yaml looks like the following:

- in: path
  name: datasets
  description: One or more dataset numbers, separated by commas.
  required: true
  schema:
    type: array
    items:
      type: integer
  explode: true
- in: query
  name: filters
  description: One or more query objects.
  required: false
  schema:
    type: array
    items:
      $ref: ../schemas/filter.json
- in: query
  name: partnumber
  required: false
  description: One or more comma separated part numbers. API will return any part numbers that match the strings.
  examples:
    single:
      value: ABC123
    multiple:
      value: ABC123,ABC1234,DEF456
  schema:
    type: array
    items:
      type: string
- in: query
  name: partnumber_vendor
  required: false
  description: One or more comma separated part numbers. API will return any part numbers that match the strings.
  examples:
    single:
      value: ABC123
    multiple:
      value: ABC123,ABC1234,DEF456
  schema:
    type: array
    items:
      type: string

When I run the linter, I get the following error:

Expected object or boolean as schema, got array
ReferenceError: err is not defined
    at /media/jloosli/storage/projects/partners/node_modules/oas-resolver/index.js:215:26
ReferenceError: err is not defined
    at /media/jloosli/storage/projects/partners/node_modules/oas-resolver/index.js:215:26
err is not defined

If I keep that same definition inline, then I don't have any errors.

Context

When displaying my API with Redoc, it handles the parameters correctly. I can keep this inline, but I'd like to separate out components a little more since my openapi.yaml file is getting quite large. It seems like this would be valid, but I'm not 100% sure.

Your environment

• Ubuntu 18.04
• Node 10.9.0

API isn't live yet, so no external links right now.

Document resolve command usage

The readme doesn't have a section for the resolve command.

Detailed description

The usage sections cover the lint and and serve commands, but not resolve. I started using a different utility to do this operation, since it wasn't clear that speccy could also do this.

Context

This would make it more clear about the full capabilities of this tool.

Regex errors should show regex pattern

Instead of just saying regex it should say what regex.

Detailed description

reference components should all match spec. regex

Not a wildly helpful error as it stands.

Request for an official Docker image

It would be great if you could provide an official docker image (published on DockerHub) with speccy installed from NPM so that it could easily be referenced by different CI tools natively.

I found this, but it lacked any credibility and doesn't include the Dockerfile source.

It would be something as simple as this added to this repo:

FROM node:latest
RUN npm install --global speccy

back-$ref's in a referenced file no longer resolve in speccy 0.8.x

If openapi.yml $ref's otherfile.yml, $ref's in otherfile.yml do not resolve.

Detailed description

To reproduce, I ran speccy 0.7.3 and 0.8.4 over the same Open API definition.

This screenshot shows that speccy 0.7.3 resolves the $ref from ping.yml to openapi.yml and 0.8.4 does not (it produces an empty object instead).

Context

I define each URL/verb combination as an endpoint and all responses as JSON Schema and use speccy to combine/resolve between the relevant files and formats. An example of this usage is in this example Open API definition (see the npm scripts for speccy cli usage).

Example

File: openapi.yml

...
paths:
    /ping:
       $ref: "endpoints/ping.yml#/paths/~1ping"

...

components:
    schemas:
        ping:
            $ref: 'schemas/ping.json'
...

File: endpoint/ping.yml

paths:
    '/ping':
        get:
          responses:
            '200':
              content:
                application/json:
                    schema:
                        $ref: '../openapi.yml#/components/schemas/ping'

Environment

• Node Version: v8.9.4
• OS: MacOS 10.13.4
• Link to your project: Reproduced in this example Open API definition.

bug: anyOf in schemas are not resolved

Detailed description

When given a Schema Object where the type field is within an anyOf, the linter reports an error.

Consider the schema:

components:
  schemas:
    AnyOfExample:
      type: object
      properties:
        intOrFloatProperty:
          anyOf:
            - type: number
               format: float
            - type: integer
               format: int32
      required:
        - intOrFloatProperty

The linter reports:

expected Object {
  anyOf: Array [
    Object { type: 'number', format: 'float' },
    Object { type: 'integer', format: 'int32' }
  ],
  ... snip ...
} to have property type

Context

anyOf is an important aspect of JSON Schema, which OAS has adopted. It's a valid part of the specification and should be supported in linters.

Your environment

• Node Version: 8.11.3
• macOS 10.13.6

Lint rule suggestion: unused components

I would like to be able to validate everything in components is actually referenced somewhere and fail linting when an unused schema, example, or other element in that section is found.

We have cases where things might get moved around in a spec and it might not be clear something is no longer used.

Combine the options.context log info

Try to make a user friendly log info, the options.context Object should have a strict uniformed structure to show.

--skip for linter, to skip specific rule actions

I'm working with some rather messy OAI v3 repos that don't belong to me, so sometimes I want to skip the parameter-description rule. I think this would be a handy feature for getting moooore feedback, especially as right now it's one rule at a time.

Support JSON schema patternProperties

Right now, referencing a json-schema that uses "patternProperties" seems to cause a linter error. Would be great if this standard (draft 4) feature was supported without throwing errors. This is currently not supported by openapi 3, but if you use json-schema through https://github.com/wework/json-schema-to-openapi-schema, you need this property.

#/paths/~1api~1v7~1floors/get/responses/200/content/application~1json/properties/attributes
Schema object cannot have additionalProperty: patternProperties

Detailed description

Here's an example of a json schema that will cause speccy to error:

"properties": {
  "attributes": {
      "description": "Members of the attributes object (\"attributes\") represent information about the resource object in which it's defined.",
      "type": "object",
      "patternProperties": {
        "^(?!relationships$|links$)\\w[-\\w_]*$": {
          "description": "Attributes may contain any valid JSON value."
        }
      },
      "additionalProperties": false
    }
}

Context

I am currently using jsonapi's json-schema: https://jsonapi.org/schema and this schema uses patternProperties. This is very helpful when documenting a jsonapi to take use of the allOf inheritance pattern and conform to jsonapi.

Your environment

Include as many relevant details about the environment you experienced the bug in and how to reproduce it.

• Node Version: v10.1.0

Question: How to resolve external files inside json-schema files using $ref

Detailed description

Which is the proper way to reference external files inside json-schema files?

For example

With this openapi.yml file:

openapi: "3.0.1"
info:
  title: "OPENAPI Test"
  description: "test"
  version: "1.0"
  contact:
    name: test
    url: test.com
    email: [email protected]
paths:
  /user:
    get:
      summary: Retrieve user information
      description: 'schema test'
      tags:
      - User
      operationId: user.details
      responses:
        200:
          description: Success
          content:
            application/json:
              schema:
                $ref: 'spec/support/api/schemas/user.json'
tags:
- name: User

If we have two json-schemas:

Filename: spec/support/api/schemas/user.json

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": “file:/user.json",
  "type": "object",
  "required": ["account"],
  "properties": {
    "account": {
      "$ref": "file:/account.json#"
    }
  }
}

Filename: spec/support/api/schemas/account.json

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "file:/account.json",
  "type": "object",
  "required": ["balance"],
  "properties": {
    "balance": {
      "type": "number"
    }
  }
}

The following test passes:

require 'rails_helper'

describe 'user schema' do
  it ‘should be a valid schema' do
    user = {
      account: {
        balance: 23
      }
    }
    expect(user).to match_json_schema('user')
  end
end

But speccy lint openapi.yml -v --json-schema fails with the following error:

GET /example/node_modules/speccy/rules/default.json
Found 17 rules in default: parameter-description,parameter-name-regex,operation-operationId,operation-summary-or-description,operation-tags,path-keys-no-trailing-slash,server-trailing-slash,openapi-tags,openapi-tags-alphabetical,reference-no-other-properties,pathItem-summary-or-description,example-value-or-externalValue,reference-components-regex,no-script-tags-in-markdown,info-contact,license-apimatic-bug,no-eval-in-descriptions
GET openapi.yml
GET /example/spec/support/api/schemas/user.json 
Rewriting external ref file:/account.json# as file:///account.json# 
Finished internal resolution
Creating initial clone of data at #/paths/~1user/get/responses/200/content/application~1json/schema
GET file:///account.json #
{ Error: ENOENT: no such file or directory, open 'file:///account.json'
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'file:///account.json' }
{ Error: ENOENT: no such file or directory, open 'file:///account.json'
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: 'file:///account.json' }
ENOENT: no such file or directory, open 'file:///account.json'

Context

I'm trying to use both speccy and json_matchers to validate and document my api.

Your environment

• Node Version: v8.11.3
• speccy 0.8.0
• json matchers 0.10.0.

Output multiple validation problems at once

Detailed description

At the moment when there are multiple problems in an OpenAPI document, Speccy will only output the first problem. For instance when a component that is being referred to using $ref is missing, it'll only output the first instance where that $ref is used.

Context

When fixing these problems requires a build step in another system it is very inconvenient to solve one of these at a time. It would be very nice if speccy would output multiple (or all!) of these problems at once.

In addition when Speccy will be used as a linter inside editors it makes sense to show all problems and not just showing the next one once the first one is fixed.

Possible implementation

Currently should is used to return validation errors. It uses exceptions, which aren't easily composable. It probably requires a lot of work, but all functions should return an array of problems. An alternative is to catch each error and compose a new one when multiple validations are being done in a single function.

npm cli can't open lint rules file

I was able to run speccy lint, and wanted to try a local custom rules file. When I do this, I get an error that looks like speccy is looking in my profile for the file.

This happens for a file that exists in the current directory as well as an HTTP resource.

Checklist

None of the checklist applied to this scenario.

Detailed Description

1. installed npm package with npm install -g speccy
2. ran speccy lint spec.yaml, got a validation rule I don't care about
3. made a copy of default.json and removed the rule for contact info
4. ran speccy lint -r custom.json spec.yaml, got the error below.
5. ran speccy lint -r http://source/raw/rules spec.yaml, got the same error except the file tried was C:\Users\jblazek\AppData\Roaming\npm\node_modules\speccy\rules\http://source/raw/rules.json

• note my incoming param didn't have a .json extension but it gets added.

I'm using Windows 10, and tried from powershell and bash with the same result.

Error: ENOENT: no such file or directory, open 'C:\Users\jb\AppData\Roaming\npm\node_modules\speccy\rules\custom.json.json'
    at Object.fs.openSync (fs.js:653:18)
    at Object.fs.readFileSync (fs.js:554:33)
    at deepMergeRules (C:\Users\jb\AppData\Roaming\npm\node_modules\speccy\lib\linter.js:13:22)
    at Object.loadRules (C:\Users\jb\AppData\Roaming\npm\node_modules\speccy\lib\linter.js:38:42)
    at Command.command (C:\Users\jb\AppData\Roaming\npm\node_modules\speccy\lint.js:79:10)
    at Command.listener (C:\Users\jb\AppData\Roaming\npm\node_modules\speccy\node_modules\commander\index.js:315:8)
    at emitTwo (events.js:125:13)
    at Command.emit (events.js:213:7)
    at Command.parseArgs (C:\Users\jb\AppData\Roaming\npm\node_modules\speccy\node_modules\commander\index.js:651:12)
    at Command.parse (C:\Users\jb\AppData\Roaming\npm\node_modules\speccy\node_modules\commander\index.js:474:21)

Spaces in paths cause errors (breaking change in v0.8.0)

Detailed description

The speccy resolve command returns the following error, which I suspect are caused by the spaces in my (Windows) paths.

GET c:/My%20API/some-file.yaml
{ [Error: ENOENT: no such file or directory, open 'c:\My%20API\some-file.yaml']
  errno: -4058,
  code: 'ENOENT',
  syscall: 'open',
  path:    'c:\\My%20API\\some-file.yaml' }

As you can see, it seems to be replacing the spaces with %20, which I suspect is the cause of the error.

This error only occurs with Speccy v0.8.0. It works fine with v0.7.3.

Your environment

Include as many relevant details about the environment you experienced the bug in and how to reproduce it.

• Node Version: v10.8.0
• Operating system and version: Windows 10

path-keys-no-trailing-slash is being detected on root path

Detailed description

Speccy is detecting a / path with path-keys-no-trailing-slash as an error. This should be excluded as / is a valid path.

Context

We use the / path to show the current timestamp, version, and other misc. information. We should be able to document this, and it is valid OAI as far as I can see.

Your environment

Include as many relevant details about the environment you experienced the bug in and how to reproduce it.

• Node Version: 8.11.3
• Operating system and version (e.g. Ubuntu 16.04, Windows 7): macOS 10.14
• Link to your project: private
• Speccy version: 0.8.0
• Ruleset: strict

Re-usable links not linting

I'm fairly new to using OpenAPI 3.0 so it's possible that I don't entirely understand it. but for some reason the re-usable links don't seem to validate properly although Swagger's editor seems to think it's fine.

Detailed description

The following OpenAPI spec validates fine without link references, but I was wanting to make the Link a reference

openapi: "3.0.1"
info:
  description: "Something to demo that Link components are not linting"
  version: "1.0.0"
  title: "Test API"
  contact:
    name: "Chris Lock"
    url: "https://catharsis.co.uk"
servers:
  - url: "https://api.not-a-real-api.com"
tags:
  - name: "administrators"
    description: "Administrator end points"
paths:
  /administrators/{userId}:
    parameters:
      - $ref: "#/components/parameters/userId"
    put:
      operationId: "assignAdministrator"
      tags:
        - "users"
        - "administrators"
      summary: "Assign an Administrator to a User"
      requestBody:
        $ref: "#/components/requestBodies/administratorRequest"
      responses:
        200:
          $ref: "#/components/responses/administratorResponse"
        404:
          $ref: "#/components/responses/404"
        405:
          $ref: "#/components/responses/405"
        410:
          $ref: "#/components/responses/410"

components:
  securitySchemes:
    oauth_auth:
      type: "oauth2"
      flows:
        clientCredentials:
          tokenUrl: "/oauth/token"
          scopes:
            "read:user": "read a users"
            "write:user": "write the users"
        password:
          tokenUrl: "/oauth/token"
          refreshUrl: "/oauth/token"
          scopes:
            "read:user": "read a users"
            "write:user": "write the users"

  parameters:
    userId:
      description: "The id of the User"
      in: "path"
      name: "userId"
      required: true
      schema:
        type: "integer"

  requestBodies:
    administratorRequest:
      content:
        application/json:
          schema:
            $ref: "#/components/schemas/Administrator"

  responses:
    204:
      description: "No content"
    404:
      description: "Resource not found"
    405:
      description: "Invalid input"
    410:
      description: "Resource has been deleted"
    administratorResponse:
      description: "The Administrator object response"
      content:
        application/json:
          schema:
            $ref: "#/components/schemas/Administrator"
      links:
        getUserById:
          operationId: "getUser"
          parameters:
            userId: "$request.path.userId"

  schemas:
    Administrator:
      type: "object"
      properties:
        id:
          type: "integer"
          format: "int64"
          readOnly: true
        name:
          type: "string"
        mail:
          type: "string"
          format: "email"
        enabled:
          type: "boolean"

security:
  - oauth_auth: []

But the below does not validate using Speccy but again using the Swagger editor it seems fine

openapi: "3.0.1"
info:
  description: "Something to demo that Link components are not linting"
  version: "1.0.0"
  title: "Test API"
  contact:
    name: "Chris Lock"
    url: "https://catharsis.co.uk"
servers:
  - url: "https://api.not-a-real-api.com"
tags:
  - name: "administrators"
    description: "Administrator end points"
paths:
  /administrators/{userId}:
    put:
      operationId: "assignAdministrator"
      tags:
        - "users"
        - "administrators"
      summary: "Assign an Administrator to a User"
      requestBody:
        $ref: "#/components/requestBodies/administratorRequest"
      responses:
        200:
          $ref: "#/components/responses/administratorResponse"
        404:
          $ref: "#/components/responses/404"
        405:
          $ref: "#/components/responses/405"
        410:
          $ref: "#/components/responses/410"

components:
  securitySchemes:
    oauth_auth:
      type: "oauth2"
      flows:
        clientCredentials:
          tokenUrl: "/oauth/token"
          scopes:
            "read:user": "read a users"
            "write:user": "write the users"
        password:
          tokenUrl: "/oauth/token"
          refreshUrl: "/oauth/token"
          scopes:
            "read:user": "read a users"
            "write:user": "write the users"

  links:
    getUserById:
      operationId: "getUser"
      parameters:
        userId: "$request.path.userId"

  parameters:
    userId:
      description: "The id of the User"
      in: "path"
      name: "userId"
      required: true
      schema:
        type: "integer"

  requestBodies:
    administratorRequest:
      content:
        application/json:
          schema:
            $ref: "#/components/schemas/Administrator"

  responses:
    204:
      description: "No content"
    404:
      description: "Resource not found"
    405:
      description: "Invalid input"
    410:
      description: "Resource has been deleted"
    administratorResponse:
      description: "The Administrator object response"
      content:
        application/json:
          schema:
            $ref: "#/components/schemas/Administrator"
      links:
        getUserById:
          $ref: "#/components/links/getUserById"

  schemas:
    Administrator:
      type: "object"
      properties:
        id:
          type: "integer"
          format: "int64"
          readOnly: true
        name:
          type: "string"
        mail:
          type: "string"
          format: "email"
        enabled:
          type: "boolean"

security:
  - oauth_auth: []

Context

I'll try to dig into why this is happening but I have not had a look at how speccy works as yet but I think it would be beneficial for other people who might want to use links as re-usable components.

Your environment

Nothing really special to mention here a basic install of Node 8.10.0 (lts) on Linux Mint

Linux chris-H81M-S2V 4.13.0-36-generic #40~16.04.1-Ubuntu SMP Fri Feb 16 23:25:58 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Suggest lint-staged

To avoid needing to make speccy understand anything about Git, let's point folks to lint-staged in the README, and make sure it works properly.

Project / User dir Config file

Much like rubocop, eslint, editorconfig, etc., we need a config file that will let folks set the rules files to use, specify skips.

We'll probably add more config options over time, but to start this would suffice.

Let's use YAML as its OpenAPI.

lint:
    rules: 
        - strict
        - ./config/foo-rules.json
    skip: 
        - info-contact

URI fragments and --json-schema option

The --json-schema is exactly what I need, but it does not work well with JSON pointers.

Here is a minimal example:

def.json:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "definitions": {
    "foobar": {
      "type": ["string", "null"]
    }
  }
}

test.yaml:

openapi: 3.0.0
info:
  version: 1.0.0
  title: Test
components:
  schemas:
    $ref: ./def.json
paths:
  /:
    get:
      parameters:
        - name: foobar
          in: query
          schema:
            $ref: ./def.json#/definitions/foobar

Output of speccy resolve test.yaml -j:

openapi: 3.0.0
info:
  version: 1.0.0
  title: Test
components:
  schemas:
    definitions:
      foobar:
        type:
          - string
          - 'null'
paths:
  /:
    get:
      parameters:
        - name: foobar
          in: query
          schema:
            type:
              - string
              - 'null'

It should render a nullable, but the type is left untouched.

Use speccy/lib/loader as module?

Detailed description

I have a project where I'm defining specs separately from code, using OpenAPI and JSON-Schema (thank you Speccy 0.6!). Using speccy to resolve includes and run redoc works very nicely. However, I'd like to use the resolved and loaded spec as an object, so I can use it directly in code. For example, providing documentation for endpoints, validating input, and so on.

Where I have behaviour that I can define as part of spec, I want to be able to use that spec to implement the behaviour instead of duplicating the configuration.

Context

Right now, I'm doing this by directly requiring speccy/lib/loader

'use strict';

const path = require('path');
const loader = require('speccy/lib/loader');

module.exports = () => loader.readOrError(
    path.join(__dirname, 'app.spec.json'),
    {
        jsonSchema: true,
        resolve: true
    }
)

Possible implementation

At this point, I don't have any real code changes or features to request. I'm just curious about how stable this approach will be over time. Are these internals expected to change?

Or conversely, if this is a good idea, should I submit some documentation? ;)

Linter show more than one rule at a time