payu/openapi-validator-middleware

Input validation using Swagger (Open API) and ajv

openapi-validator-middleware's Issues

Inconsistent body validation behavior

I find there are some inconsistencies in the validation of request bodies, especially with multipart data. First of all, empty body produces TypeError: validator.validate is not a function on validation, where specification requires a multipart body. Secondly the Content-Type attribute is not checked, I can send multipart body where e.g. a JSON body is expected.

I am not sure if this is the desired behavior. But from my point of view the existence of the body should be checked during validation if the specification requires a body in the request. Otherwise the consumer of the API will see a mysterious error message and will not be informed about the specification violation.

My setup is like follows:

Specification

    post:
      requestBody:
        required: true
        content:
          multipart/form-data:
            schema:
              type: object
              properties:
                image:
                  type: string
                  format: binary
                  ...

Route

router.post(
  "/users",
  multer().single("image"),
  validator.validate,
...
);

Making a request to this endpoint without a body produces an error:

TypeError: validator.validate is not a function
    at Middleware._validateBody (/home/serg/Documents/Projects/drm/services/user-management/node_modules/openapi-validator-middleware/src/middleware.js:92:28)
    at Middleware._validateRequest (/home/serg/Documents/Projects/drm/services/user-management/node_modules/openapi-validator-middleware/src/middleware.js:63:43)
    at /home/serg/Documents/Projects/drm/services/user-management/node_modules/openapi-validator-middleware/src/middleware.js:51:90
    at Middleware.validate [as validationMiddleware] (/home/serg/Documents/Projects/drm/services/user-management/node_modules/openapi-validator-middleware/src/frameworks/express.js:4:24)
    at Middleware.validate (/home/serg/Documents/Projects/drm/services/user-management/node_modules/openapi-validator-middleware/src/middleware.js:41:21)
    at Layer.handle [as handle_request] (/home/serg/Documents/Projects/drm/services/user-management/node_modules/express/lib/router/layer.js:95:5)
    at next (/home/serg/Documents/Projects/drm/services/user-management/node_modules/express/lib/router/route.js:137:13)
    at multerMiddleware (/home/serg/Documents/Projects/drm/services/user-management/node_modules/multer/lib/make-middleware.js:18:41)
    at Layer.handle [as handle_request] (/home/serg/Documents/Projects/drm/services/user-management/node_modules/express/lib/router/layer.js:95:5)
    at next (/home/serg/Documents/Projects/drm/services/user-management/node_modules/express/lib/router/route.js:137:13)

But I expect to see a validation error here. I assume that this is a bug, because a application/json body does not cause this error. Although the validation of a missing (but required in spec) raw body correctly indicates that the required fields are not present, but it would be more correct to report that a body itself is missing in the request.

So please let me know if my expectations are legitimate or if I have misunderstood the world.

Unknown query parameters handling

It seems that free-form query parameters is not supported by the validator.
For this yaml:

      - in: query
        name: params
        required: false
        schema:
          type: object
          additionalProperties: {}
        example:
          user_id: 123456

The Swagger shows this for a valid request to send:

However, when I make the request I get Input validation error:

Error: Input validation error
    at _validateRequest (/home/roees/Desktop/CommonStorage/node_modules/openapi-validator-middleware/src/middleware.js:62:21)
{ .... }

Allow json schema object as init parameter (TypeScript)

Instead of a yaml file I only have a json swagger document object. When digging into the code I figured out that I can pass the document to the init method as follows:

swaggerValidator.init(document as any)

Is it possible to change the init signature (TypeScript) to allow objects as well?

Unable to catch the error

I'm currently using it with google cloud functions + express setup.
Surprisingly, I'm unable to capture the error on validation failure.

I'm using validate at the router path and wondering if anyway I can debug this.

Coercion doesn't work correctly due to Swagger 2.0 JSON Schema limitations

Since Swagger 2.0 doesn't support specifying multiple types for a field (as JSON Schema does), it is not possible to correctly coerce numeric types as Ajv converts null integers into 0 if null is not one of possible types, and typically this is a very bad idea.

My suggestion is to provide slightly adjusted JSON schema to the Ajv for validation, using following rule: if property is not listed as required, it has one additional type - 'null'. Probably this behaviour should be regulated via a middleware option.

I can work on this functionality if approved :).

Additional properties parameter is not shown in validation error

Experience: The error message is not descriptive when using flag additionalProperties: false, which avoids adding attributes which are not part of the described schema.

Current behavior: error message contains the message without the parameters.
body should NOT have additional properties

Expected behavior: body should NOT have additional properties [<param>]

Validation fails on Swaggers without basePath

middleware.js:

let parsedPath = dereferenced.basePath !== '/' ? dereferenced.basePath.concat(currentPath.replace(/{/g, ':').replace(/}/g, '')) : currentPath.replace(/{/g, ':').replace(/}/g, '');

This will fail if no basePath is specified in Swagger.

Express multer with openapi 3

Hi, I want to use openapi/swagger version 3 and I have couple of endpoints using multipart/form-data.

Not sure if this is a bug or I'm doing something wrong.

My setup:

const inputValidationOptions = {
    expectFormFieldsInBody: true
};
swaggerValidator.init('src/openapi/api.schema.yml', inputValidationOptions);

const multerMiddleware = multer({ ... });

router.put(
  '/:id',
  multerMiddleware.single('photo'),
  swaggerValidator.validate,
  async (req: Request, res: Response) => {
    // ...
  }
);

Swagger:

requestBody:
  content:
    multipart/form-data:
      schema:
        type: object
        properties:
          photo:
            type: string
            format: binary

Getting this error:

TypeError: validator.validate is not a function

Checked the source code and in src/middleware.js -> _validateBody is this const validator = methodSchema.body[contentType] || methodSchema.body;. So it's using content-type header to get the validator. In my case the value is multipart/form-data; boundary=--------------------------517000622299074484111234, but it's expecting only multipart/form-data. Thus it can't access the validator.

Did I miss something or it's issue in middleware?

Endpoint that do not have any parameters fail with an error

let bodySchema = dereferenced.paths[currentPath][currentMethod].parameters.filter(function (parameter) { return parameter.in === 'body' });

This line predictably fails if parameters are null or undefined, which is a valid case.

Default values

I need a validator that inserts default values in the request body AND honors with discriminators, yours does the latter, but not the former. I have found others that are the other way around.

Am I doing something wrong, or is it a know issue that this package does not insert default values from the schema?

Performance problem

Currently it's taking around 1.2 seconds to load the module. Any way to get better loading times?

Validation fails to parse string formData

If parameters look like this:

     parameters:
       - name: username
         in: formData
         required: true
         type: string
         description: Authentication username.
       - name: password
         in: formData
         required: true
         type: string
         description: Authentication password.
       - name: params

then express-ajv-swagger-validation fails during initialization:

(node:14192) UnhandledPromiseRejectionWarning: TypeError: Cannot set property 'username' of undefined
at parameters.forEach.parameter (C:\sources\crm-api\api\node_modules\express-ajv-swagger-validation\src\middleware.js:278:41)
at Array.forEach ()
at buildParametersValidation (C:\sources\crm-api\api\node_modules\express-ajv-swagger-validation\src\middleware.js:257:16)
a

For some reason destination.properties is not set in such case.

Error with external $ref pointers in open api 3

Hi guys.
Recently update from 'swagger 2.0' to 'open api 3'
I used external $ref pointers in my swagger file like this:

paths:
  /countries:
    $ref: './validator/countries.yaml#/routes/base'

For 'swagger 2.0' it's working perfectly fine but for 'open api 3' it throws an error

TypeError: Cannot read property 'countries.yaml#' of undefined

It began working if we change $ref to this style:

    $ref: './validator/countries.yaml/routes/base'

But this is nonconventional use of the $ref that doesn't support by the other libraries.

Is there any solutions for this Issue?

Make it optional to add basurl to path in _getParameters - Express

would it be possible to make the concat optional based on prop form options obj?

using this with routing and would have to append basePath to all paths in schema for paths to match, could avoid this if it was possible to avoid baseurl.

my routes go via /api, but paths in schema do not have this prefixed, thus causing a mismatch that results in the request not being validated.

Req points to /api/somewhere
While path goes directly to /somewhere

ref:

function _getParameters(req) {
const requestOptions = {};
const path = req.baseUrl.concat(req.route.path);
requestOptions.path = path.endsWith('/') ? path.substring(0, path.length - 1) : path;
requestOptions.headers = req.headers;
requestOptions.params = req.params;
requestOptions.query = req.query;
requestOptions.files = req.files;
requestOptions.method = req.method;
requestOptions.body = req.body;

return requestOptions;

}

I could create a PR on this if this is interesting? Would certainly help me out

Validator does not seem to validate against OpenAPI 3.0 schema

If I specify the "basePath" parameter (from v2) in my schema, it will validate correctly (but then my schema is not valid according to the OpenAPI 3.x specification). If I remove the "basePath" parameter from the schema, it will not validate. If I add the "servers" parameter with the url value according to OpenAPI 3.0, it seems to ignore it and will not validate.

e.g. (this works):

basePath: /api/v1/node-api-data

e.g. (this does not work):

servers:
- url: /api/v1/node-api-data

Request content-type is not validated against 'consumes' field

If Swagger specifies 'consumes' field for the service or the route, would be nice if express-ajv-swagger-validaiton library would use this data to check if data was sent of appropriate type.

Add types file to exported object

Float type is not supported

When using 'float' format in Swagger schema, the package fails to parse the Swagger file.

Generating the following error:

Encountered an error during start up: Error: unknown format \"float\" is used in schema at path \"#/properties/properties/currency_conversion/properties/fx_quote/properties/fx_rate\"\n at Object.generate_format [as code] (/home/app/node_modules/ajv/lib/dotjs/format.js:69:15)\n at Object.generate_validate [as validate] (/home/app/node_modules/ajv/lib/dotjs/validate.js:347:35)\n at Object.generate_properties [as code] (/home/app/node_modules/ajv/lib/dotjs/properties.js:204:26)\n at Object.generate_validate [as validate] (/home/app/node_modules/ajv/lib/dotjs/validate.js:347:35)\n at Object.generate_properties [as code] (/home/app/node_modules/ajv/lib/dotjs/properties.js:204:26)\n at Object.generate_validate [as validate] (/home/app/node_modules/ajv/lib/dotjs/validate.js:347:35)\n at Object.generate_properties [as code] (/home/app/node_modules/ajv/lib/dotjs/properties.js:204:26)\n at Object.generate_validate [as validate] (/home/app/node_modules/ajv/lib/dotjs/validate.js:347:35)\n at Object.generate_properties [as code] (/home/app/node_modules/ajv/lib/dotjs/properties.js:204:26)\n at generate_validate (/home/app/node_modules/ajv/lib/dotjs/validate.js:347:35)\n at localCompile (/home/app/node_modules/ajv/lib/compile/index.js:87:22)\n at Ajv.compile (/home/app/node_modules/ajv/lib/compile/index.js:56:13)\n at Ajv._compile (/home/app/node_modules/ajv/lib/ajv.js:358:27)\n at Ajv.compile (/home/app/node_modules/ajv/lib/ajv.js:118:37)\n at buildBodyValidation (/home/app/node_modules/express-ajv-swagger-validation/src/middleware.js:187:51)\n at /home/app/node_modules/express-ajv-swagger-validation/src/middleware.js:44:67\n at Array.forEach (<anonymous>)\n at /home/app/node_modules/express-ajv-swagger-validation/src/middleware.js:35:18\n at Array.forEach (<anonymous>)\n at /home/app/node_modules/express-ajv-swagger-validation/src/middleware.js:30:41\n at <anonymous>"

Handling different baseUrl when using firebase functions

When using firebase functions, the baseUrl is pre-fixed with the region+function name and this is dynamically added at the time of launching the cloud function.

Is there any way to handle without modifying the input spec baseUrl?

const path = req.baseUrl.concat(req.route.path);

The above code is not matching the path that was added in the spec. As we maintain different environments during development, its not possible to provide multiple schema json in init just to avoid this problem

Any solutions would be great!

Exclude irrelevant files from package and source control

Currently, npm publish publishes also test files, history files and more.

Using files prop in package.json file we can whitelist only the required files

Upgrade Dependency api-schema-builder to 2.0

api-schema-builder is now 2.0. Breaking changes are minimal.

This would help with a lot of the old packages in the dependency chain including, among other things, old core-js v2 module.

Primary benefit - from change-log:

Speed-up import of api-schema-builder and reduce total bundle size by removing polyfills

OpenAPI 3.0 support

An approach to investigating missing swagger features is a good question.
Suggested reading:

1. Supported data types
2. Validation for parameters
3. Security Defenitions

Those are probably the core features that related to input validation.

The approach in my opinion should be to start writing tests for each of the keywords and see the behavior, In case it is not as expected we can open an issue and fix it.

We also know that we miss support in multiple swagger file but that's a known issue that we shouldn't address now.

How to Validate multiple versions

Is there currently a way to have multiple validators with each assigned to a different OpenAPI file and subroute?

Add type safety

Since it's possible to generate typescript definitions from an OpenAPI specification (ie with https://www.npmjs.com/package/dtsgenerator), I wonder if it would be possible to implement type safety between the validator middleware and the following middlewares.

For example, dtsgenerator would transform https://github.com/PayU/openapi-validator-middleware/blob/master/test/pet-store-swagger.yaml to the following types declarations file:

// extract of the file...
declare namespace Paths {
    namespace CreatePets {
        export interface BodyParameters {
            body: Parameters.Body;
        }
        namespace Parameters {
            export interface Body {
                name: string;
                age?: number;
                tag?: string;
                test: {
                    field1: "enum1" | "enum2";
                };
            }
        }
        namespace Responses {
            export type Default = Definitions.Error;
        }
    }
// ...

Ideally, the validator would return a request with the expected type (something like Request<{}, {}, Paths.CreatePets.Parameters.Body>, see https://stackoverflow.com/a/55413670/7351594), so that the following middleware / route handler would know the type of its req argument:

app.post(
  '/pets',
  swaggerValidation.validate,
  (req: Request<{}, {}, Paths.CreatePets.Parameters.Body>, res, next) => {
    return res.json({ result: 'OK' });
  }
);

That way, the following route handler would not have to assume validation has already been done, thus reducing code coupling.

Create a synchronous version of init function

Currently the init function is async. I have a use case where I have the swagger file as JSON in memory. I would like to feed that in to the init function and make it synchronous.

swaggerValidator.initSync(myJsonSwaggerFile)
// other code follow.

I am finding the async version of init to be unnecessary and makes adding the validation lib to existing code base much harder.

Throws error when a definition refers to self

Given the schema below, swaggerValidator.init throws Cannot convert undefined or null to object

I believe the OAS 2.0 spec allows definitions to self-reference.

{
	"swagger": "2.0",
	"definitions": {
		"RecursiveObject": {
			"type": "object",
			"properties": {
				"StringProperty": {
					"type": "string"
				},
				"subItems": {
					"type": "array",
					"items": {
						"$ref": "#/definitions/RecursiveObject"
					}
				}
			}
		}
	}
}

Parameters with schema not used (OpenAPI 3.0)

In version 3.0, it looks like you put the schema for a parameter under the schema field. The code in buildParametersValidation doesn't seem to look there yet.

I can file a PR with a fix and test if this is a real issue. Thanks!

Clarification of OpenAPI 3 known issues

The discriminator support in the inheritance chain stops when getting to a child without a discriminator (a leaf in the inheritance tree), meaning a child without a discriminator cannot point to another child with a discriminator.

Is the implication of this that the provided discriminator-on-child example will not work under OpenAPI 3?

Init the middleware with the required openapi formats

Currently in order to support openapi formats which are part of the standard and are not supported by ajv, we required to pass them explicitly when calling init function. otherwise, we'll get an exception when a definitions file with these formats is passed to the middleware.

My proposal is to init the middleware with these formats by default.

Support for AWS Lambda HTTP event

Hello, I'm really excited to find this project.
I'm working on my Serverless framework project with OpenAPI features like documentations, API discoveries and etc. By the way I want to use this middleware as a request/response validator of AWS Lambda HTTP event but the project seems not supporting it for now. Do you have any plans to support AWS Lambda as a new framework like Express, Koa or Fastify? Maybe it would be helpful for you to consider making this middleware compatible with middy framework. Thanks.

Support passing error response formatting function

It would be useful to have possibility to customize responses generated in case validation fails.

Support routes with existing baseUrl

appending the req.baseUrl to the req.route.path should do the job

function validate(req, res, next) {
    let fullRoutePath = req.baseUrl ? req.baseUrl + req.route.path : req.route.path;
    return Promise.all([
        _validateParams(req.headers, req.params, req.query, fullRoutePath, req.method.toLowerCase()),
        _validateBody(req.body, fullRoutePath, req.method.toLowerCase())
    ]).then(function () {
        return next();
    }).catch(function (error) {
        return next(new InputValidationError(error));
    });
};

More flexible path resolution with regards to path parameters

Currently validator will fail to resolve correct schema if request route is resolved as /users/:userId and specification has route /users/:user_id.

Since it is not really important for router variable names to match the ones in specification, we should compare path just on the basis of having variable there, ignoring its name completely.

Missing function definition wheb used as Koa middleware

Currently validate function is defined as follows

export function validate(req: object, res: object, next: Function): void;

When using TypeScript and Koa, vscode displays an error as Koa middleware should be defined as follows

export function validate(ctx: Context, next: Function): void;

Expose an API or option so that it is possible to add custom keywords

It seems like this middleware didn't expose either an API or option that we are able to add custom keywords during the initialisation. Is it a good idea to add it on? See https://ajv.js.org/custom.html#define-keyword-with-validation-function for more info about custom keywords in ajv

Use express-ajv-swagger-validation in middleware

I am using nestjs and I use the express-ajv-swagger-validation in the middleware (couldn't figure out how to do it otherwise in nestjs and I don't need routes).

I got a crash when calling swaggerValidator.validate(req, res, next) which I fixed by manually adding the route object to the request object:

req.route = {
   path: req.path
};

Do you think its possible to check for the route property and if it does not exist use req.path? This would make express-ajv-swagger-validation a bit more flexible to use.

parseAjvError function throws un handle error

• when calling enum path of swagger (e.g: /v3/path/:some_enum) with unknown enum value, the _validateParams functions throw error that the parseAjvError function does not know how to handle:

from code:
line 135-137:
if (error.keyword === 'enum') {
error.message += ' [' + error.params.allowedValues.toString() + ']';
}

error.params.allowedValues is undefined and to toString function throws error.

use strict issue in node 4.5.0

Please add 'use strict'; at the beginning of the following file

express-ajv-swagger-validation/src/customKeywords/files.js

The Following error is thrown while requiring the module with node 4.5.0

SyntaxError: Block-scoped declarations (let, const, function, class) not yet supported outside strict mode

[BUG] - path build without basePath

error code 400 or 422

currently if the schema does not match the error thrown is a 500 series which is not the best.
500 is server based errors while a schema error is a 400 or 422.

Or is there a way using the errorFormatter that we can update the status code?

Passing explicit path to middleware instance

Currently middleware itself is responsible for determining path that is currently being accessed (let path = req.baseUrl.concat(req.route.path);). It would be helpful to be able to attach middleware to specific endpoints with predefined swagger path already filled during instantiation.

This is useful for Google Cloud Functions: while they seem to use Express under the hood and in general support swagger-style middleware, you can't attach middleware on app or router level and you don't have access to routing params, making usage of express-ajv-swagger-validation impossible.

Basically API could look like this:

app.post("/echo", validator.validate("post", "/echo"), (req, res, next) => {
  res.json({ output: req.body.input });
});

1.0.0

This we should do before 1.0.0:

• Change the middleware loading to be synchronous
• Check for missing swagger features. for example, the use of patterns (regex) and probably some types
• support OpenAPI 3.0 - not sure it's a must.

@kibertoad feel free to add your thoughts

[BUG] - OpenAPI 3.0 - fix mandatory validation for get body

Every GET request in openAPI 3 is being validated for its requestBody. therefore the validator fails when trying to validate the requestBody of a get request which is undefined.\

line failing:
const bodySchemaV3 = dereferenced.paths[currentPath][currentMethod].requestBody.content['application/json'].schema;

Internationalization

Hello, it is possible to set internationalized error messages like ajv-i18n combined with beautfy set to true?

I've tried to use it, but I have to choose between localized or beautified message:

const swaggerValidation = require('openapi-validator-middleware')
const localize = require('ajv-i18n')
(...)
swaggerValidation.init('swagger.yml', { beautifyErrors: true })
(...)
api.use((err, req, res, next) => {
  localize.fr(err.errors) 
  console.log(err.errors) //-> in english 
})

Renaming to express-openapi-validation

Hey, I think the name express-openapi-validation would be much more fitting from this project. Since the name change from OpenAPI to Swagger 3 and a bit years ago, the word Swagger is only referring to tools by SmartBear, and it causes a lot of confusion in the community to have folks keep calling everything Swagger when that is no longer the name.

Also, it's great that you're using AJV under the hood, but you probably want to protect that implementation detail from your end users allowing you to change the validator in the future.

I'm going to accept the PR for openapi.tools for now (apisyouwonthate/openapi.tools#87), but it would be awesome if you could update the name sometime. :)

Validation fails on file formData

v 0.3.1

When I try to run validation on endpoint that is defined like this:

/leads/import:
  post:
    description: Bulk import operation for leads from uploaded file
    produces:
      - application/json
    consumes:
      - multipart/form-data
    parameters:
      - name: sourceFile
        in: formData
        required: true
        type: file
        description: File to import from.
    responses:
      '200':
        description: Import result

I get this error:

TypeError: Cannot read property 'required' of undefined\n at parameters.forEach.parameter (/Users/igorsavin/sources/lms-api/api/node_modules/express-ajv-swagger-validation/src/middleware.js:244:48)\n at Array.forEach (<anonymous>)\n at buildParametersValidation (/Users/igorsavin/sources/lms-api/api/node_modules/express-ajv-swagger-validation/src/middleware.js:230:16)\n at /Users/igorsavin/sources/lms-api/api/node_modules/express-ajv-swagger-validation/src/middleware.js:41:73\n at Array.forEach (<anonymous>)\n at /Users/igorsavin/sources/lms-api/api/node_modules/express-ajv-swagger-validation/src/middleware.js:28:18\n at Array.forEach (<anonymous>)\n at /Users/igorsavin/sources/lms-api/api/node_modules/express-ajv-swagger-validation/src/middleware.js:23:41\n

Any ideas what could be wrong?.. Swagger Editor confirms that I have a valid Swagger.

Beautify Errors, does not beautify

I am struggling to beautify my errors without editing the code directly. I was able to accomplish my goal by changing line 51 in middleware.js from:
return Promise.resolve(error);
to:
return Promise.resolve(error.errors);
However, I should not have to do this manually to receive the beautified error. Is there any way to receive the error other than modifying the code?

Option to enable type coercion

Ajv supports type coercion for automatic conversion e. g. from String to Number:

var ajv = new Ajv({ coerceTypes: true });

It would be extremely helpful if we would be able to enable this option from express-ajv-swagger-validation as well.

Throws error when a schema contains reference to external file

since opening schema allows added references to external files over HTTP with $ref the validator should allow loading schemes that use this feature.

currently the init function throws "TypeError: Cannot convert undefined or null to object" if the schema contains external references.

this can be solved by adding async init function that use "apiSchemaBuilder.buildSchema"
instead of "apiSchemaBuilder.buildSchemaSync"

Support for remote references

Does the middleware support remote references, as allowed by the official spec: https://swagger.io/docs/specification/using-ref/

Remote Reference – $ref: 'document.json' Uses the whole document located on the same server and in the same location.
The element of the document located in the same folder – $ref: 'document.json#/myElement'
The element of the document located in the parent folder – $ref: '../document.json#/myElement'
The element of the document located in another folder – $ref: '../another-folder/document.json#

What is the strategy for remote references?

Thanks!