waleedashraf/asyncapi-validator

Message validator for Kafka/RabbitMQ/Anything through AsyncAPI schema

asyncapi-validator

Message validator through AsyncAPI schema

_Note: This package only support AsyncAPI Schema v2.0.0 and above. Since v3.0.0, support for older versions of AsyncAPI Schema has been removed.

npm i asyncapi-validator

Features

• Validate your messages against your AsyncApi Document
• Validate your AsyncApi Document against AsyncApi Schema definition
• Load your AsyncApi Schema from local file or any URL
• Supports AsyncApi in JSON and YAML format
• Supports AsyncAPI v2.0.0 and above

Content

• Class Methods
  • AsyncApiValidator.fromSource()
    • Options
• Instance Methods / Properties
  • .validateByMessageId()
  • .validate()
  • .schema
• Example usage with .validateByMessageId() method
• Example usage with .validate() method
• Errors
  • Error Example

Class Methods

AsyncApiValidator.fromSource()

/** 
 * Load and Parse the schema from source.
 * @param {string | Object} source - local PATH or URL of schema or schema Object
 * @param {Object} options - options for validation
 * @returns {Promise}
 */
AsyncApiValidator.fromSource(source, options)

Options

value	type		description
ignoreArray	boolean	optional	If true, then if schema is defined as an array and payload is an object, then payload will be placed inside an array before validation.
msgIdentifier	string	optional (required only if you use .validate() method)	Name of the parameter whose value will be used as "key" in .validate() method. Recommendation is to use "name" as described in message-object. You can also use Specification Extensions.
path	string	optional	Path to the AsyncAPI document. It will be used to resolve relative references. Defaults to current working dir. As used in asyncapi-parser

Instance Methods / Properties

.validateByMessageId()

Here messageId should be as defined in AsyncAPI Schema v2.4.0. To use this method, your AsyncAPI Schema version should be >= v2.4.0.

/**
 * Method to validate the Payload against schema definition.
 * @param {string} key - required - messageId
 * @param {Object} payload - required - payload of the message
 * @returns {boolean}
 */
.validateByMessageId(key, payload)

.validate()

To use this method for validation, you should provide msgIdentifier in AsyncApiValidator options.

/**
 * Method to validate the Payload against schema definition.
 * @param {string} key - required - message key
 * @param {Object} payload - required - payload of the message
 * @param {string} channel - required - name of the channel/topic
 * @param {string} operation - required - publish | subscribe
 * @returns {boolean}
 */
.validate(key, payload, channel, operation)

.schema

.schema property can be used to access AsyncAPI schema in JSON format and with all the refs resolved.

Example usage with .validateByMessageId() method

Schema

asyncapi: 2.4.0

info:
  title: User Events
  version: 1.0.0

channels:
  user-events:
    description: user related events
    publish:
      message:
        messageId: UserRemoved
        payload:
          type: object
          properties:
            userEmail:
              type: string
            userId:
              type: string

const AsyncApiValidator = require('asyncapi-validator')
let va = await AsyncApiValidator.fromSource('./api.yaml')

// validate messageId 'UserRemoved'
va.validateByMessageId('UserRemoved', {
  userId: '123456789',
  userEmail: '[email protected]',
})

Example usage with .validate() method

Schema

asyncapi: 2.0.0

info:
  title: User Events
  version: 1.0.0

channels:
  user-events:
    description: user related events
    publish:
      message:
        name: UserDeletedMessage
        x-custom-key: UserDeleted
        payload:
          type: object
          properties:
            userEmail:
              type: string
            userId:
              type: string

const AsyncApiValidator = require('asyncapi-validator')
let va = await AsyncApiValidator.fromSource('./api.yaml', {msgIdentifier: 'x-custom-key'})

// validate 'UserDeleted' on channel 'user-events' with operation 'publish'
va.validate('UserDeleted', {
  userId: '123456789',
  userEmail: '[email protected]',
}, 'user-events', 'publish')

In above example, "msgIdentifier" is "x-custom-key". That is why, "UserDeleted" is used as "key" in "va.validate()" method.

Errors

Error thrown from asyncapi-validator will have these properties.

key	type	value	description
name	string	AsyncAPIValidationError	AsyncAPIValidationError
key	string		"key" of payload against which schema is validated
message	string		errorsText from AJV
errors	array		Array of errors from AJV

Error Example

{
  AsyncAPIValidationError: data.type must be equal to one of the allowed values at MessageValidator.validate (.....
  name: 'AsyncAPIValidationError',
  key: 'hello',
  errors:
    [
      { keyword: 'enum',
        dataPath: '.type',
        schemaPath: '#/properties/type/enum',
        params: [Object],
        message: 'must be equal to one of the allowed values'
      }
    ]
}

asyncapi-validator's Issues

[QA] how to get property from Error object?

When I validate my schema I get something like this:

{
  ...
  errors: [
    {
      keyword: 'required',
      dataPath: '',
      schemaPath: '#/required',
      params: [Object],
      message: "should have required property 'maxPlayers'"
    }
  ]
}

It would be vey useful, if I get the property or better path of the property that has this error. So I know where in my HTML form I should show it. In my case maxPlayers. I am reading the https://ajv.js.org/api.html docs and there is instancePath for that? No? But it's not present in this solution.

How to use validate() with channel with parameter?

I have a channel /a/{id}/c.

How do I use your validator with such a channel?

Does not support version 3.0.0

As per the README, it's supported, but when passing a valid yaml through the fromSource method, you would get this error in return:
Version 3.0.0 is not supported. Error Details: Please use latest version of the specification.

Feature: Use ajv-formats to validate data formats

While validating the event payload we use the ajv option unknownFormats at here Doing so, ignores the data format validation. So instead of this can we use ajv-formats to validate these formats? This package provides rules to validate all the common standard data formats like int32, int64, date, and date-time and is maintained by ajv community.

const Ajv = require("ajv")
const addFormats = require("ajv-formats")

const ajv = new Ajv()
addFormats(ajv)

What's the preferred way to handle invalid messages?

It looks like the function MessageValidator.validate throwsValidationError once it finds that a message is invalid. So how do I catch and handle the error properly? From what I see the type is not exported, and so what I'm currently doing is:

try {
  va.validate('UserMessage', messageObj, 'user-events', 'publish')
} catch (ex) {
  if (ex.name === 'AsyncAPIValidationError') {
    // handle invalid message
  } else {
    throw ex
  }
}

Which is not ideal because the Typescript transpiler doesn't recognize the type of the error.

custom validation: how to call validate()

I'd like to have a custom validation for my asyncapi v2.0 schema.

I have a message which is a metadata message and defines the structure of another message, e.g. it uses the following parts in "schema" to define the allowed types:

    dpDataTypeV1:
      type: string
      oneOf:
        - const: 'Bool'
          x-dpDataType:
            $ref: "#/components/schemas/dpDataType_Bool"
        - const: 'Byte'
          x-dpDataType:
            $ref: "#/components/schemas/dpDataType_Byte"
 ...

    dpDataType_Bool:
      oneOf:
        - type: boolean
        - type: integer
          minimum: 0
          maximum: 1
        - type: string
          oneOf:
            - const: 'true'
            - const: 'false'
            - const: '0'
            - const: '1'

    dpDataType_Byte:
      oneOf:
        - type: integer
          minimum: 0
          maximum: 255
        - type: string
          regex: '^[1-9][0-9]{0,2}$'

The schema element of the value is defined like this:

      anyOf:
        - type: integer
        - type: number
        - type: string
        - type: boolean
        - type: object
        - type: array

Unfortunately the schema cannot be more specific, because the real type is dynamic, that's why I need a custom validator.

The other message with the values should then be validated, e.g. if the values really matches the definition of the metadata (e.g. the type and range).

I managed to find the correct schema element using this code:

    let dt = dpMeta.dataType;
    let schemaDt = validator.schema.components.schemas.dpDataTypeV1.oneOf.find(element => element.const == dt);

Unfortunately I didn't find a way how to call validate, e.g. with the value true and the schema element dpDataType_Bool.

Can you help me?

Question: How to validate messages with Headers

i have defined a message with message traits, how to validate such messages if header is present and correct?

heartbeat:
  name: heartbeat
  title: Heartbeat signal of service will be sent every 5m (default)
  summary: Inform about current device status.
  traits:
    - $ref: "#/components/messageTraits/commonHeaders"
  payload:
    $ref: "#/components/schemas/heartbeat"

messageTraits:
  commonHeaders:
    headers:
      type: object
      additionalProperties: false
      properties:
        messageId:
          type: string
          format: uuid
        correlationId:
          type: string
          format: uuid
        responseTopicId:
          type: string
          format: uuid
      required:
        - messageId
        - correlationId

heartbeat:
  type: object
  additionalProperties: false
  properties:
    healthy:
      type: boolean
      description: healthy state
    sentAt:
      type: string
      format: date-time
      description: Date and time when the message was sent.
  required:
    - healthy
    - sentAt

Invalid message gets marked valid?

Hi. Either I'm missing something, or the package doesn't actually.. Validate the message?
Relevant code:

let validator: AsyncApiValidator.Validator;
const loadValidator = async () => {
  // Load the AsyncAPI validator. In a function as it's async
  validator = await AsyncApiValidator.fromSource(
    // Testing yaml:
    path.join(__dirname, "../spec/schemas/testschema.yaml")
  );
};
loadValidator();

const isValidMessage = validator.validateByMessageId(
    "UserRemoved",
    decodedMessage
  );
  console.log("Message is valid:", isValidMessage);

the testschema.yaml is the yaml from the validatebymessageid example in the readme:

However, when I run the above with a decodedMessage of: { timeout: 12 } I get output: Message is valid: true

While none of the properties of the messages overlap?! It should obviously not be valid? Am I missing something here?

Can find msgIdentifier for fallowing schema

{
    "asyncapi": "2.3.0",
    "info": {
        "title": "XXX game API",
        "version": "0.0.1",
        "license": {
            "name": "XXX"
        }
    },
    "servers": {
        "mosquitto": {
            "url": "https://[enviroment_backend_url]",
            "protocol": "https"
        }
    },
    "components": {
        "schemas": {
            "RoomType": {
                "$id": "RoomType",
                "enum": [
                    "private",
                    "public"
                ],
                "type": "string",
                "description": "Room type"
            }
        }
    },
    "channels": {
        "/": {
            "subscribe": {
                "summary": "Socket backend API",
                "message": {
                    "oneOf": [
                        {
                            "name": "Created a room",
                            "payload": {
                                "$schema": "http://json-schema.org/draft-07/schema#",
                                "type": "object",
                                "$id": "CreateRoomReponse",
                                "additionalProperties": false,
                                "properties": {
                                    "type": {
                                        "$ref": "#/components/schemas/RoomType"
                                    },
                                    "secret": {
                                        "type": "string",
                                        "description": "Password for private room",
                                        "example": "qwerty"
                                    },
                                    "maxPlayers": {
                                        "type": "integer",
                                        "description": "Maximum players who can join",
                                        "example": 6
                                    }
                                }
                            }
                        }
                    ]
                }
            },
            "publish": {
                "summary": "Socket backend API",
                "message": {
                    "oneOf": [
                        {
                            "name": "Create a room",
                            "x-CreateRoomPayload-key": "CreateRoomPayload",
                            "payload": {
                                "$schema": "http://json-schema.org/draft-07/schema#",
                                "type": "object",
                                "$id": "CreateRoomPayload",
                                "additionalProperties": false,
                                "required": [
                                    "type",
                                    "maxPlayers"
                                ],
                                "properties": {
                                    "type": {
                                        "$ref": "#/components/schemas/RoomType"
                                    },
                                    "secret": {
                                        "type": "string",
                                        "description": "Password for the room (ignored if public)",
                                        "example": "qwerty",
                                        "minLength": 6,
                                        "maxLength": 6
                                    },
                                    "maxPlayers": {
                                        "type": "integer",
                                        "minimum": 2,
                                        "maximum": 12,
                                        "description": "Maximum players who can join",
                                        "example": 6
                                    }
                                },
                                "if": {
                                    "properties": {
                                        "type": {
                                            "const": "private"
                                        }
                                    }
                                },
                                "then": {
                                    "required": [
                                        "secret"
                                    ]
                                }
                            }
                        }
                    ]
                }
            }
        }
    }
}

"msgIdentifier "x-CreateRoomPayload-key" does not exist"

unknown format "int32" is used in schema

Hey, I'm getting the following error:
Error: unknown format "int32" is used in schema

Id:
   type: integer
   format: int32

According to AsyncApi 2.0 specification "format: int32" should be valid.

Parser expects there to be a components/messages property in the main schema

The message validator loads the schema messages as follows:
this._messages = this._schema.components.messages

However, the schema messages could be in another file, such as through a reference as follows:

channels:
  test:
    publish:
      message:
        oneOf:
          - $ref: './ref1.yml#components/messages/msg1'
          - $ref: './ref2.yml#components/messages/msg2'

Additionally, the logic assumes that all messages are in components/messages, which isn't always the case - the spec will allow messages to be placed directly under channels.

Could the parser be updated to actually parse the schema, and then walk down the tree accordingly?

Question - CLI wrapper

Hi @WaleedAshraf

Currently the "asyncapi-validator" seems to build like a JS module, or I must have missed in the documentation.

Would there be interest or plans to also provide a CLI wrapper to validate the asyncapi documents via CLI commands?

JSON references to external files are not resolved correctly

When passing a local file name to fromSource that refers to a different directory (e.g. ../schemas/asyncapi.json) and that file references other files via $ref, those won't be found by the parser.
The cause of this is that Parser.parse first reads the file into a string and then invokes the asyncapiParser on that string, so those files will be searched for in the current directory and not in the directory where the AsyncAPI file is located.
Maybe you want to change the current directory to the file's directory before invoking the parser and change it back afterwards.

Example file with reference:

{
    "asyncapi": "2.1.0",
    "info": {
        "title": "test",
        "version": "v1.0"
    },
    "channels": {
        "channel": {
            "subscribe": {
                "message": {
                    "contentType": "application/json",
                    "payload": {
                        "$ref": "schema.json"
                    }
                }
            }
        }
    }
}

Bug: Validation fails with exclusiveMinimum / exclusiveMaximum in schema

Version 3.1.1

fails with exclusiveMaximum active:

        myProperty:
          type: number
          minimum: 0.0
          maximum: 360.0
    # TODO     exclusiveMaximum: true
          example: 10.5

Feature: Support for operationId

Hello,

It would be great to add support for validating messages using OperationId in addition to the regular key, channel and operation way. The advantage would be that an operationId is usually immutable after designed while the rest of the parameters could change. For example, sometimes we change the key or the channel name, but (at least the way I design), the operation ID remains the same. This way of doing things is quite useful when using a generated client (such as in OpenAPI), for which the exact channel name would be an implementation detail, with an abstraction layer based on the tag and the operationId.

The operationId field is defined in the AsyncAPI specification as an id that "[...] MUST be unique among all operations described in the API." (https://www.asyncapi.com/docs/specifications/2.0.0#fixed-fields-7), so it would be sufficient to search the YAML for the specified ID in order to run the validation.

Proposed implementation:

/**
 * Method to validate the Payload against schema definition using an OperationId.
 * @param {string} operationId - required - operationId
 * @param {Object} payload - required - payload of the message
 * @returns {boolean} 
 */
validateByOperationId(operationId, payload);

I'll be glad to submit a PR if the idea is acceptable.

Thanks!

How to determine message from topic, with variables?

Hi!
Let's say I have a topic ${deviceid}/status (amongst other topics).
I have a worker, processing messages from all topics (with a wildcard subscribe).
Is there a way to have the topic "automatically" match, so I can validate the payload?
The examples all have a hardcoded topic, and I see no examples with variables (like deviceId above) in the examples.

I could make matching myself - (with regexes?) but I feel this is quite a common usecase, and something the library could support. Or am I missing something? Eg openApiValidator does this. Thanks!

Package asyncapi is deprecated

Does not support avro in AsynApi

Validation fails with the following error when the AsyncApi has an avro schema:

/Users/ikanel/code/SwaggerHub/node_modules/asyncapi-validator/src/Parser.js:27
throw new ValidationError(this._formatError(err), undefined, err.validationErrors)
^

AsyncAPIValidationError: PARSERS[String(...)] is not a function Error Details:
at Parser.parse (/Users/ikanel/code/SwaggerHub/node_modules/asyncapi-validator/src/Parser.js:27:13)
at async ValidatorFactory.fromSource (/Users/ikanel/code/SwaggerHub/node_modules/asyncapi-validator/src/ValidatorFactory.js:13:29)
at async file:///Users/ikanel/code/SwaggerHub/Contracts/AsyncApi/Validator/validate.js:6:10 {
key: undefined,
errors: undefined
}

Node.js v19.2.0

The fragment of the asynApi schema which causes this exception:

channels:
avroExample:
publish:
message:
#see https://github.com/asyncapi/avro-schema-parser
name: avro-message
schemaFormat: 'application/vnd.apache.avro;version=1.9.0'
payload: # The following is an Avro schema in YAML format (JSON format is also supported)
type: record
name: User
namespace: com.company
doc: User information
fields:
- name: displayName
type: string
- name: email
type: string
- name: age
type: int

Expose TypeScript types

Hi, thanks for this great tool! To use this with TypeScript I'm currently using a custom declaration file (see below). Are you planning on exposing one with this package?

It might just require adding types: index.d.ts to the package.json and creating an index.d.ts like:

declare module 'asyncapi-validator' {
  type Validator = {
    validate: (
      key: string,
      payload: unknown,
      channel: string,
      operation: 'publish' | 'subscribe',
    ) => void;
  };

  const fromSource: (
    path: string | Record<string, unknown>,
    options: { msgIdentifier: string; ignoreArray?: boolean },
  ) => Promise<Validator>;
}

Question: Circular References in the scheme

Hi,

I do have a question, as I couldn't find in the docs, if circular references are supported by the asyncapi-validator ?
Currently, for a definition that has circular references the validation fails with:

/node_modules/json-schema-traverse/index.js:67
function _traverse(opts, pre, post, schema, jsonPtr, rootSchema, parentJsonPtr, parentKeyword, parentSchema, keyIndex) {
                  ^

RangeError: Maximum call stack size exceeded
    at _traverse (/node_modules/json-schema-traverse/index.js:67:19)

Thank you,