mena-devs/objectron

Compare an object to a generic model to test equality and extract matches

objectron's Issues

Support closures as testing rules

Objectron currently supports primitive types, regular expressions, nested objects and arrays as testing rules. This ticket is to add support for functions and closures as testing rules.

Values stored in groups don't retain the original types

I was using the regex groups features and noticed that for fields with non-string type values e.g. numbers, boolean etc..., end up being cast into string when they are added into the groups object in the result. The matches object in the result seems to be normal.

Here is an example test to illustrate

  test('Maintain types in groups', () => {
    const payload = {
      response: {
        status: 200,
        _transferSize: 2863
      },
    }

    const result = match(payload, {
      response: {
        status: /^(?<respStatus>[0-9]{3})/,
        _transferSize: /(?<respSize>\d+)/,
      }
    })

    const expected = {
      match: true,
      total: 2,
      matches: {
        response: {
          status: 200,
          _transferSize: 2863
        }
      },
      groups: {
        respStatus: 200,
        respSize: 2863
      }
    }

    assert.isTrue(result.match)
    assert.deepEqual(result, expected)
  })

Test result

      AssertionError: expected { Object (match, total, ...) } to deeply equal { Object (match, total, ...) }
      + expected - actual

       {
         "groups": {
      -    "respSize": "2863"
      -    "respStatus": "200"
      +    "respSize": 2863
      +    "respStatus": 200
         }
         "match": true
         "matches": {
           "response": {

We might want to look into this issue, especially if the user is actually writing a regex group in the pattern to be matching by specific type.

Array of object patterns not matching properly against payload if indexes are different

Based on some usage, I've noticed that objects in an array don't seem to be properly matching with the array of objects in the payload. There is a corner case that can happen when the match object in the payload is not at the same index as that of the pattern. This would lead to wrong evaluation of values or missing an evaluation altogether.

Example test

 test('Match array objects further in the array', () => {
    const payload = {
      response: {
        headers: [
          {
            "name": "accept-ranges",
            "value": "bytes"
          },
          {
            "name": "vary",
            "value": "Accept-Encoding"
          },
          {
            "name": "cache-control",
            "value": "public, max-age=31536000, stale-while-revalidate=2592000"
          },
          {
            "name": "content-encoding",
            "value": "gzip"
          },
        ]
      }
    };

    const result = match(payload, {
      response: {
        headers: [
          { 
            name: 'cache-control', 
            value: /(?<responseHeaderCacheControl>.*)/ 
          },
        ]
      }
    });

    const expected = {
      match: true,
      total: 1,
      matches: {
        response: {
          headers:[
            {
              'name': 'cache-control',
              'value': 'public, max-age=31536000, stale-while-revalidate=2592000'
            },
          ]
        }
      },
      groups: {
        responseHeaderCacheControl: 'public, max-age=31536000, stale-while-revalidate=2592000'
      }
    }

    assert.deepEqual(result, expected);
  });

Test output

       {
         "groups": {
      -    "responseHeaderCacheControl": "bytes"
      +    "responseHeaderCacheControl": "public, max-age=31536000, stale-while-revalidate=2592000"
         }
      -  "match": false
      +  "match": true
         "matches": {
           "response": {
             "headers": [
               {
      -          "value": "bytes"
      +          "name": "cache-control"
      +          "value": "public, max-age=31536000, stale-while-revalidate=2592000"
               }
             ]
           }
         }

Analysis

Looking at the objectron code, I've noticed this is happening because there is an assumption that the matched object in payload should be of the same index and order as that of the one in the pattern.

Taking a look at line 47 in the source code, we're looping into the array based on the pattern's array of objects size, then on line 65 sending the index of the current value (pattern) to test against the payload which might actually be at a totally different index.

Ability to match multiple patterns in 1 call

Problem:

There are scenarios that require matching an item in an array but also the following item (or items at any other index). Example:

[
  {
    type: 'code',
    raw: '```\n{ issues: [1,2] }\n```\n',
    lang: '',
    text: '{ issues: [1,2] }'
  },
  { type: 'hr', raw: '---\n\n' },
  {
    type: 'heading',
    raw: '## Action Items\n',
    depth: 2,
    text: 'Action Items',
    tokens: [ [Object] ]
  },
  {
    type: 'heading',
    raw: '#### Test Issue #1 - #1 - 03 October 2020\n',
    depth: 4,
    text: 'Test Issue #1 - #1 - 03 October 2020',
    tokens: [ [Object] ]
  },
  {
    type: 'list',
    raw: '- [ ] Action item #1 for @Link- \n' +
      '- [ ] @Link- has to do action items #2\n' +
      '\n',
    ordered: false,
    start: '',
    loose: false,
    items: [
      [Object], [Object],
      [Object], [Object],
      [Object], [Object],
      [Object], [Object],
      [Object]
    ]
  }
]

For the payload I would like to match the following objects:

  {
    type: 'heading',
    raw: '#### Test Issue #1 - #1 - 03 October 2020\n',
    depth: 4,
    text: 'Test Issue #1 - #1 - 03 October 2020',
    tokens: [ [Object] ]
  },
  {
    type: 'list',
    raw: '- [ ] Action item #1 for @Link- \n' +
      '- [ ] @Link- has to do action items #2\n' +
      '\n',
    ordered: false,
    start: '',
    loose: false,
    items: [
      [Object], [Object],
      [Object], [Object],
      [Object], [Object],
      [Object], [Object],
      [Object]
    ]
  }

However, the match() function accepts only 1 pattern. If arguments are passed as follows:

match(
      block,
      {
        type: 'heading',
        depth: 4,
        text: /.*(?<issueNumber>#[0-9]*) - (?<date>.*)/
      },
      {
        type: 'list',
        ordered: false,
        loose: false
      }
    )

The method will assume the second pattern object as the callback.

Proposed Solution

Allow match() to accept an array of patterns while providing a new optional flag that will inform the method that it should handle an array of patterns not lookup the array structure in the payload.

Function breaks when trying to evaluate regex patterns against number values via match

Payload

{
   response: {
      status: 200
  }
}

Pattern

{
  response: {
    status: /\d*/
}

Instead of getting the expected match result, the function breaks returning the error TypeError: payload[key].match is not a function. This is because match is being called via the number value while it doesn't exist since number isn't an object nor has it within its prototype.

An alternative to fix this would be to call exec on the regex pattern itself. e.g. value.exec(payload[key]) (when value is a RegExp) instead of payload[key].match(value)

Provide regex pattern shortcuts to help users define cleaner models

Defining larger models with several regex patterns with lots of repetition can become a maintenance hell for users. Especially that many patterns are usually common ones such as URLs, numbers only, negative numbers etc...

Was thinking, it would be cool to have a pre-defined set of patterns in the lib to allow users to do shortcuts when defining models. Example:

Instead of having to define something like this as a model:

  const baseEntryPattern = {
    pageref: /(?<pageRef>.*)/,
    startedDateTime: /(?<startedDateTime>.*)/,
    request: {
      method: /(?<requestMethod>GET|POST)/,
      url: /(?<requestUrl>[-a-zA-Z0-9@:%._\+~#=]{1,256}\.[a-zA-Z0-9()]{1,6}\b([-a-zA-Z0-9()@:%_\+.~#?&//=]*))/,
      httpVersion: /(?<requestHttpVersion>.*)/,
      headersSize: /^(?<requestHeaderSize>\-?(\d+\.?\d*|\d*\.?\d+))$/,
      bodySize: /^(?<requestHeaderSize>\-?(\d+\.?\d*|\d*\.?\d+))$/,
    },
    response: {
      status: /^(?<responseStatus>[0-9]{3})/,
      content: {
        size: /^(?<responseContentSize>\-?(\d+\.?\d*|\d*\.?\d+))$/,
      },
      headers: [
        { name: /content-type/i, value: /(?<responseContentType>.*)/ },
        { name: /content-length/i, value: /(?<responseContentLength>.*)/ },
        { name: /cache-control/i, value: /(?<responseCacheControl>.*)/ },
      ]
    },
    timings: (val) => val,
    time: /^(?<time>\-?(\d+\.?\d*|\d*\.?\d+))$/
  };

Would be cleaner to (optionally) use a helper method in the module to define common patterns this way:

  const baseEntryPattern = {
    pageref: Objectron.patterns.any(),
    startedDateTime: Objectron.patterns.any('startDateTime'),
    request: {
      method: /(?<requestMethod>GET|POST)/,
      url: Objectron.patterns.url('requestUrl'),
      httpVersion: Objectron.patterns.any('httpVersionName'),
    },
    response: {
      status: /^(?<responseStatus>[0-9]{3})/,
      content: {
        size: Objectron.patterns.anyNumber('responseContentSize')
      },
    },
    timings: (val) => val,
    time: /^(?<time>\-?(\d+\.?\d*|\d*\.?\d+))$/
  };

Can save lots of effort in maintaining and re-implementing common patterns by the user especially that as the model gets more complex, repeated patterns become confusing and error prone.This could potentially be achieved via implementing a set of methods to generate regex expressions based on pre-defined patterns.

What do you think?