Expand Tolerance options

[chaostoolkit]

Expand Tolerance options

[Lawouach]

Hey,

Indeed, the basic tolerance with a sequence like this one cannot handle the returned body from a HTTP probe. The reason is that such probe returns an object (a mapping) such as:

{
    "status": 200,
    "headers": {...},
    "body": "..."
}

Here, the default tolerance range will look at the status, not the actual body.

To support this, we actually need a more complex new tolerance type, following the principles of the jsonpath and regex tolerances.

"tolerance": {
    "type": "range",
    "target": "body",
    "range": {
        "min": 10,
        "max": 1000
    }
}

When the body is a string, this can be reduced to:

"tolerance": {
    "type": "range",
    "target": "body",
    "range": ["good", "better", "best"]
}

assuming:

{
    "status": 200,
    "headers": {...},
    "body": "bad"
}

Please have a look #84 and see if it helps?

[Lawouach]

While discussing with @GerardMoscardo, I wondered if it wasn't confusing to have "range": ["good", "better", "best"]? Indeed, this could be achieved with a simple regex already. Maybe the range, should really stick to numerical values between low and high?

[GerardMoscardo]

Hello, IMO, as u said, range should be only used for numerical values.

[Lawouach]

I agree.

In that case, should we use a shorter syntax:

"tolerance": {
    "type": "range",
    "target": "body",
    "range": [-10, 10]
}

Basically, low/high and the value must between those boundaries.

[GerardMoscardo]

This looks perfect, also shares syntax with the other probes already existing:

A sequence tolerance with lower and upper bounds:

"tolerance": [4, 9]

Thanks!

[Lawouach]

Exactly!

I'll update the PR asap.

[Lawouach]

Documented in chaostoolkit/chaostoolkit-documentation@c8a58f7