Enforce validation of model input datatypes

[gss-cogs]

Enforce validation of model input datatypes

[robons]

It's a bit nasty but I've created a model which allows us to defer the validation until we want to call it:

from pydantic import BaseModel, validate_model, ValidationError


class SomeModel(BaseModel):
    name: str

    class Config:
        validate_assignment = False
        extra = "forbid"

    def __init__(self, name: str):
        values, fields_set, validation_error = validate_model(self.__class__, {})
        object.__setattr__(self, "__dict__", values)
        object.__setattr__(self, "__fields_set__", fields_set)
        self._init_private_attributes()

        self.name = name

    def validate(self):
        values, fields_set, validation_error = validate_model(self.__class__, self.dict())
        if validation_error:
            raise validation_error


model = SomeModel(None)
model.validate()

I do not trust this code if we upgrade pydantic from the current version.

[robons]

The following is a more robust approach using python's dataclasses combined with pydantic's version of them.

Upon validation, I use the python dataclasses functionality to convert the model to a dictionary. I pass these values into the same model wrapped with pydantic's dataclass annotation. I try to catch any validation error exceptions which are then passed back to the caller. If validation is successful, I copy all of the attributes from the pydantic class back to the model - pydantic coerces values to match the annotated datatype where possible.

The SomeModel dataclass defined below is what I've used to test validation:

import dataclasses

from pydantic import BaseConfig
import pydantic
import pydantic.dataclasses
from dataclasses import dataclass, asdict
from typing import ClassVar, Dict, Type, List
from abc import ABC


from csvqb.models.validationerror import ValidationError


@dataclass
class ValidatedModel(ABC):
    """
        ValidatedModel - an abstract base class to be inherited by models which want a `validate` method which verifies
        that the model's attributes agree with the corresponding type annotations.

        Uses pydantic under the hood, but rather than using pydantic's constructor validation approach, we delay
        validation until the `validate` method is called.
    """

    _map_class_to_pydantic_constructor: ClassVar[Dict[Type, Type]] = dict()
    """_map_class_to_pydantic_constructor - Cache of pydantic constructor corresponding to a given class."""

    class Config(BaseConfig):
        """pydantic Configuration - see https://pydantic-docs.helpmanual.io/usage/model_config/"""
        extra = "forbid"

    @classmethod
    def _get_pydantic_constructor(cls) -> Type:
        if cls not in ValidatedModel._map_class_to_pydantic_constructor:
            ValidatedModel._map_class_to_pydantic_constructor[cls] = pydantic.dataclasses.dataclass(cls)
        return ValidatedModel._map_class_to_pydantic_constructor[cls]

    def as_dict(self) -> dict:
        """Use python dataclasses method to return this model as a dictionary."""
        return asdict(self)

    def validate(self) -> List[ValidationError]:
        """
            Validate this model using pydantic.

            Checks that all model attributes match the expected annotated data type. **Coerces values** where possible.
        """

        pydantic_class_constructor = self.__class__._get_pydantic_constructor()
        try:
            validated_model = pydantic_class_constructor(**self.as_dict())
        except pydantic.ValidationError as error:
            return [ValidationError(f"{e['loc']} - {e['msg']}") for e in error.errors()]

        #  Update this model's values with pydantic's coerced values
        for field in dataclasses.fields(self):
            setattr(self, field.name, getattr(validated_model, field.name))

        # there are no validation errors
        return []


@dataclass
class SomeModel(ValidatedModel):
    name: str
    age: int


model = SomeModel(name="Jim", age="20")
print([str(e) for e in model.validate()])
print(model)

in this example, the age is coerced into an int yielding the output:

[]
SomeModel(name='Jim', age=20)

Example with errors:

model = SomeModel(name=None, age=20)
print([str(e) for e in model.validate()])
print(model)

["ValidationError(('name',) - none is not an allowed value)"]
SomeModel(name=None, age=20)