maximkulkin/lollipop

Python data serialization/validation library

lollipop

Data serialization and validation library

Features

• flexible schema definition API with powerful type combinators
• data validation
• serialization/deserialization
• in-place deserialization

Example

from lollipop.types import Object, String, Date
from lollipop.validators import Length
from collections import namedtuple
from datetime import date

Person = namedtuple('Person', ['name'])
Book = namedtuple('Book', ['title', 'publish_date', 'author'])

PersonType = Object({
    'name': String(validate=Length(min=1)),
}, constructor=Person)

BookType = Object({
    'title': String(),
    'publish_date': Date(),
    'author': PersonType,
}, constructor=Book)

harryPotter1 = Book(
    title='Harry Potter and the Philosopher\'s Stone',
    publish_date=date(1997, 6, 26),
    author=Person(name='J. K. Rowling')
)

# Dumping
BookType.dump(harryPotter1)
# => {'title': 'Harry Potter and the Philosopher\'s Stone',
#     'publish_date': '1997-06-26',
#     'author': {'name': 'J. K. Rowling'}}

# Loading
BookType.load({'title': 'Harry Potter and the Philosopher\'s Stone',
               'publish_date': '1997-06-26',
               'author': {'name': 'J. K. Rowling'}})
# => Book(title='Harry Potter and the Philosopher\'s Stone',
#         publish_date=date(1997, 06, 26),
#         author=User(name='J. K. Rowling'))

# Partial inplace loading
BookType.load_into(harryPotter1, {'publish_date': '1997-06-27'})
# => Book(title='Harry Potter and the Philosopher\'s Stone',
#         publish_date=date(1997, 06, 27),
#         author=User(name='J. K. Rowling'))

# Loading list of objects
List(BookType).load([
    {'title': 'Harry Potter and the Philosopher\'s Stone',
     'publish_date': '1997-06-26',
     'author': {'name': 'J. K. Rowling'}},
    {'title': 'Harry Potter and the Chamber of Secrets',
     'publish_date': '1998-07-02',
     'author': {'name': 'J. K. Rowling'}},
])
# => [Book(...), Book(...)]

# Validation
BookType.validate({
    'title': 'Harry Potter and the Philosopher\'s Stone',
    'author': {'name': ''},
})
# => {'author': {'name': 'Length should be at least 1'},
#     'publish_date': 'Value is required'}

Installation

$ pip install lollipop

Documentation

Documentation is available at http://lollipop.readthedocs.io/ .

Requirements

• Python >= 2.6 or <= 3.6

Project Links

• Documentation: http://lollipop.readthedocs.io/
• PyPI: https://pypi.python.org/pypi/lollipop
• Issues: https://github.com/maximkulkin/lollipop/issues

License

MIT licensed. See the bundled LICENSE file for more details.

lollipop's Issues

Feature request: support for specifying type for dict keys

The current type Dict allows specifying types only for values, but we have a need to specify one type for all keys too..

Add support to specify descriptions for individual fields in schema

We have a lot of complex Lollipop schemas and we feel the need to supply both field level and object level descriptions. This could benefit our doc generation tools to enrich fields with associated descriptions..

Implement loading values for MethodField

Allow optionally specifying both serialize and deserialize method names. Serialize method should take no arguments and return value. Deserialize method should take value argument.

Implement time-related types

Add types to handle Date, DateTime, Time with builtin support for common formats (ISO8601, RFC3339, RFC822) as well as allowing specifying custom formats.

Number().validate(data=False)

Is it by design or is it a bug that this validation passes?

Number().validate(data=False)

I expected that it'd raise an Exception 'Value should be number'.

request: auto constructor for simple namedtuple use-case

Let's say a common use-case is the constructor class is from a namedtuple. Presently, a user would need to do the following:

Person = namedtuple('Person', ['name', 'age'])

PersonType = Object({
    'name': String(validate=Length(min=1)),
   'age': Integer()
}, constructor=Person)

The user is required to create a duplicate mapping of name fields in both Person and PersonType, in this case name and age.

Would it be conceivably useful to have do something like the following, and the lollipop library meta-creates the actual namedtuple type instance when an instance of the PersonType is created. Thus the sequence:

PersonType = Object({
    'name': String(validate=Length(min=1)),
    'age': Integer()
}, constructor='Person')       # lollipop see's a string vs. a class, knowns to do the magic

me = PersonType.loads(dict(name='jeremy', age=40))

me.name
# 'jeremy'

me.age
# 40

Asymmetry in load/dump for Dict with key_type and dict as values_type

I think there is a problem with the Dict type when using both key_type verification and a values_type dictionary:

In Dict.dump, the key is used in its original / non-dumped form to lookup the value type. However, in Dict.load, the dumped key is used to lookup the value type.

This works fine when using native types as key type such as String and Integer since they map to the same loaded/dumped value. But it's causing a problem when using a more complex key type (e.g. in my case an Enum that dumps to a string).

I believe, in Dict.load, key_type.load should be called before the lookup of the value type, so that the lookup is again performed with the original / non-dumped value.

Deprecation Warnings for `inspect.getargspec()` and `collections.abc`

Just a friendly reminder.

When I run tests through pytest on Python 3.8.10, I see warnings:

../home/vagrant/.local/lib/python3.8/site-packages/lollipop/compat.py:29
  /home/vagrant/.local/lib/python3.8/site-packages/lollipop/compat.py:29: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
    from collections import MutableMapping as DictMixin

../home/vagrant/.local/lib/python3.8/site-packages/lollipop/utils.py:46
  /home/vagrant/.local/lib/python3.8/site-packages/lollipop/utils.py:46: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
    arg_count = len(inspect.getargspec(func.__call__).args) - 1

../home/vagrant/.local/lib/python3.8/site-packages/lollipop/utils.py:42
  /home/vagrant/.local/lib/python3.8/site-packages/lollipop/utils.py:42: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
    arg_count = len(inspect.getargspec(func).args)

../home/vagrant/.local/lib/python3.8/site-packages/lollipop/utils.py:44
  /home/vagrant/.local/lib/python3.8/site-packages/lollipop/utils.py:44: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
    arg_count = len(inspect.getargspec(func.__init__).args) - 1

../home/vagrant/.local/lib/python3.8/site-packages/lollipop/utils.py:24
  /home/vagrant/.local/lib/python3.8/site-packages/lollipop/utils.py:24: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
    return isinstance(value, collections.Mapping)

-- Docs: https://docs.pytest.org/en/stable/warnings.html

I'm not creating a pull request since I'm not fully aware of what python versions are going to be supported. But it looks like some actions to resolve this are required.

Implement OneOf type

A special type that allows passing list of other types. When loading/dumping it sequentially tries to load/dump value with types and stopping on first successful result. Also, it should allow passing function that could hint on what particular type to use.

Misleading Tuple type

Is it on purpose that the Tuple type dumps into / loads from a list instead of a tuple? If yes, I think the naming is a bit misleading…

Tuple([String(), Integer(), Boolean()]).load(('foo', 123, False))
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "site-packages\lollipop\types.py", line 443, in load self._fail('invalid')
  File "site-packages\lollipop\errors.py", line 63, in _fail raise ValidationError(msg)
lollipop.errors.ValidationError: Invalid data: 'Value should be list'

Improve default error messages for lollipop fields

Current messages are not enough specific and don't actually help end user to figure out the problem.

For example if we use constant type:

>>> from lollipop import types
>>> x = types.Constant(2)
>>> x.validate(1)
'Value is incorrect'
# Instead of something more reasonable 
"Value 1 is incorrect, field is constant and can be only equal to 2"