reagento/adaptix

An extremely flexible and configurable data model conversion library.

adaptix's Issues

Ошибка парсинга datetime если он уже нужного типа

from dataclasses import dataclass
from dataclass_factory import Factory
from datetime import datetime


@dataclass()
class Traffic:
    time: datetime
    rx: int
    tx: int


if __name__ == '__main__':
    dict_traff = {
        'time': datetime(2019, 12, 1, 1, 20, 13),
        'rx': 10,
        'tx': 15
    }

    factory = Factory()
    traff: Traffic = factory.load(dict_traff, Traffic)
    print(factory.dump(traff))
    
    

Ошибка:

Traceback (most recent call last):
  File "E:/NMS/NMSProgect/modules/traffic.py", line 24, in <module>
    traff: Traffic = factory.load(dict_traff, Traffic)
  File "E:\NMS\lib\site-packages\dataclass_factory\factory.py", line 94, in load
    return self.parser(class_)(data)
  File "E:\NMS\lib\site-packages\dataclass_factory\parsers.py", line 139, in dataclass_parser
    for field, name, parser in field_info
  File "E:\NMS\lib\site-packages\dataclass_factory\parsers.py", line 140, in <dictcomp>
    if name in data
  File "E:\NMS\lib\site-packages\dataclass_factory\parsers.py", line 186, in class_parser
    k: parser(data.get(k)) for k, parser in parsers.items() if k in data
  File "E:\NMS\lib\site-packages\dataclass_factory\parsers.py", line 186, in <dictcomp>
    k: parser(data.get(k)) for k, parser in parsers.items() if k in data
TypeError: argument of type 'datetime.datetime' is not iterable

Поддержать работу с спецсимволами

node.js возвращает поля с спецсимволами "foo-bar$1"
былобы удобно обьявить поле value_1 а получать значения из value-1

There is no ignore namestyle parameter

There is no Schema parametr, which do not change fields namestyle to one concrete, like bool ignore_namestyle

Implement update_forward_refs for delayed forward refs calculation

аналогичная функциональность в Pydantic

model_a.py

from pydantic import BaseModel


class ModelA(BaseModel):
    field: int

model_b.py

from __future__ import annotations

from typing import TYPE_CHECKING

from pydantic import BaseModel

if TYPE_CHECKING:
    from model_a import ModelA


class ModelB(BaseModel):
    field: int
    a: ModelA

main.py

from model_b import ModelB

print(ModelB.__fields__)
-> {'field': ModelField(name='field', type=int, required=True), 'a': ModelField(name='a', type=ForwardRef('ModelA'), required=PydanticUndefined)}

по type=ForwardRef и в этом месте неизвестно что именно из себя представляет ModelA для ModelB потому что его нет в скоупе модуля model_b.

для этого есть ModelB.update_forward_refs()
вызываем из main.py
только прямой вызов ModelB.update_forward_refs() ничего не сделает - NameError: name 'ModelA' is not defined, потому что в модуле где находится класс все еще нет такого имени

делаем следующее - ModelB.update_forward_refs(ModelA=ModelA)

и далее снова смотрим поля:
{'field': ModelField(name='field', type=int, required=True), 'a': ModelField(name='a', type=ModelA, required=True)}
ModelA теперь имеет реальный тип а не референс

very few examples with namestyles

What will happen if some of the fields are in one style and some in another? How does it work in practice? I would like to see more examples in Readme.nd

'list' can't work

import dataclass_factory
from dataclasses import dataclass


@dataclass
class Problem:
    name: str
    eventid: int
    urls: list


data = {
    'name': 'TestProblem',
    'eventid': 1,
    'urls': []
}

factory = dataclass_factory.Factory()
problem: Problem = factory.load(data, Problem)
print(problem)

Traceback (most recent call last):
  File "C:/Users/themida/Documents/project/test.py", line 19, in <module>
    problem: Problem = factory.load(data, Problem)
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\factory.py", line 118, in load
    return self.parser(class_)(data)
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\factory.py", line 77, in parser
    return self._parser_with_stack(class_, StackedFactory(self))
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\factory.py", line 93, in _parser_with_stack
    schema.parser = create_parser(stacked_factory, schema, self.debug_path, class_)
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\parsers.py", line 256, in create_parser
    parser = create_parser_impl(factory, schema, debug_path, cls)
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\parsers.py", line 340, in create_parser_impl
    parsers = {
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\parsers.py", line 341, in <dictcomp>
    field.name: factory.parser(resolved_hints[field.name])
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\factory.py", line 29, in parser
    return self.factory._parser_with_stack(class_, self)
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\factory.py", line 93, in _parser_with_stack
    schema.parser = create_parser(stacked_factory, schema, self.debug_path, class_)
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\parsers.py", line 256, in create_parser
    parser = create_parser_impl(factory, schema, debug_path, cls)
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\parsers.py", line 316, in create_parser_impl
    if args_unspecified(cls):
  File "C:\Users\themida\Documents\project\venv\lib\site-packages\dataclass_factory\type_detection.py", line 119, in args_unspecified
    (not cls.__args__ and cls.__parameters__) or
AttributeError: type object 'list' has no attribute '__args__'

Too slow

Leave None type as it is.

I have a dataclass which str item default is "a" and int item default is 2.

But in data dict, their value is None.

When I call factory.load(data, DataClass), I want to leave the None type as it is, don't change it.
Or I don't want to get an exception when type .

Any solution?
Thanks

Skip default None

When serialising skip None values if it is also set as default

We need new field in Schema and change logic of dataclass serializer.

Skipping None for all Optionals is not safe, as object cannot be created from such a dict.

Skipping other defaults is not safe enough, but can be separatly implemented

Setting this feature for each field should be done separately

Detect float and complex numbers

Deserialize recursive hierarchy of optional lists

How can I deserialize a recursive structure?

I have seen #17 but I am getting errors trying to deserialize the following:

@dataclass(frozen=True)
class JsonSport:
    sport_id: int
    sport_name: str
    sport_description: Optional[str]
    sports: Optional[List["JsonSport"]]

The error is
dataclass_factory.exceptions.InvalidFieldError: Invalid data at path [sports]: No suitable parsers in union found for [{'sportId': 63, 'sportName': 'Idrettsskoler', 'sportDescription': 'Idrett generelt...`

Make errors really verbose

It is very difficult to understand what going wrong during parsing

Validate TypedDict

1. Support of creating TypedDict instances (from both mypy_extensions and typing) from normal Dict.
2. Do not break compatibility with 3.6
3. Do not add new dependencies
4. Need validation of __total__

Recursive parsing

@dataclass 
class List:
   data: int
   next: Optional[List]

This should be parsed as well

Provide JSONEncoder

Make JSONEncoder compatible version of factory

https://docs.python.org/3/library/json.html#json.JSONEncoder

Name mapping is not disclosed

Example with https://github.com/Tishka17/dataclass_factory#structure-flattening full of magic
Could you explain how this works? So far, all this is not obvious. What kind of magic zero..?

None for Optinal[bool] can be parsed as False

Parsing interfaces/protocols

Add to schema "impl" attribute which provides real type of interface.

So factory.load(data, Interface) will be equal to factory.load(data, impl)
Also must be applyed to fields with Interface type

Option to parse invalid Enum value as None

Incorrect parsing of tuple

Tuple type; Tuple[X, Y] is the type of a tuple of two items with the first item of type X and the second of type Y.

Example: Tuple[T1, T2] is a tuple of two elements corresponding to type variables T1 and T2. Tuple[int, float, str] is a tuple of an int, a float and a string.

To specify a variable-length tuple of homogeneous type, use literal ellipsis, e.g. Tuple[int, ...]. A plain Tuple is equivalent to Tuple[Any, ...], and in turn to tuple.

Incorrect schema detection during serialization of Generics classes

see feature/schema_serial

Deserialization fails for Optional[Dict[...]]

Trying to deserialize a nullable Dict[T, U] results in get_dict_parser being called anyway and raising an AttributeError due to the target dict being None.

Testing with Python 3.7 and dataclass_factory 2.4.1 the following code will raise:

from typing import Optional, Dict, Any
from dataclass_factory import Factory

@dataclass
class Test:
    x: Optional[Dict[str, Any]]

f = Factory()
f.load({"x": None}, Test)

The root issue seems to be that Optional is not being detected correctly as it is indistinguishable from Union[T, NoneType], which ends up in get_dict_parser raising due to the order of the type arguments.

Поддержать работу с Generic

Пример:

T = TypeVar('T')

@dataclass
class Foo(Generic[T]):
    value: T

Должен парситься Foo[int]
Он не детектится как датакласс, надо работать с Origin

Prepare data before passing to custom parser

Custom parser can be provided in Schema.

It is ok to parse data in dict before passing to custom parser or provide a way to call use factory inside custo mparser.

1. Pass factory instance
2. Check custom parser anotations
3. Add zero-step parsing setting (e.g. link to other type)

model inheritance

Допустим есть классы с наследованием

@dataclass
class A:
   a: int

@dataclass
class B(A):
   b: int

Сейчас, если мы хотим сделать предобработку поля a, мы должны сделать две схемы: для класс A и для класса B и в обеих определить pre_parse.

Необходимо продумать как это делать более удобно

Flatten nested structures

Serialize classes in case of init-based parsing

We can parse class if it has init with annotations. Probably it is ok to serialize it as well. All schema features must be included

Ошибка парсинга иерахии, если она начинается со списка

@dataclass
class A:
    x: str
    y: str


schema_num = Schema[A](
    name_mapping={
        "x": (1, "a", "b"),
    }
)


class Test1(TestCase):
    def test_load_num(self):
        factory = Factory(
            schemas={
                A: schema_num
            }
        )
        data = {
            "a": [None, {"b": "hello"}],
            "y": "world"
        }
        expected = A("hello", "world")
        self.assertEqual(expected, factory.load(data, A))

Ошибка:

======================================================================
ERROR: test_load_num (test_path.Test1)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tishka17/src/dataclass_factory/tests/test_path.py", line 39, in test_load_num
    self.assertEqual(expected, factory.load(data, A))
  File "/home/tishka17/src/dataclass_factory/dataclass_factory/factory.py", line 94, in load
    return self.parser(class_)(data)
  File "/home/tishka17/src/dataclass_factory/dataclass_factory/parsers.py", line 139, in dataclass_parser
    for field, name, parser in field_info
TypeError: __init__() missing 1 required positional argument: 'x'

fractions.Fraction support

json factory wrapper

class JsonFactory:
   ...

   def dump(self, data, type):
      return json.dumps(self.real_factory.dump(data, type))

   def load(self, data, type):
      return self.real_facory.load(json.loads(data), type)

Parse user-defined classes

Idea: check class __init__ method args

missing import in docs

https://github.com/Tishka17/dataclass_factory#schemas
from dataclass_factory import Schema
skipped in example

Programming Language :: Python :: 3.7

Нужно поднять версию Programming Language так как с 3.7 добавлены датаклассы
https://www.python.org/dev/peps/pep-0557/

Validate Literal

• check value
• check type
• support multiple values
• both typing in python 3.8 and typing_extensions

missing import in readme

https://github.com/Tishka17/dataclass_factory#polymorphic-parsing
Missing import dataclass

Parse user-defined collections

Get form multivalue using methods like getlist

In werkzeug MultiDict allows to use getlist to get multiple values for single name in form data.

This could be useful for filing dataclass List field

Polymorphic parsing

Create instruction or provide builin mechanism for creating polymorphics parsers.

Skip some field when serializing

Store unparsed fields separately

Store all extra input fields, which absent in dataclass, separately in field with name from schema.rest, if provided.

Also unpack rest when serialising

dict_factory with all features of parse method

Do serialization to primitives, that can be stored in extrernal structured format (e.g. json) and than parsed

Enum is not serializing as Value

>>> class A(Enum):
...   a=1
...   b="s"
... 
>>> Factory().dump(A.a)
<A.a: 1>

Serialization

• name styles
• dataclass
• list
• dict
• tuple
• Any
• Optional
• Union
• classes
• cache serializers
• custom serializers

Typing support

Hi! Thanks for the awesome project.

It looks like you have covered almost all of your source code with type annotations, but currently it is not shipped to the final user.

I suggest to fix this situation. Here's how:

1. Create py.typed file in the root of the dataclass_factory folder, it is required for PEP561 to work. Example: https://github.com/dry-python/returns/blob/master/returns/py.typed
2. Change how public API is typed. Some method do miss return values. Some of them are missing generic parameters. Like this one: https://github.com/Tishka17/dataclass_factory/blob/master/dataclass_factory/factory.py#L80 It should be something like: def load(self, data: Any, class_: Type[T]) -> T: (do not forget about Generic base class)
3. Integrate mypy to check these annotations

This feature would improve developer experience and make this project even more awesome!

Field validation

Did not see custom validation of fields except types. For example here https://marshmallow.readthedocs.io/en/stable/quickstart.html#validation . For example, expect a certain length or a certain value from a list of valid. Also multiple validation of one field, for example a password field

dataclass <-> tuple

Load/dump many objects

Do you think to write the example in the docs with many parameter (like marshmallow), i.e. list comprehesion or as an alternative with a new dataclass and List[Item]?

обрабатывать правила для полей в случае init-based парсинга и TypedDict

A lot of parameters examples are missing in docs

A lot of parameters, which description we have at docs, have no practical examples, which can helps to understand it better. Could you write more examples in the documentation?

конвертация между сложными типами

dataclass <-> dataclass
dataclass <-> protobuf