Feature Request: Support "camelCase" and "snake_case" manipulations

[lidatong]

Feature Request: Support "camelCase" and "snake_case" manipulations

[lidatong]

Hi, this is now supported in 0.2.10. Note I'm aware it is currently not ergonomic to override each field individually... next on my list is enabling class (rather than field) level config

[Glandos]

I don't know how easy it would be to support attribute and data_key from marshmallow.fields, but it would partially solve this issue and #47.

[Glandos]

See https://gist.github.com/Glandos/fe5910ff217b65b570386105cda75c1f

This is a proof of concept. But for now, it needs two things:

• declare your field with something special. Here, dcj_data_key just create a field with some templated metadata
• transform the field at the class level with the @callable_data_keys decorator. This is needed since there's no way to know the field name when creating it.

[lidatong]

Thanks, @USSX-Hares, this is definitely something I think this library should support.

In fact I've been pondering whether it should encode/decode fields with camelCase by default, since that is the JSON convention. Of course that would be a breaking change, and would result in an appropriate notice / version bump for the next release.

For now, I'm going to keep things backwards compatible with the existing API, and allow the user to specify casing.

Thanks @Glandos for the proposal. Rather than creating a separate decorator though, I'm going to favor parameterizing the existing dataclass_json one. I'm working on implementing something right now, and am aiming to have something in by the end of this weekend

[dodgyville]

Hi, I think the changes related to this have caused a problem in the latest release (0.2.8 or maybe earlier).

The decorator defined in api.py:dataclass_json needs *args, **kwargs so something like:

def dataclass_json(cls, *args, **kwargs):

or

def dataclass_json(cls):

instead of

def dataclass_json(cls,
                   *,
                   encode_letter_case: LetterCase,
                   decode_letter_case: LetterCase):

or else I get
TypeError: dataclass_json() missing 2 required keyword-only arguments: 'encode_letter_case' and 'decode_letter_case'

[lidatong]

Thanks for reporting this @dodgyville. I've fixed this in 0.2.9

[Glandos]

Hi, this is now supported in 0.2.10. Note I'm aware it is currently not ergonomic to override each field individually... next on my list is enabling class (rather than field) level config

I was thinking of that during the week-end. Your current implementation is fine, because it sanely uses the dataclass metadata:

@dataclass_json
@dataclass
class Person:
    given_name: str = field(
        metadata={'dataclasses_json': {
            'letter_case': LetterCase.CAMEL
        }}
    )

However, it might be better from a user point-of-view to have some new methods on the Field class that give something much cleaner like this:

@dataclass_json
@dataclass
class Person:
    given_name: str = field().with_dcj_meta(letter_case=LetterCase.CAMEL)

I just made up the naming of with_dcj_meta so it is not the best one. But the advantages of this are numerous:

• Calling field() remains easy and uncluttered, even with non-default arguments.
• If the Field class init change, there is no argument name clash, compared to a function like dcj_field(letter_case=LetterCase.CAMEL) that tries to forward all arguments to field() function.
• Having some specialized methods ensure backward compatibility if the metadata structure changes
• Chained call are better than composed call like dcj_meta(field(), letter_case=LetterCase.CAMEL) IMHO.

Of course, this requires monkey patching of Field class. This is something that a lot of people want to avoid, but it can be done nonetheless.

[lidatong]

@Glandos yep, i agree with your point on not using a dict

i think your other points aim to make the API stylistically "sexier", i think the main pain point is it's using a dictionary when it really shouldn't... basically the config parameters are "ad-hoc" when they should be "well-defined", e.g. obvious to the user what the params to pass in are without having to resort to docs (#103)

opinion-wise i agree with you that chained is more readable than composed! a.do_this().do_that() is just more natural then do_that(do_this(a)). however, python is unfortunately not a language that supports that well. as you note, you need to monkey patch, rather than have built-in language support for something like extension methods (e.g. a much loved feature in Kotlin)

[USSX-Hares]

@lidatong @Glandos
Anyway, it would be nicer to have case manipulations be on the top-most level, instead of having a naming policy being declared for each field individually.

I mean something like this:

# This will declare a class-related style,
# which would be used for class
# and all its fields (recursively)
# unless stated otherwise.
@dataclass_json(letter_case=LetterCase.KEBAB)
@dataclass
class Person:
    given_name: str

person = Person("Peter")
person.to_json() # {"given-name":"Peter"}
# These will override the class-declared styles:
person.to_json(letter_case=LetterCase.CAMEL) # {"givenName":"Peter"}
person.to_json(letter_case=LetterCase.KEBAB) # {"given-name":"Peter"}
person.to_json(letter_case=LetterCase.SNAKE) # {"given_name":"Peter"}

I ask for this because of some models having a VERY large number of fields, and, furthermore, situations where one style should be converted to another

Update:

I've seen the configured_dataclass_json() decorator, so it covers half of I've written above

[lidatong]

yep, i definitely agree with providing config at the method level

i can't say i agree with recursive application. when i think about the design of the API, i would want the child's configuration to propagate, not the parent's. an "override" maybe should be configurable via the API, but i have been and want to actively avoid a combinatoric explosion of parameters -- i would want to reflect on the applicability of the use-case of overriding the child

[Glandos]

built-in language support for something like extension methods (e.g. a much loved feature in Kotlin)
If I understood correctly, built-in support is needed for statically typed language. But in Python, you can simply do:

import dataclasses
import types

def dcj_letter_case(self, case):
    new_metadata = dict(self.metadata)
    new_metadata.setdefault('dataclasses_json', {})['letter_case'] = case
    self.metadata = types.MappingProxyType(new_metadata)

dataclasses.Field.with_case = dcj_letter_case

This is called monkey patching, but I do not see a real difference with Kotlin extension methods, except for the fact that you have access to all class attributes, and not only public ones.

@USSX-Hares of course, a global setting is needed, but I think this was already in @lidatong mind. I have some use cases where it is useful to disable the conversion for some given field. Using the above method, I'd like to use it like this:

@dataclass_json(letter_case=LetterCase.KEBAB)
@dataclass
class Person:
    given_name: str
    raw_argument: str = field().with_case(None)

Regarding recursion, I agree that it shouldn't be on by default, if even available. It shouldn't be too much difficult to add the parameter to each class. And if you have too much classes, maybe you can generate them to avoid that.

[USSX-Hares]

I did not mean a global override.
I meant only override for the current class and classes it contains as fields, and override for current to/from dict/json method

[USSX-Hares]

Example:

@dataclass
class Person(DataClassJsonMixin):
    given_name: str

PoliticalSystem = NewType('PoliticalSystem', str)

@with_camel_case
@dataclass
class Goverment(DataClassJsonMixin):
    goverment_residence: str
    political_system: PoliticalSystem
    leader: Person

@dataclass
class Country(DataClassJsonMixin):
    country_name: str
    country_goverment: Goverment
    loyal_people: List[Person] = field(default_factory=list)

c1 = Country("USSR", Goverment("kremlin", PoliticalSystem("communism"), Person("J. Stalin")))
c2 = Country("USA", Goverment("white house", PoliticalSystem("capitalism"), Person("H. Trueman")))

print(c1.to_json(indent=4, sort_keys=True))
print(c1.to_json(indent=4, sort_keys=True, letter_case=LetterCase.KEBAB))

To get output:

{
    "country_government": {
        "governmentResidence": "kremlin",
        "leader": {
            "givenName": "J. Stalin"
        },
        "politicalSystem": "communism"
    },
    "country_name": "USSR",
    "loyal_people": []
}
{
    "country-government": {
        "government-residence": "kremlin",
        "leader": {
            "given-name": "J. Stalin"
        },
        "political-system": "communism"
    },
    "country-name": "USSR",
    "loyal-people": []
}

However, if add with_snake_case decoration on the Person, the output will change:

{
    "country_government": {
        "governmentResidence": "white house",
        "leader": {
            "given_name": "H. Trueman"
        },
        "politicalSystem": "capitalism"
    },
    "country_name": "USA",
    "loyal_people": []
}
{
    "country-government": {
        "government-residence": "white house",
        "leader": {
            "given-name": "H. Trueman"
        },
        "political-system": "capitalism"
    },
    "country-name": "USA",
    "loyal-people": []
}

[USSX-Hares]

Probably (probably) the style override should work this way (not the way we all discussed above):

1. If the field has the specific encoder/decoder or style configured, use it.
2. If the field is a dataclass with encoder/decoder or style configured, use it.
3. If the calling method (from the current recursion level, not the initial one) has an override configured, use it.
4. If the calling method is initial

Something like this pseudocode:

from enum import Enum, unique, auto

import json
from dataclasses import dataclass, field, fields, Field, is_dataclass
from typing import NewType, List, TypeVar, Generic, Dict, Any, Optional, Union

from camel_case_switcher import dict_keys_underscore_to_camel_case, underscore_to_camel_case
from dataclasses_json import DataClassJsonMixin, LetterCase

# @unique
# class LetterCase(Enum):
#     CAMEL = auto()
#     KEBAB = auto()
#     SNAKE = auto()
# 
def with_letter_case(cls, letter_case: LetterCase):
    meta: dict = getattr(cls, '__meta__', {})
    meta.setdefault('dataclasses_json', {})
    meta['dataclasses_json']['letter_case'] = letter_case
    setattr(cls, '__meta__', meta)
    return cls

def with_snake_case(cls):
    return with_letter_case(cls, LetterCase.SNAKE)
def with_camel_case(cls):
    return with_letter_case(cls, LetterCase.CAMEL)
def with_kebab_case(cls):
    return with_letter_case(cls, LetterCase.KEBAB)

JsonObject = Dict[str, Any]
Json = Union[JsonObject, List, str, int, float, bool, None]

T = TypeVar('T', bound=DataClassJsonMixin)
class Encoder(Generic[T]):
    @classmethod
    def encode_key(cls, f: Field) -> str:
        pass
    @classmethod
    def encode(cls, instance: T) -> JsonObject:
        pass
    @classmethod
    def decode(cls, j: JsonObject) -> T:
        pass

def encode_key(key: str, style: LetterCase) -> str:
    if (style == LetterCase.CAMEL):
        return underscore_to_camel_case(key)
    elif (style == LetterCase.KEBAB):
        return key.replace('_', '-')
    else:
        return key

def encode_object(instance: T, *, encoder: Encoder[T] = None, letter_case: LetterCase = None) -> Json:
    # print(f"encode_object() called on {instance} (type={type(instance).__name__}, encoder={encoder}, letter_case={letter_case})")
    
    if (encoder is not None):
        return encoder.encode(instance)
    
    if (not isinstance(instance, DataClassJsonMixin)):
        return json.dumps(instance)[1:-1]
    
    res = dict()
    _cls_meta = getattr(instance, '__meta__', { }).get('dataclasses_json', { })
    _cls_encoder = _cls_meta.get('encoder')
    _cls_letter_case = _cls_meta.get('letter_case')
    for f in fields(instance): # type: Field
        field_value = getattr(instance, f.name)
        _meta = f.metadata.get('dataclasses_json', {})
        _enc: Optional[Encoder] = _meta.get('encoder', _cls_encoder or encoder)
        _style: Optional[LetterCase] = _meta.get('letter_case', _cls_letter_case or letter_case)
        if (_enc is not None):
            key = _enc.encode_key(f)
            value = _enc.encode(field_value)
        else:
            key = encode_key(f.name, _style)
            value = encode_object(field_value, encoder=_enc, letter_case=_style)
        # print(f"Encoding key: '{f.name}' => '{key}'")
        res[key] = value
    
    return res

# @with_snake_case
@dataclass
class Person(DataClassJsonMixin):
    given_name: str

PoliticalSystem = NewType('PoliticalSystem', str)

@with_camel_case
@dataclass
class Government(DataClassJsonMixin):
    government_residence: str
    political_system: PoliticalSystem
    leader: Person

@dataclass
class Country(DataClassJsonMixin):
    country_name: str
    country_government: Government
    loyal_people: List[Person] = field(default_factory=list)

c1 = Country("USSR", Government("kremlin", PoliticalSystem("communism"), Person("J. Stalin")))
c2 = Country("USA", Government("white house", PoliticalSystem("capitalism"), Person("H. Trueman")))

print(json.dumps(encode_object(c2), sort_keys=True, indent=4))
print(json.dumps(encode_object(c2, letter_case=LetterCase.KEBAB), sort_keys=True, indent=4))

{
    "country_government": {
        "governmentResidence": "white house",
        "leader": {
            "givenName": "H. Trueman"
        },
        "politicalSystem": "capitalism"
    },
    "country_name": "USA",
    "loyal_people": ""
}
{
    "country-government": {
        "governmentResidence": "white house",
        "leader": {
            "givenName": "H. Trueman"
        },
        "politicalSystem": "capitalism"
    },
    "country-name": "USA",
    "loyal-people": ""
}