Yaml serializer produces inconsistent results

[slackfan]

Yaml serializer produces inconsistent results

[fabiocaccamo]

@slackfan it's not mentioned in the documentation because the initial idea was to use it only "behind the scenes", I will probably add an official to_dict method.

[fabiocaccamo]

@slackfan thank you for reporting this issue, which version did you use before?

Could you provide me a minimal test case / example please?

[slackfan]

@fabiocaccamo, I have to partially correct myself and should have provided/analyzed this directly.

The observed behavior is not related to the versions but related to how the benedict looks like (as it seems to me).

Example

docker run -it --rm library/python:3 python3 -c '
import pip
pip.main(["install", "python-benedict==0.30.2", "pyyaml==6.0"])

class Custom:
    def __init__(self, val):
        self.val = val
    def __repr__(self, *args, **kwargs):
        return f"{self.__class__.__name__}(val={str(self.val)!r})"
    def __eq__(self, other):
        return isinstance(other, self.__class__) and self.__dict__ == other.__dict__
    def __ne__(self, other):
        return not self.__eq__(other)

def _representer(dumper, data):
    return dumper.represent_scalar("!custom", data.val)

import yaml

yaml.add_representer(Custom, _representer, Dumper=yaml.Dumper)

import benedict

output = benedict.benedict({"foo": "foo", "bar": Custom("bar")}, keypath_separator="/")
print(yaml.dump({"world": dict(output)}, default_flow_style=False))

output["hello/world"] = "hello/world"
print(yaml.dump({"world": dict(output)}, default_flow_style=False))
'

Produces the output

world:
  bar: !custom 'bar'
  foo: foo

world:
  bar: !custom 'bar'
  foo: foo
  hello: !!python/object/new:benedict.dicts.benedict
    dictitems:
      world: hello/world
    state:
      _dict:
        world: hello/world
      _keyattr_dynamic: false
      _keyattr_enabled: true
      _keypath_separator: /
      _pointer: true

See the two print() statements. Both print the same dict (even wrapped in a dict()). The first print() produces a yaml with serialized dict. The second print() produces a yaml with serialized benedict. The difference is "only" the use of the keypath feature.

My expectation was and is that both calls produce a yaml which looks like what the first print() produces. In any way, consistency would be great, how can it be accomplished?

My expected outcome would be

world:
  bar: !custom 'bar'
  foo: foo
  hello:
    world: hello/world

[fabiocaccamo]

@slackfan the problem is that you are serializing a benedict instance (that is a custom dict) using your own yaml serializer.

To avoid this problem you can:

• use the benedict internal yaml serializer using to_yaml method
• pass the internal pure dict instance to the yaml serializer using the dict() method

Unfortunately it's not possible to intercept when a benedict instance gets casted to dict.

Here a minimal test:

from benedict import benedict
import yaml


b = benedict({"foo": "foo"})
b["hello.world"] = "hello world"

# output as custom object using yaml manually (includes state)
print(yaml.dump({"world": dict(b)}))

# output as custom object using yaml manually (includes state)
print(yaml.dump({"world": b}))

# output as normal dict using yaml manually
print(yaml.dump({"world": b.dict()}))

# output as normal dict using benedict yaml serializer
print(benedict({"world": b}).to_yaml())

[slackfan]

Indeed. This boils down to the simple problem that I was not aware of the benedict.dict() method! It is not mentioned in the documentation. I am happy to close the issue. Thanks!