Generate ObjectField/NestField properties from a DocType class

[django-es]

Generate ObjectField/NestField properties from a DocType class

[stevelacey]

Update: I actually just got pretty far trashing dupe fields with this:

def get_properties(doctype):
    return dict(
        id=fields.StringField(),
        **doctype()._doc_type._fields(),
    )

# ...

    fields.ObjectField(properties=get_properties(AssetFileDocument))

An added benefit of using documents here is that you can also rely on the type inference for fields taken from the models, I am most using the Meta fields list now in place of the properties dict.

[stevelacey]

Another thought related to above, it would be handy to be able to build ObjectFields out of models using a fields list – right now I have documents I never intend to index alone, but use as ObjectFields using above get_properties call – which could be simplified this way.

[stevelacey]

If you want to feedback on the approach of any of the above @sabricot I'll take a look at implementing and submit a PR.

[stevelacey]

Slight improvement:

from django_elasticsearch_dsl import fields


class DocumentField(fields.ObjectField):
    """
    An ObjectField that inherits its properties from a DocType class
    https://www.elastic.co/guide/en/elasticsearch/reference/current/object.html#object
    """
    def __init__(self, doctype, *args, **kwargs):
        kwargs['properties'] = doctype()._doc_type._fields()
        super().__init__(*args, **kwargs)

    def get_value_from_instance(self, instance):
        """Store nulls in Elasticsearch instead of empty objects"""
        objs = super().get_value_from_instance(instance)
        if objs == {}:
            return None
        return objs

quotes = fields.ListField(DocumentField(NestedPressQuoteDocument))
upload = DocumentField(UploadDocument, attr='get_upload')

Edit: I've updated the above with a more recent working example

[stevelacey]

I've updated the example above to match what I've been using for a while now, works well

I couple this with some helpers for building out simple DocTypes with less boilerplate

utils.py

from django_elasticsearch_dsl import DocType


def fields_to_index(model, exclude=[]):
    """Returns list of field names to index, given a Django model class"""
    return [
        field.name for field in model._meta.get_fields()
        if field.name not in ['id', *exclude] and field.concrete and not field.is_relation
    ]


def model_to_doctype(model, bases=(DocType,), fields={}):
    """An extension of the type built-in for creating django_elasticsearch_dsl DocTypes"""
    return type(model.__name__ + 'DocType', bases, {
        **fields,
        'Meta': type('Meta', (), {
            'model': model,
            'fields': fields_to_index(model, exclude=fields.keys()),
        })
    })

This can brings a pretty elaborate nested document structure down to under a 40 LOC – I only use full class declarations for the actual (root) documents that are indexed, and all of the DocType's used for DocumentField's are magicked up using these helpers

[sabricot]

Thanks @stevelacey,
If you want to make a PR for this it will be with pleasure.