m1ha-shvn/django-pg-hll

Provides a django wrapper for postgresql-hll library by CitusData

django-pg-hll's Issues

Can't create empty HllSet

Can't create empty HllSet() without parameters

Add functions from reference and fix tests

As provided in PR #7 by @markallanson, not all hll functions available are implemented in this library.
I've found a reference.

TODO list

• Implement hll_set_defaults, hll_print
• Implement hll_schema_version, hll_type, hll_regwidth, hll_log2m, hll_expthresh, hll_sparseon functions
• Implement hll_empty, hll_add_agg optional configuration arguments
• Implement hll_eq, hll_ne filters and functions
• Rewrite test HllFieldTest.test_create_custom_params using hll empty optional parameters

Annotating and ordering by daily/weekly/monthly stats

I'm trying to leverage the power of hll by having annotations for stats within a date range, but unsure of the best way to approach this. Basically, I'm trying to somewhat mimic the date stuff they've described in https://github.com/citusdata/postgresql-hll#data-warehouse-use-case

My use case is needing all of the results, but ordered by view count (like needing to order by top viewed this month). So just .filter() within a date range wouldn't work by itself since it filters out objects from the queryset.

Here's a model example:

class Thing(models.Model):
    ...


class Stats(models.Model):
    parent = models.ForeignKey(Thing, related_name='stats', on_delete=models.CASCADE)
    hll = HllField()
    date = models.DateField(auto_now_add=True, editable=False)

My attempt at it:

from django.utils.datetime_safe import date

monthly = date.today() - timedelta(days=30)

queryset = Thing.objects.all().annotate(
    month_view_count=Coalesce(UnionAggCardinality('stats__hll', filter=Q(stats__date__gte=monthly)), 0),
).order_by('month_view_count')

queryset[0]

django.db.utils.ProgrammingError: FILTER specified, but hll_cardinality is not an aggregate function
LINE 1: ...l_timestamp", "things_thing"."featured", COALESCE(hll_cardin...
                                                             ^

Any ideas of what I should look more into or try out? I've been trying to avoid getting really messy, and it feels like there's a 1 liner here that I'm not thinking of that would do it.

I'll be looking into postgres conditionals next, but curious if this is easier than what I'm making it out to be.