netzkolchose/elastipy

python elasticsearch query module for easily accessing nested aggregations and such

elastipy's Issues

support dict_rows (and df) for non-bucket aggregations

e.g. make that possible:

(Search.
.metric_x()
.metric_y(return_self=True)
.execute().df())

currently raises: Can not call dict_rows() on non-bucket aggregation Aggregation('a0', 'x')

elasticsearch v8 support

ES 8 comes with security by default which is a bit annoying because it's not well documented how to tune the Elasticsearch python client.

However! First try to make the unittests pass by disabling security for the ES 8 node

wrongly structured compound queries

Below is an example of an elastipy query that is written out for clarity. Usually it's created generative via for-loops.

In any case, it does not create the expected query DSL.

from elastipy import Search, query

Search() & (
    (
        ~query.Term("channel_id", "channel-1")
        | query.Range("timestamp", gte="2021-01-01")
    )
    & (
        (
            query.Term("channel_id", "channel-1")
            & query.Term("fulfillment_by", "self")
        ) 
        | (
            query.Term("channel_id", "channel-1")
            & query.Term("fulfillment_by", "other")
        )
    )
)

e.g. the above results in

{
  "bool": {
    "must": [
      {
        "match_all": {}
      },
      {
        "bool": {
          "should": [
            {
              "bool": {
                "must_not": [
                  {
                    "term": {
                      "channel_id": {
                        "value": "channel-1"
                      }
                    }
                  }
                ]
              }
            },
            {
              "range": {
                "timestamp": {
                  "gte": "2021-01-01"
                }
              }
            },
            {
              "bool": {
                "must": [
                  {
                    "term": {
                      "channel_id": {
                        "value": "channel-1"
                      }
                    }
                  },
                  {
                    "term": {
                      "fulfillment_by": {
                        "value": "self"
                      }
                    }
                  }
                ]
              }
            },
            {
              "bool": {
                "must": [
                  {
                    "term": {
                      "channel_id": {
                        "value": "channel-1"
                      }
                    }
                  },
                  {
                    "term": {
                      "fulfillment_by": {
                        "value": "other"
                      }
                    }
                  }
                ]
              }
            }
          ]
        }
      }
    ]
  }
}

KeyError in Aggregation.dict_rows()

using parking-data export:

(search().terms("place_id", valid_place_ids)
    .agg_terms("place",field="place_id", size=1000)
    .agg_date_histogram("week", calendar_interval="week")
    .metric_avg("free", field="num_free")
    .pipeline_derivative("grad", buckets_path="free")
    .execute()
    .dict_rows()
)