TDigestSQL results from the TDigestUDAF cannot themselves be aggregated in Spark SQL.

[isarn]

TDigestSQL results from the TDigestUDAF cannot themselves be aggregated in Spark SQL.

[erikerlandson]

Thanks @w00te, I will integrate this into the package!

[erikerlandson]

@johnwhumphreys I added t-digest aggregators (reducers) TDigestReduceUDAF and TDigestArrayReduceUDAF, they will land in my next release:

https://github.com/erikerlandson/isarn-sketches-spark/blob/feature/pyspark/src/main/scala/org/isarnproject/sketches/udaf/TDigestUDAF.scala#L232

[johnwhumphreys]

Awesome :) thank you for pulling that in! Still have yet to try the parquet serialization fix you suggested; but it's in my notes for later when I have time to go back to that. Using the object file serialization for the moment as you provided earlier.

…

On Sat, Jul 29, 2017 at 2:55 PM, Erik Erlandson ***@***.***> wrote: @johnwhumphreys <https://github.com/johnwhumphreys> I added t-digest aggregators (reducers) TDigestReduceUDAF and TDigestArrayReduceUDAF, they will land in my next release: https://github.com/erikerlandson/isarn-sketches- spark/blob/feature/pyspark/src/main/scala/org/isarnproject/sketches/udaf/ TDigestUDAF.scala#L232 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AGeRcso2detEBYR7j0SxB7A5UtKMKJqPks5sS4ApgaJpZM4OeYuy> .

[erikerlandson]

I'll optimistically close this in anticipation of merging for upcoming release