Handle transform errors in `FTSeries`

[brucala]

Handle transform errors in `FTSeries`

[joshday]

I've become fond of wrapper types since you can compose them. At some point I should change FTSeries to be FilterTransform wrapper type.

How about something like this that tracks the CountMap of exceptions you run into?

struct TryCatchStat{T, O<:OnlineStat{T}} <: OnlineStat{T}
    stat::O
    exceptions::CountMap{Exception}
end
TryCatchStat(stat::OnlineStat) = TryCatchStat(stat, CountMap(Exception))

nobs(o::TryCatchStat) = nobs(o.stat)

value(o::TryCatchStat) = value(o.stat)

function _fit!(o::TryCatchStat, y)
    try
        _fit!(o.stat, y)
    catch ex
        _fit!(o.exceptions, ex)
    end
end

[joshday]

Doing a little bit of work on this and I'm liking how it's turning out:

julia> o = TryCatch(FilterTransform(Mean(), String; transform = x -> parse(Int, x)))
TryCatch(FilterTransform(Mean)): n=0 | value=0.0

julia> fit!(o, "1")
TryCatch(FilterTransform(Mean)): n=1 | value=1.0

julia> fit!(o, "1.5")
TryCatch(FilterTransform(Mean)): n=1 | errors=1 | value=1.0

julia> OnlineStatsBase.errors(o)
OrderedCollections.OrderedDict{DataType, Int64} with 1 entry:
  ArgumentError => 1

[joshday]

Awesome, thanks for your input! I've incorporated most of your suggestions.

The only thing I left out was using Groupby to organize error messages by Exception for TryCatch. I tried it out and the code started looking a little too gnarly for the added benefit.

[brucala]

oh, I think that's really smart!

With that proposal one would't just be keeping track of failed transformations like in my proposal, but also of any other possible exceptions. Following the FTSeries example, these extra exceptions could be for instance errors in the filter function call, cases where the type of the transformed input does't correspond to the expected type, ....

And I agree, moving from FTSeries to a FilterTransform wrapper makes a lot of sense.

[brucala]

I had a quick look at the new commit and it looks awesome 🎉

Few comments came to mind, I'll leave few suggestions (even though the code is already merged) if I find some time later today.

[joshday]

Please do. I haven't tagged a new release yet.

[brucala]

Left few suggestions. Things are already looking great, so feel free to ignore them

[brucala]

awesome! Makes sense to leave out the GroupBy suggestion... i liked the idea but it was a bit convoluted.

Did you push the changes?

[joshday]

Oops, they're pushed now