[BUG] The `cdf` for the `Empirical` distribution returns a dataframe of objects

[sktime]

[BUG] The `cdf` for the `Empirical` distribution returns a dataframe of objects

[Alex-JG3]

was starting a new job so busy with admin, moving, etc.

Good luck with your new job!

Would you like to fix this, or should I?

Yes I am happy to fix this.

[Alex-JG3]

@fkiraly, tagging you incase you have not seen this.

It looks like the pandas' convert_dtypes function might be the problem.

https://github.com/sktime/skpro/blob/main/skpro/distributions/empirical.py#L129

Here is an example that is similar to the for loop here https://github.com/sktime/skpro/blob/main/skpro/distributions/empirical.py#L118.

>>> import pandas as pd
>>> df = pd.DataFrame(index=[1, 2, 3], columns=['a', 'b'])
>>> df.dtypes
a    object
b    object
dtype: object
>>> df.loc[1, 'a'] = 1.5
>>> df.loc[2, 'b'] = 1.6
>>> df
     a    b
1  1.5  NaN
2  NaN  1.6
3  NaN  NaN
>>> df.dtypes
a    object
b    object
dtype: object
>>> df_conv = df.convert_dtypes()
>>> df_conv.dtypes
a    Float64
b    Float64
dtype: object
>>> df_conv.values
array([[1.5, <NA>],
       [<NA>, 1.6],
       [<NA>, <NA>]], dtype=object)

So using convert_dtypes on a dataframe containing objects seems to give some unexpected datatypes. The resulting numpy array df_conv.values also has type object here which makes it difficult to work with.

Avoiding convert_dtypes might solve the issue.

[fkiraly]

Yes, I did not see this - was starting a new job so busy with admin, moving, etc. Also you opened this first in skbase, by accident (all fixed).

Would you like to fix this, or should I? If you are already working on tests, might be more synergistic if you do.

I'm busy with making the refactored version of the package "feature complete" for the presentation in mid-Sep and interoperable with sktime.

[fkiraly]

Yes I am happy to fix this.

thx!