BUG: IndexError with pandas.DataFrame.cumsum where dtype=timedelta64[ns]

[omor1]

BUG: IndexError with pandas.DataFrame.cumsum where dtype=timedelta64[ns]

[pmhatre1]

The code seems to be taking an single list instead of numpy array below

[pmhatre1]

original_list = func(y)
result = [[x] for x in original_list]
result = np.array(result)

This might help?

[Aloqeely]

The problem exists in core\array_algos\datetimelike_accumulations.py, _cum_func:

result = func(y)
result[mask] = iNaT

Essentially, what this is doing is performing func (np.cumsum in this case) on your whole data array, and by default np.cumsum computes the sum over a flattened array and returns it, but then when masking the NA values, it is expecting that the data array was in it's original shape and not flattened.

To fix this, you have to pass axis as 0 to func (again, np.cumsum in this case), so that it performs the cumsum along the given axis and not over the whole flattened array (it's the only func with axis=None by default, unlike cummin and cummax which have axis=0 by default and work properly)

I'd like to try fixing this issue, but it would be helpful to understand why the developers chose not to pass axis to func and instead handle its behavior internally.

And to explain why your other examples worked, in the first one, it did not use _cum_func function, as the values were converted to a timedelta64[ns] after computing the cumsum, but in the second and third examples, it was handled well and only 1 column at a time was passed to _cum_func (I couldn't trace back to how that was happening)

[pmhatre1]

@phofl @mroeschke any comment on this based on #41720. I found result is 1D while mask is an array. Thus I tried

flattened_mask = mask.flatten()
result[flattened_mask] = iNaT

but ended up in error at

block = self.blocks[self.blknos[i]]
values = block.iget(self.blklocs[i])
bp = BlockPlacement(slice(0, len(values)))

    in managers.py

[Aloqeely]

take

[pmhatre1]

Thanks @Aloqeely for helping me understand this.