Support Series generation with serial dependence

[NowanIlfideme]

Support Series generation with serial dependence

[cosmicBboy]

Hi @NowanIlfideme

Pandera strategies are currently quite limited, as you've experienced. The limitation is sort of bounded by the fact that it's leveraging the hypothesis data_frames API: https://hypothesis.readthedocs.io/en/latest/numpy.html#hypothesis.extra.pandas.data_frames. Basically, you need to specify columns and their elements, each of which are drawn from a strategy that generates a scalar.

Ideally, I would like the ability to generate a whole series with a custom function, or at least with the hs.builds function.

Yes, so #561 is the issue for improving this in pandera, I just haven't had the time to work on this because it'll pretty much involve a re-write of the pandas_strategy module.

I consider this issue, #1220, and #1275 to be problems to be addressed by the re-write (#1275 sounds pretty hard to implement tho, I'd maybe keep that out of the design and rely on docs/recipes on how to generate strategies with a fixed column based on the data generated from another strategy).

If you have to time/capacity, would you be able to chime in on #561 with a high-level set of requirements and (ideally) a code sketch of how this might be implemented in pandera? It would involve departing from hypothesis.extra.pandas.data_frames altogther.

From my understanding, we want:

• Strategies that work for all pandera schemas (this is a really high bar, but I think possible), with reasonable escape hatches when pandera cannot automatically figure out how to generate a df.
• Generating entire columns instead of individual elements
• Incorporating cross-column dependencies
• A user-friendly way of overriding strategies (from pre-existing Checks) or custom strategies
• Columns with multiple checks should not chain strategies with filter, it should maybe override data with the new constraint.
• others I might have missed?