Add snapshotting to allow comparison between points and time.

[matsonj]

Add snapshotting to allow comparison between points and time.

[matsonj]

I think I can do this a slightly coarser grain - which is weekly for football sims and daily for nba. The basic way to do this will be hold some folders in s3, and then based on when the most recent actual is recorded, we can use write the new results into a folder.

something like s3://some_bucket/sim_version/year/week/model.parquet, or for the nfl model it would be currently this path: s3://some_bucket/nfl/2023/0/nfl_reg_season_summary.parquet

[matsonj]

Pipeline then basically has a final step of dropping new files into S3, then building a duckdb file that can hit the entire directory. A thin copy of the parquet files. Then evidence hits that duckdb file instead of the local file.

[matsonj]

related to this conversation here: #102

[matsonj]

Actually this is a dupe of #102 and will be solved as part of that.