Question: How to add table from DB SQL like postgresql that data can query live data

[dask-contrib]

Question: How to add table from DB SQL like postgresql that data can query live data

[nils-braun]

That is a very good question and you nearly summarized the situation correctly. However, for dask dataframes (not for pandas dataframes), there is a small difference: the data is not "created" (or cached) just once, but actually, every time you run a calculation (if you do not persist or cache it on purpose). The reason is, that dask (in contrast to pandas), does not store any "real" data but only the computations which should be applied to the data (plus some metadata). Onle if you trigger it (e.g. with compute()), the data is fetched.

This means for dask-sql, that if you run

df = ... some dask method, e.g. read_sql ...
c.create_table("table", df, persist=False)  # important: persist is True by default, but you do not want that here
c.sql("SELECT * FROM table").compute()

the data is only fetched the moment you run the .compute().

[nils-braun]

As I did not hear back from you, I assume the question is solved.