[3](5) Map out airflow tasks

[stellar]

[3](5) Map out airflow tasks

[Isaiah-Turner]

We also will need to determine if we can create an airflow tasks that will export all ledger entries to BigQuery even if they are all in the same file, or if we will need to separate out changes by type/batch.

[Isaiah-Turner]

DAG organization

1. History archive DAG
   1. exports ledgers, txs, ops
   2. Load exported files into Google Cloud Storage
   3. Load GCS files into BigQuery (append new rows)
2. Long Running Core DAG
   1. Exports accounts, trustlines, offers
   2. All in single command; long running stellar-core instance exports incrementally
3. Changes Export DAG
   1. File sensors look at the location where ledger entry changes are exported in batches
   2. As new files are created, they are uploaded to BigQuery and deleted from local storage
   3. Files are used up update BigQuery tables (can create new rows, update existing rows, or delete rows)
4. Catchup DAG
   1. Exports accounts, Trustlines, offers
   2. Everything in separate commands; uses bucket list to catchup