jamiew/helium-etl-queries

A collection of SQL views used to enrich data produced by a Helium blockchain-etl

helium-etl-queries's Issues

challenge_receipts_parsed does not work from scratch

Currently it expects a value for MAX(block) from challenge_receipts_parsed, so it only loads new data. But if there is no data at all, it loads nothing. Should add a conditional or similar

Add indexes to transctions_payment

How is the Helium API doing efficient payment txn lookups? Our current setup is extremely slow

Specifically like to index payer and payee

Might require turning this into a materialized view and adding to the cronjobs, but I assume blockchain-http has some secrets

Switch to using dbt and dbt-metabase

Use dbt pipelines to handle transormations and view creation more reliably and sanely.

Then use dbt-metabase to sync the dbt data model w/ Metabase (!)

Will make it a lot easier to test, easier to automate, and make it easier for others to contribute new tables and views

Merge this repo with `dewi-alliance/dewi-etl`

https://github.com/dewi-alliance/dewi-etl started as issue tracker for the server, but maybe merge together? Or use submodules/subtrees

It's a package deal at this point really

Same with https://github.com/jamiew/helium-etl-maintenance and some of @danksu's helper apps like go-etl-shenanigans

Add witness_address index to challenge_receipts_parsed

Weird that transmitter has indexes on both address and name, but witness is only on name

OR make both of these only use address... shave some space off a large index

Name not unique so shouldn't really be an index at all