wietsewind/fetch-xrpl-transactions

Fetch (and store) all XRPL transactions (to a custom datastore)

fetch-xrpl-transactions's Issues

DeliveredAmount DEX

Outcome of OfferCreate

When an OfferCreate is successful, it would be great to see the outcome (wallet balance changes). Now we have no way of querying traded amounts.

DeliveredAmount field is empty for older transactions

Apparently only more recent transactions have the DeliveredAmount field, older transactions use the delivered_amount field.

Required: insert-fix + restart backfilling.

Move transactions data to partitioned table

The way the ledger and transaction tables are laid out, virtually all queries are going to be very expensive because of the table scans. If you were to merge the ripple time, correct CloseTime, and the remainder of the transaction table into a "partitioned table" (and dropping the original transaction table), it would then be possible to search by date in a near-index-like efficiency, radically reducing the cost of querying when dates are involved.

Possible shortest/cheapest path to accomplish this:
https://stackoverflow.com/questions/38993877/migrating-from-non-partitioned-to-partitioned-tables/43130999#43130999

Duplicates

I have found some duplicates in the BigQuery dataset (transactions with the same hash), example:

  SELECT
    *
  FROM
    xrpledgerdata.fullhistory.transactions
  WHERE
    `hash` = "93DA464BB51C01563E6848CFF482A6C84029AF633E711FD881772957A998994B"

Maybe a cleanup routine could be made to avoid these (or a constraint on the table, if possible in BQ)?

Memo's aren't stored correctly

in BigQuery(xrpledgerdata.fullhistory.transactions) next information about column memo (type: REPEATED RECORD) for "r3i3CHq2WNR2H81ZdY8M9ykCnW8uN9pBJb" (XRP adress) :

But values from bithomp & developers.ripple.com (recoded from HEX to ASCII) is different with BigQuery

I encountered differences when building distribution of clients (bithomp wallet, ripple client desktop etc.). Maybe you know, what can it be.
adress: r3i3CHq2WNR2H81ZdY8M9ykCnW8uN9pBJb
transaction: 6BCE5DECEC550A9F890D70109AAECB8280AD71C0061FB383B93D32B3D5C31202
Best regards,
Marat

No data since 2020-12-23

It seems that the transactions table stopped updating since 12/23. Are there any plans to fix this? Thanks!

Small dataset for testing

Would it be possible to make a small dataset, such as an hour/day's worth of ledgers available via direct download to enable testing before taking the plunge and signing up for a 'bigquery' account.