Store Token Transfers in DB during Indexing

[acravenho]

Store Token Transfers in DB during Indexing

[KronicDeth]

Unlike the other worker issues, this is NOT covered by indexer in #162.

[KronicDeth]

@acravenho I can't find getPastEvents in https://github.com/ethereum/web3.js to determine which JSONRPC method that is using. Is it just a way of interpreting getPastLogs's data, in which case the logs table has the data already, or is it different?

[acravenho]

@KronicDeth We are already storing all the information we need to obtain the token transfer information.

Here is an example of a token transfer: https://etherscan.io/tx/0x3f51128311e0000d108ae83b1a5cc470d404fffe5d6eade307897ed6cf90bb25#eventlog

The first argument: 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef stands for a token transfer. This is always the same.

The second argument: 0x0000000000000000000000009a4a90e2732f3fa4087b0bb4bf85c76d14833df1 is the from address

The third argument: 0x000000000000000000000000f0967af4b05014ddd3add8ef39848a06aa502fff is the to address.

The data field is the hex value of the amount of the token transfer.

In the parent transaction, the to field is the contract we are interacting with. If we do not have information on this token, we need to query the ABI to obtain the total supply, token name, token symbol and decimals, then store the information for future transfers from that token contract.

Here is how I am indexing token transfers. As we are indexing, if we see an event beginning with 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef, we should check the DB for token info. If the token info does not exist, we query the ABI to fetch the token info. Once we have the token info we store the transfer to, from, amount, and block.

[KronicDeth]

Why does 0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef mean token transfer? Where is that defined? Is it the call encoding of a function everyone uses for token transfers? I'd prefer to be able to link to so other projects source or a white paper saying that value means token transfer than have a magic constant.

So, we do need separate token table to store total supply, token name, token symbol and decimals and it probably needs to work with @alexgaribay's market information for converting to USD.

[acravenho]

The first topic in the array is the sha3 hash of the hexadecimal representation of the canonical signature of the event. The remaining items in the topics array (there may be up to three) are the hexadecimal values of the indexed parameters to the event. The event's remaining parameters (non-indexed) are stored in the 'data' field of the log.

For example, the ERC20 token Transfer event, which is defined as

Transfer(address indexed _from, address indexed _to, uint256 _value)

https://ethereum.stackexchange.com/questions/12553/understanding-logs-and-log-blooms

[KronicDeth]

Are the topics always sha3 hashes and 32-bytes long when present? If so, we'll need to change the column type from :string to :bytea and the schema type to Explorer.Chain.Hash.Full. If only the first_topic is guaranteed to be a sha3 hash, then that one should be changed and we'll have to leave the others as :string.

[alexgaribay]

@acravenho What information do we want to gather about the token? I'm guessing at least name and symbol?

[acravenho]

@alexgaribay We want to capture the following:

• name
• symbol
• total supply
• decimals
• owner
• contract address

[alexgaribay]

@acravenho We already cache some information about tokens by hitting CoinMarketCap's API. total_supply is an available field but we don't currently hold it. Would you rather that value be stored from the API or in a table holding the attributes you listed?

[acravenho]

@alexgaribay I'm not sure the best place to hold them. I believe CMC is only storing just a small number of the total 20k+ tokens. It would seem that creating a table might be a better way to go.

[alexgaribay]

I think I may have made that unclear. There will be tokens table that holds some information needed for token transfers. Additionally, the there will also be some cached information about tokens related to exchange rates and supply. total_supply can come from two sources; the contract call from web3 or from CMC. Seeing as how total supply can change, at least with POA, it makes more sense to me to not store that value in the new tokens table and add token_supply as a field we capture from CMC.

[acravenho]

I think we should be reading the smart contract. This is the most accurate data possible. The POA ERC20 token contract is an edge case because it updates based on the burn and minting of tokens when someone uses the bridge.

The idea behind this table would be to populate a page like this from Etherscan: https://etherscan.io/token/0xc528c28fec0a90c083328bc45f587ee215760a0f