null blocks

[decred]

null blocks

[chappjc]

There are a couple of things that are likely to be causing this, and most of them need to be fixed to be sure.

First, I have several processes running on the public web server, including two versions of dcrdata that fight for resources, so it seems to be common when mempool is going crazy with ticket purchases to have this sort of thing happen (same block you pointed to):

2017-07-01 04:37:14.221 [INF] SKDB: Connected block 146881 to stake DB.
2017-07-01 04:37:14.311 [DBG] MEMP: Vote in new block triggering mempool data collection
2017-07-01 04:37:15.693 [INF] DCRD: Block height 146881 connected
2017-07-01 04:37:33.936 [DBG] MEMP: mempoolDataCollector.Collect() completed in 19.591016827s
2017-07-01 04:37:35.698 [ERR] DCRD: Block data collection TIMEOUT after 20 seconds.
2017-07-01 04:37:36.120 [DBG] DCRD: blockDataCollector.Collect() completed in 20.42313802s

The blocks before and after that one did not take nearly that long, so it looks like having the mempool data collection even be able to run first is a bad idea since it can be an order of magnitude slower with several hundred tickets in mempool, and having two dcrdata instances hammering the same dcrd is worse.

Second, and related to the first actually, is if blocks roll in quickly, requests pile up and because of some limitations in the dcrd RPCs, it was a bit hard to collect data for a specific block rather than the latest block. So it's easy to completely miss a block (and overload dcrd). The latest PR, merged June 30 was intended to address this to some extent.

These probably don't explain why your block 146443 is null though.

Third, reorganizations are only partly handled. 😢 I've worked a lot on getting it handled enough not to crash dcrdata, but I still need to it to fix up new blocks added when switching a side chain to become the new main chain. It's handled in the internal stake database (the ffldb_stake folder) but the sqlite db needs to be fixed up still. This isn't terribly hard, but I've prioritized other stuff.

[chappjc]

Just submitted PR #55 that should address "third" in my post above.

[chappjc]

I also snuck in commit 0cc7437 to address "first" in the above post. It's a band-aid, but it should help for now.

[chappjc]

Merged #55, so I think this can be closed.

After syncing master, be sure to glide install again before build, then rm -r ffldb_stake dcrdata.sqlt.db to start fresh.