AnalyticsDB atomic operations

[DMWMBot]

AnalyticsDB atomic operations

[vkuznet]

valya: Related, how we should handle swap of data when existing data are still in cache. There are two scenarios:

1. new data is identical to one in a cache, action: change timestamp
2. new data is different from one in a cache.

This issue can be critical for expensive data requests, get all blocks/datasets.

[vkuznet]

valya: Hi Gordon,
instead of email I put it to trac. Somehow I don't get email updates, so we need to include CC for tickets.

I thought about atomic update of newly retrieved records from DAS robot with existing records presented in db. Here is the proposal:

• add new field das.ahead for all records
  • if query hash not found in DB, das.ahead=0 and all records just added to DB otherwise
    add new field das.ahead=1 to all records and insert them as usual
  • at this step we will have 2 copy of data, one with das.ahead=0 and visible to users
    and another with das.ahead=1 which can be used by das web to not show such records
• once we ready to perform a swap
  1. perform records removal for the same query hash && das.ahead=0
  2. perform atomic update (find in mongo docs) of records with given query hash and das.ahead=1
     • change das.ahead to 0 during this operation.

This will require to add das.ahead field and also some manipulation in mongocache code for both DAS cache and DAS merge collections.

Please let me know what do you think,
V.

[vkuznet]

valya: I don't think this use case is valid anymore, therefore closing the ticket. Will re-open if use case will show up again.