*help* I am confused about why this function just using the catche[0]?

[facebookresearch]

*help* I am confused about why this function just using the catche[0]?

[myleott]

The cache is structured as a queue, where elements in the queue come from different workers. For example, if you had 3 workers then at initialization it might look like:

cache:
  0: first_result_generated_by_worker_0      <-- top of queue
  1: first_result_generated_by_worker_1
  2: first_result_generated_by_worker_2
  3: second_result_generated_by_worker_0
  4: second_result_generated_by_worker_1
  5: second_result_generated_by_worker_2
  (...)

Every time we consume a result from the cache, we remove it (cache.pop(0)) and enqueue another batch on the relevant worker. This keeps the cache a fixed size and ensures that all workers are busy.