gtp_mex implement LRU cache for input tensor indexing

[cankav]

gtp_mex implement LRU cache for input tensor indexing

[cankav]

Tested lru and random cache, both of them become the bottleneck in normal operation.

Considered sparse cache data structure. But it does not work because missing elements of the sparse data structure may be missing or not cached yet. Thus searching for missing elements each time they are accessed defeats the purpose of the cache mechanism. This is equivalent to having a cache mechanism only for the existing elements of the sparse data structure, which is pointless.

Need to find a feasible cache approach or abondon cache idea. It feels like this effort is overkill. Working on map-reduce implementation would be more meaningful. If you need cache, use a system which can accomodate your memory needs!