shibihe/model-free-episodic-control

This is the implementation of paper Model Free Episodic Control

model-free-episodic-control's Issues

Speeding up Episodic control

Hey! really great work with the episodic control agent. I put it to run on my Nvidia tesla and left it overnight. The next morning only 7 epochs were done. Is there any way we could speed it up?

I ran a profiler for 2 epochs, these are the results:
Ordered by: internal time

ncalls	tottime	percall	cumtime	percall	filename:lineno(function)
577059602	1077.328	0.000	1077.328	0.000	{method 'reduce' of 'numpy.ufunc' objects}
149385847	1067.267	0.000	2702.390	0.000	numeric.py:2340(within_tol)
149385847	633.005	0.000	4695.321	0.000	numeric.py:2281(isclose)
298771703	422.120	0.000	921.010	0.000	numeric.py:2478(seterr)
448157543	345.168	0.000	1792.992	0.000	fromnumeric.py:1968(all)
298771703	303.658	0.000	333.810	0.000	numeric.py:2578(geterr)
133794	236.010	0.002	5984.444	0.045	EC_functions.py:48(estimate)
128898286	229.696	0.000	654.351	0.000	EC_functions.py:76(_calc_distance)
448161306	222.306	0.000	272.530	0.000	numeric.py:476(asanyarray)
896398980	218.985	0.000	219.419	0.000	{numpy.core.multiarray.array}
448157556	199.122	0.000	1175.297	0.000	{method 'all' of 'numpy.ndarray' objects}
149385847	194.654	0.000	5588.565	0.000	numeric.py:2216(allclose)
149385847	171.426	0.000	235.043	0.000	numeric.py:1970(isscalar)
298772806	158.011	0.000	158.011	0.000	{abs}
448157556	136.889	0.000	976.175	0.000	_methods.py:40(_all)
298771704	116.438	0.000	116.438	0.000	{numpy.core.umath.seterrobj}
128898286	110.123	0.000	424.655	0.000	fromnumeric.py:1737(sum)
149385851	106.513	0.000	522.222	0.000	numeric.py:2874(exit)
278396649/278396642	105.359	0.000	105.361	0.000	{isinstance}
149385851	99.092	0.000	604.393	0.000	numeric.py:2869(enter)
149385853	97.069	0.000	97.069	0.000	{numpy.core.multiarray.result_type}
149385851	94.841	0.000	115.454	0.000	numeric.py:2865(init)
597543408	78.794	0.000	78.794	0.000	{numpy.core.umath.geterrobj}
128898286	65.159	0.000	65.159	0.000	EC_functions.py:16(init)
118009	51.381	0.000	65.375	0.001	{_heapq.nsmallest}
128898287	34.789	0.000	272.828	0.000	_methods.py:31(_sum)
149387544	20.613	0.000	20.613	0.000	{method 'pop' of 'dict' objects}
80842	15.486	0.000	15.486	0.000	ale_python_interface.py:140(act)
128898286	13.995	0.000	13.995	0.000	EC_functions.py:65()
20000	10.066	0.001	656.611	0.033	EC_functions.py:81(update)
19888	9.805	0.000	5994.359	0.301	EC_agent.py:95(_choose_action)
129045706	9.425	0.000	9.425	0.000	{method 'append' of 'list' objects}

How about using sklearn's Knn instead?

VAE implementation

Hello! Have you implemented the VAE as deepmind paper settings?

Add what changes were made to the README

In the readme, you mention that you made some changes so that opencv wasn't needed anymore -- it would be great to get a short list of them!

Please add a license

Hi. A year ago or so I forked your model-free-episodic-control repository. I refactored, deleted and added a lot of code. I published it on my github-account under the MIT-license and mention you as the original author. At that time I never thought about the fact that you do not include a license. Now I ask you to add a license to your project (preferably MIT or GPL v3) not just for me but also for all other people who may would like to use your code but are afraid because your have no license. Thank you!