justjoshtings/ms.pacman.ai

Trained a reinforcement learning agent to play the Atari 2600 game of Ms. Pac-Man. Built a web-app to live stream gameplay with TCP/IP in real-time with Flask as the app interface.

Data Scientist and Data Engineer

Leveraging my background in tech, clean energy engineering, and data science/machine learning to drive solutions in climate and sustainability.

Learn more about me and my work at my personal website.

ms.pacman.ai's Issues

You can do what you want with the score here :)

ms.pacman.ai/web_app/app.py

Lines 63 to 65 in c54a6cb

	def do_something_with_score(score):
	print('SCORE!!',score)
	pass

@Saharae

Reviewed GPU Final Script

Hey @adamkritz, just looked at the GPU final script and thanks for doing all that stuff, it looks good!

Up to you but feel free to remove this random model part if you want as that's not needed for training.

ms.pacman.ai/GPU script/Final GPU Script.py

Lines 62 to 107 in 112aff8

	print('Begin Random Model')
	skip = False
	if not skip:
	# Run for 1000 games
	episodes = 1000
	random_model_scores = []
	random_model_steps = []
	for episode in range(1, episodes+1):
	state = env.reset()
	done = False
	score = 0
	steps = 0
	while not done:
	action = random.choice([0,1,2,3,4,5,6,7,8])
	n_state, reward, done, info = env.step(action)
	score += reward
	steps += 1
	print('Episode:{} Score:{} Steps:{}'.format(episode, score, steps))
	# env.play()
	random_model_scores.append(score)
	random_model_steps.append(steps)
	env.close()
	print('random model finished playing')
	# Make scores results directory;
	random_model_scores_file = abspath_curr + '/results/random_model_scores_1k_episodes_3_9_22.txt'
	random_model_steps_file = abspath_curr + '/results/random_model_steps_1k_episodes_3_9_22.txt'
	print('Made Random Model Text Files')
	# Save random_model_scores
	random_model_scores = np.array(random_model_scores)
	random_model_steps = np.array(random_model_steps)
	np.savetxt(random_model_scores_file, random_model_scores, fmt='%d')
	np.savetxt(random_model_steps_file, random_model_steps, fmt='%d')
	print('Saved Random Model Scores')

Also don't forget to set nb_steps to something like 600k for the 1mill. This will give it sufficient number of steps to got from 1.0 eps to 0.1 eps.

ms.pacman.ai/GPU script/Final GPU Script.py

Line 196 in 112aff8

policy = LinearAnnealedPolicy(EpsGreedyQPolicy(), attr='eps', value_max=1., value_min=.1, value_test=.1, nb_steps=8000) # For 1 million total steps, I think having the policy nb_steps around 600k is a good slope.

Also, let's try memory limit to be 500k cause why not lol.

ms.pacman.ai/GPU script/Final GPU Script.py

Line 202 in 112aff8

memory = SequentialMemory(limit=100000, window_length=window_length)