bikaylee/pixel-jump

Deep Reinforcement Project in Malmo - UCI CS175 Fall 2020

CS175 Fall 2020

Pixel Jump

Pixel Jump is a gameplay simulation in Minecraft that the agent will perform physical jumps from one platform to another with user-control jump simulation based on projectile motion in 3D. The purpose of this project is to let the agent observe his surrounding platforms and then pick an initial velocity from a continuous action space to ensure that the agent can land on various positions of the next platform and hopefully jump onto the glass (goal) block for greater reward.

---

Our agent James is trapped by a swamp of lava. In order to escape, he must jump through a variety of platforms without falling to the lava. Our job is to improve his jumping skills by training him in his “mind palace” because he only lives once.

Website

https://bikaylee.github.io/Pixel-Jump/.

Basics

• Classical Mechanics
• Deep Reinforcement Learning
• Proximal Policy Optimization

Level of Difficulty

all maps are randomly generated for each mission

1. No degree, complete 3x3 platform, glass always centered
   
2. No degree, complete 3x3 platform, glass randomly at x = 1.5
   
3. Restricted degree, complete 3x3 platform, glass randomly at any block
   
4. Restricted degree, incomplete 3x3 platform, glass may randomly be at any block
   
5. Wider degree, incomplete 3x3 platform, glass may randomly be at any block
   

Reward System

Glass Block of Platform: +100
Other Block Type of Platform: < 90, based on the relative distance to glass block
Lava: < -10, based on the relative distance to glass block

Observation Space

Layers that store information about the next platform with one layer containing all available blocks and the other containing only glass blocks. Also, information of the current platforms or other platforms besides the next platform are excluded.

First Layer: 5 x 10 storing all available blocks information and air
Second Layer: 5 x 10 storing only glass block information and air

Game Demo GIF

References

Malmo

• Mission Schemas
• Malmo XML Schema Documentation

Physics

• Physics: Projectile Motion in 2D
• Physics: Projectile Motion in 3D
• Verifying proper displacement range for particular velocity

Machine Learning Algorithm

• RLlib
• RLlib Pytorch Models
• PPO OpenAI
• PPO algorithm from Kolby Nottingham

Tools

• Draw.io
• Jupyter Notebook
• LaTex Overleaf
• Table Generator
• Linear Regression