agnar22/cartpole

A q-learning approach to the cartpole environment.

Cartpole

Reinforcement learning solution to the Cartpole environment from OpenAI gym.

This agent uses a Q-learning approach to learn how to balance the pole by moving the cart, given the horizontal position- and velocity of the cart in addition to the angle and angle velocity of the pole.

Installation

Clone the repository

git clone https://github.com/Agnar22/Cartpole.git

navigate into the project folder

cd Cartpole

install requirements

pip install -r requirements.txt

if everything went well, you should now be able to run the code

python3 Main.py

Motivation

This project served several purposes. I wanted to try out the entry level RL problem from OpenAI gym, Cartpole, as well as learn how to implement the suggested solution from scratch. Being familiar with Q-learning also has other advantages due to the applicability of this algorithm in the RL world; for instance, it may be applied to the other "Classic controll" problems from Open AI gym because it is a model-free algorithm that is able to handle stochastic transitions and rewards. Additionally, it is a precursor of Deep Q-networks, thus it is fundamental to have a certain understanding of Q-learning before moving onwards to DQN.

Results

The problem is easily solvable with q-learning, as demonstrated above (episodes from various parts of the training are shown). It needed about 3000 episodes to reach the final fitness where it perfectly balances the pole in the middle of the screen (wait until the end of the gif).

Other resources

• In this article from freecodecamp, Thomas Simonini explains Q-learning from scratch.
• I also recommend reading Metthew Chans medium article on how to solve this problem with Q-learning.