ntasfi/pygame-learning-environment

PyGame Learning Environment (PLE) -- Reinforcement Learning Environment in Python.

pygame-learning-environment's Issues

Where's percent_round_int in utils dir?

from .catcher import Catcher

File "/home/vitob/git_projects/PyGame-Learning-Environment/ple/games/catcher.py", line 3, in
from utils import percent_round_int
ImportError: cannot import name 'percent_round_int'

I can only see vec2d.py in the dir ple/games/utils.

Run without creating pygame window?

I know PLE has an argument to run without updating the screen, but the screen window is created regardless, which seems odd. I'm trying to run simulations on headless servers, but because of this bug I can't. Is there a work-around that I'm missing?

bug in flappy bird game

position of the keys in state (where state = p.getGameState() and is a dictionary of key value pairs ) changes every time the game is re run

lets us say that in one instance player_vel is the first key as shown in the figure below

but in another run this changes

how one is supposed to train an reinforcement learning algorithm with this type of observation inputs

pygame begginer game

on hit it takes wrong ants position
Here's the code:

import pygame
pygame.init()
clock = pygame.time.Clock()
win = pygame.display.set_mode((1000, 500))
y = 250

class Ant:
def init(self, x, side, colour):
self.x = x
self.width = 20
self.side = side
self.left = False
self.right = True
self.vel = 5
self.counter1 = 0
self.colour = colour
self.dir = dir
self.moving = True
self.counter = 0

def start(self):
    if self.side == "Left":
        self.right = True
    else:
        self.left = True
    self.counter1 = 1

def side1(self):
    if self.x < 500:
        return "left"
    else:
        return "right"

def site(self):
    if self.left:
        return "left"
    else:
        return "right"

def move(self):
    if self.moving:
        if self.counter1 == 0:
            self.start()
        if self.left:
            self.right = False
            self.x -= self.vel
        else:
            self.left = False
            self.x += self.vel

def draw(self, win):
    self.move()
    pygame.draw.circle(win, self.colour, (self.x, 250), 10)

def wall(self):
    if self.x + self.width >= 1010:
        self.moving = False
    elif self.x <= 10:
        self.moving = False

def updateWin(win):
win.blit(pygame.display.set_mode((1000, 500)), (0, 0))
text(ant1, ant2, ant3)
ant1.draw(win)
ant2.draw(win)
ant3.draw(win)
pygame.draw.rect(win, (139, 69, 19), (0, 260, 1000, 10))
pygame.display.update()

def hit(ants):
for ant4 in ants:
if ant4.moving:
for ant5 in ants:
if ant4.x < ant5.x:
if ant4.x + ant4.width >= ant5.x:
ant4.right = False
ant4.left = True
ant5.left = False
ant5.right = True
ant4.counter += 1
ant5.counter += 1

def text(ant1, ant2, ant3):
font1 = pygame.font.SysFont("comicsans", 50)
text1 = font1.render(str(ant1.counter), 1, (255, 255, 255))
text2 = font1.render(str(ant2.counter), 1, (255, 255, 255))
text2 = font1.render(str(ant3.counter), 1, (255, 255, 255))
win.blit(text1, (ant1.x, 200))
win.blit(text2, (ant2.x, 200))
win.blit(text1, (ant3.x, 200))

ant1 = Ant(40, "Left", (255, 0, 0))
ant2 = Ant(960, "Right", (100, 255, 255))
ant3 = Ant(200, "Left", (0, 200, 0))
run = True
while run:
clock.tick(15)
keys = pygame.key.get_pressed()
for event in pygame.event.get():
if event.type == pygame.QUIT:
run = False
ants = (ant1, ant2, ant3)
hit(ants)
ant1.site(); ant1.wall()
ant2.site(); ant2.wall()
ant3.site(); ant3.wall()
updateWin(win)
pygame.quit()

Monster Kong always returning same number of lives

When I run the .lives() function from a Monster Kong game it always returns 0 (on a reset game, on an initialized game, and after getting hit by a fireball). When I trained on Catcher and some of the other games, I ended episodes after a life was lost but couldn't do that for Monster Kong since I couldn't tell how many lives are left. As a work around I modified the starting lives to 1 instead of 3.

Flappy Bird Game: Check whether the bird is in the tube in the hit condition

I am using the flappy bird game to develop reinforcement learning algorithm. I found that the bird fails at some conditions when it should not fail. So I looked into the code and found that a condition is missing when determining whether the bird hit the tube. Please see my modified code here:

for p in self.pipe_group:
    hit = pygame.sprite.spritecollide(
        self.player, self.pipe_group, False)
    for h in hit:
        is_in_pipe = ((p.x - p.width / 2) <= self.player.pos_x < (p.x - p.width / 2))
        # do check to see if its within the gap.
        top_pipe_check = (
            (self.player.pos_y - self.player.height / 2) <= h.gap_start) and is_in_pipe
        bot_pipe_check = (
            (self.player.pos_y +
             self.player.height) > h.gap_start +
            self.pipe_gap) and is_in_pipe

display screen too small

screen size is too small what todo ??!

Initial display screen

Hi,

Thanks for a great library!

I had a question about the display_screen flag. Even when the flag is turned off, I see a black pygame window (see image attached) pop up. Is there a way to disable it?

Thanks!

Python3 support?

Wanting to use this alongside openai gym, but it doesn't seem to support python3. After installation, importing ple yields the error:
"No module named 'games'"
I'm assuming this is due to the import differences between the two versions; is this a known issue? Is python3 support planned?

NameError: name 'myAgentHere' is not defined

from ple.games.flappybird import FlappyBird
from ple import PLE

game = FlappyBird()
p = PLE(game, fps=30, display_screen=True)
agent = myAgentHere(allowed_actions=p.getActionSet())

p.init()
reward = 0.0

for i in range(nb_frames):
if p.game_over():
p.reset_game()

observation = p.getScreenRGB()
action = agent.pickAction(reward, observation)
reward = p.act(action)
On line 7 it say does not work. I don't understand the error. I think the problem it could be it's running on a window pc

in flappy bird game action takes a value of 119

p.act(action)

action takes a value of 119 to push the bird upwards
119 is ASCII number for w which is used in games for upward/forward movement.

but since this game has only two possible action it would be better if it is 0/1

Game

Error raycast.py

The main of raycast.py has a small error, namely that it's not passing any "actions" parameter:

TypeError: __init__() takes exactly 12 arguments (11 given)

which refers to this call:

rc = RayCastPlayer(
        map_grid,
        init_pos,
        init_dir,
        width,
        height,
        resolution,
        move_speed,
        turn_speed,
        plane,
        block_types
    )

How to slow down the execution?

I ran the NaiveAgent in Flappy Bird and noticed that the game runs very fast. Is there a way to slow this down in order to actually see what is happening?

PS: I tried decreasing the FPS, even the allowed_fps attribute of the game, but it did not help

Flappy bird can bypass pipe over screen

It really messed up my research with DQN RL variations, because it is possible to bypass pipe over the screen.

See screenshot: http://www.evernote.com/l/AClrdjOMAFBLSLm1-ZvYqa-63AOtriZ9NoI/
This state gives reward = 1.0 (notice bird partially over pipe)

player height 24, but it is enough to be at position -20 to bypass pipe.

Can you please give me rights to push patch to repo? I am planning to work with your repo and I could find more bugs.

Changes needed:

--- a/ple/games/flappybird/__init__.py
+++ b/ple/games/flappybird/__init__.py
@@ -423,7 +423,7 @@ class FlappyBird(base.PyGameWrapper):
             self.lives -= 1
 
         # went above the screen
-        if self.player.pos_y < -self.player.height:
+        if self.player.pos_y < 0:
             self.lives -= 1

Something wrong happened when I run a demo

from ple import PLE
from ple.games.pong import Pong
game = Pong()
p = PLE(game, fps=30, display_screen=True, force_fps=False)
p.init()
myAgent = MyAgent(p.getActionSet())

nb_frames = 1000
reward = 0.0

for f in range(nb_frames):
	if p.game_over(): #check if the game is over
		p.reset_game()
	obs = p.getScreenRGB()
	action = myAgent.pickAction(reward, obs)
	reward = p.act(action)

This is my test code.But it did not succeed.The error is:

NameError: name 'MyAgent' is not defined
I did not find MyAgent class in PLE or Pong ,anywhere.So,how can I run the code?

Thanks for all kind humans!

Install from source using setup.py partially installs games

When attempting to use this in combination with gym-ple I notice that the installed PLE does not include assets for some games:

$ sudo python setup.py install
$ cd ../gym-ple
$ python ./gym-ple/example.py
...
pygame.error: Couldn't open /usr/local/lib/python2.7/dist-packages/ple-0.0.1-py2.7.egg/ple/games/flappybird/assets/redbird-upflap.png`
...
$ ls /usr/local/lib/python2.7/dist-packages/ple-0.0.1-py2.7.egg/ple/games/flappybird
__init__.py  __init__.pyc

can't import doom and doomish

when I installed PLE, I want to import ple but got this error

import ple
pygame 1.9.4
Hello from the pygame community. https://www.pygame.org/contribute.html
couldn't import doomish
Couldn't import doom

Hi, I would like to join

Hi Norm,

Sorry to make this an "Issue", but I did not find any other way of contacting you.

My name is Sven, I am an AI/RL enthusiast with a PhD in Computational Biology and I would like to join forces with you. I just finished the Udacity deep learning nanodegree and am quite fluent in tensorflow, NN-topologies and RL-algos. Also, I have written my own Pygame/RL library (http://www.github.com/sven1977/shine), which is not fully functional yet. It's based on tmx files, through which one can define complete MDPs (games) that can then be solved by different RL algos. It heavily uses the concept of components, which are plug-and-play objects for game agents/enemies/etc.. (e.g. Physics). However, at this point - rather than reinventing the wheel - I would like to contribute to your lib and maybe merge mine (or ideas of it) with PLE. I have also implemented "The Lost Vikings" as a Pygame plugin for my lib (I like the fact that the levels in that game are extremely reward-sparse and would be nice challenges for RL algorithms). Please let me know if you are interested.

Best, Sven

Redundancy in Dockerfile?

I am referring to there two lines here

Could you explain why did you clone your repo twice here?

Pygame or pygame-ce ?

Hello,
Do you use pygame or pygame-ce for this library ?

ImportError: No module named 'games'

OSX 10.11, Python 3.5.2
All required packages are well installed, but still get following error.

Traceback (most recent call last):
File "", line 1, in
File "/Users/Name/workspace/PyGame-Learning-Environment/ple/init.py", line 1, in
from .ple import PLE
File "/Users/Name/workspace/PyGame-Learning-Environment/ple/ple.py", line 5, in
from games import base

Running the example `keras_nonvis.py` fails

Hey @ntasfi . Thanks for putting the Dockerfile up for everybody

What I did was

$ wget http://raw.githubusercontent.com/ntasfi/PyGame-Learning-Environment/master/docker/Dockerfile
$ docker build -t prodicus/pygame .
$ docker run -it -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY prodicus/pygame /bin/bash

I get the following error inside the docker container,

root@9bacfcf2f0a4:/ple/examples# python keras_nonvis.py 
Using Theano backend.
No protocol specified
No protocol specified
No protocol specified
Fatal Python error: (pygame parachute) Segmentation Fault
Aborted (core dumped)
root@9bacfcf2f0a4:/ple/examples# 

What could be the cause of the problem?

---

Relevant information

$ docker --version
Docker version 1.11.2, build b9f10c9

$ uname -r
4.4.0-28-generic

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04 LTS
Release:    16.04
Codename:   xenial

Porting PyGames

I'm currently trying to port Chrome's T-Rex Game to PyGame Learning Environment but having a hard time of doing it. Can anyone help out?

Py game

Py game turorial

Windows platform

I could not see any windows installation, is it suitable for windows?

Thanks

player is stuck on top of ladders (Monster Kong)

# pseudocode of event handler from MonsterKong
if event.key == "up":
    player.y -= player.getSpeed() / 2
    if not player.checkCollision(ladderGroup):  # why?
         player.y += player.getSpeed() / 2

Not able to install it properly

(py36)  ➜  PyGame-Learning-Environment git:(master) sudo pip install -e .

The directory '/home/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Obtaining file:///home/PyGame-Learning-Environment
Requirement already satisfied: Pillow in /usr/lib/python2.7/dist-packages (from ple==0.0.1)
Requirement already satisfied: numpy in /usr/lib/python2.7/dist-packages (from ple==0.0.1)
Installing collected packages: ple
  Found existing installation: ple 0.0.1
    Can't uninstall 'ple'. No files were found to uninstall.
  Running setup.py develop for ple
Successfully installed ple
(py36)   PyGame-Learning-Environment git:(master) pip install gym_ple
Collecting gym_ple
  Using cached https://files.pythonhosted.org/packages/0d/d6/41e3510bf943a6383b89d1b3db785efbd2c63c55ac2947a4ba31bf9c9009/gym_ple-0.3.tar.gz
Building wheels for collected packages: gym-ple
  Running setup.py bdist_wheel for gym-ple ... done
  Stored in directory: /home/.cache/pip/wheels/48/a7/28/fd4f7105b5960ce04ddeb161679e2343a02e037ec34e992a54
Successfully built gym-ple
Installing collected packages: gym-ple
Successfully installed gym-ple-0.3
(py36)   PyGame-Learning-Environment git:(master) python
Python 3.6.7 |Anaconda, Inc.| (default, Oct 23 2018, 19:16:44) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ple
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/PyGame-Learning-Environment/ple/__init__.py", line 1, in <module>
    from .ple import PLE
  File "/home/shekharrajak/Documents/githubRepos/personal/rl_games/tools/PyGame-Learning-Environment/ple/ple.py", line 5, in <module>
    import pygame
ModuleNotFoundError: No module named 'pygame'
>>> exit()
(py36) PyGame-Learning-Environment git:(master) pip install pygame
Collecting pygame
  Downloading https://files.pythonhosted.org/packages/8e/24/ede6428359f913ed9cd1643dd5533aefeb5a2699cc95bea089de50ead586/pygame-1.9.6-cp36-cp36m-manylinux1_x86_64.whl (11.4MB)
    100% |████████████████████████████████| 11.4MB 2.5MB/s 
Installing collected packages: pygame
Successfully installed pygame-1.9.6
(py36) PyGame-Learning-Environment git:(master) python
Python 3.6.7 |Anaconda, Inc.| (default, Oct 23 2018, 19:16:44) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ple
pygame 1.9.6
Hello from the pygame community. https://www.pygame.org/contribute.html
couldn't import doomish
Couldn't import doom
>>> 

Let me know if I did something wrong.

"TabError: inconsistent use of tabs and spaces" on Python 3

Hello,
I'm trying out your code on Python 3 and have found a problem in that python 3 disallows mixing tabs and spaces for indentation. As a result the library doesn't work /w 3. For some context around this see:

http://stackoverflow.com/questions/19295519/indentation-error-with-python-3-3-when-python2-7-works-well

You can probably change all the files in bulk using sed, like in this example:

http://stackoverflow.com/questions/11094383/how-can-i-convert-tabs-to-spaces-in-every-file-of-a-directory

getting individual values from the state in flappy bird game

how to get the individual values from the dictionary

Incorrect import in example

The import line
from ple.games.raycast_maze import RaycastMaze

should read
from ple.games.raycastmaze import RaycastMaze

pygame.error: posted event keycode must be int

MacBook Air --MAC OS 10.14.6
Python3.73 64bit
Wrappered the environment:
`#Code based on https://github.com/KibaAmor/rl-code/blob/master/demos/dqn_flappybird/flappybird_wrapper.py
import numpy as np
from gym import Env, spaces
from ple import PLE
from ple.games import FlappyBird

class FlappyBirdWrapper(Env):
def init(self, **kwargs):
self.game = FlappyBird()
self.p = PLE(self.game, **kwargs)
self.action_set = self.p.getActionSet()

    # 
    self.observation_space = spaces.Discrete(3)
    # 
    self.action_space = spaces.Discrete(2)

def _get_obs(self):
    # 
    state = self.game.getGameState()
    # 
    dist_to_pipe_horz = state["next_pipe_dist_to_player"]
    dist_to_pipe_bottom = state["player_y"] - state["next_pipe_top_y"]

    velocity = state['player_vel']
    return np.array([dist_to_pipe_horz, dist_to_pipe_bottom, velocity])

def reset(self):
    self.p.reset_game()
    return self._get_obs()

def step(self, action):
    reward = self.p.act(self.action_set[action])
    obs = self._get_obs()
    done = self.p.game_over()
    return obs, reward, done, dict()

def seed(self, *args, **kwargs):
    pass

def render(self, *args, **kwargs):
    pass`

Error happened when running:
File "/Users/Sam/Desktop/MSACoures/WorkShop/RL_FlappyBird/Test.py", line 31, in <module> watch() File "/Users/Sam/Desktop/MSACoures/WorkShop/RL_FlappyBird/Test.py", line 19, in watch next_state, reward, done, _ = env.step(action) File "/Users/Sam/Desktop/MSACoures/WorkShop/RL_FlappyBird/Env.py", line 34, in step reward = self.p.act(self.action_set[action]) File "/Users/Sam/PyGame-Learning-Environment/ple/ple.py", line 376, in act return sum(self._oneStepAct(action) for i in range(self.frame_skip)) File "/Users/Sam/PyGame-Learning-Environment/ple/ple.py", line 376, in <genexpr> return sum(self._oneStepAct(action) for i in range(self.frame_skip)) File "/Users/Sam/PyGame-Learning-Environment/ple/ple.py", line 395, in _oneStepAct self._setAction(action) File "/Users/Sam/PyGame-Learning-Environment/ple/ple.py", line 411, in _setAction self.game._setAction(action, self.last_action) File "/Users/Sam/PyGame-Learning-Environment/ple/games/base/pygamewrapper.py", line 79, in _setAction pygame.event.post(ku) pygame.error: posted event keycode must be int

And I found the last action may be '[]', error happens here
if last_action is None: last_action = self.NOOP

if last_action is None or last_action==[]:` fixed the issue

Running monsterkong and flappybird game

Hi, how can I manage to run the monsterkong or flappy bird game? They don't have a main method attached to them.

I am able to run all the other games like Pong etc. via python.

Thanks!

Does ple support multi-thread?

I am trying to use A3C to train flappy bird, however, when I create multiple thread, it crashed. Does any body meet this problem? The error code appears at this line:

    p = PLE(game, fps=30, display_screen=True)
    p.init()

The error is

[xcb] Unknown sequence number while processing reply
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
python: ../../src/xcb_io.c:635: _XReply: Assertion `!xcb_xlib_threads_sequence_lost' failed.
Aborted (core dumped)

This is my code:

import threading
import signal
from ple.games.flappybird import FlappyBird
from ple import PLE

stop_requested = False
def flappy_bird(i):
    print 'index:===', i
    game = FlappyBird()
    p = PLE(game, fps=30, display_screen=True)
    p.init()

    # actions = p.getActionSet()
    # print actions
    # reward = 0.0

    # for i in range(1000):
    #     if p.game_over():
    #         p.reset_game()
    #     # observation = p.getScreenRGB()
    #     if i % 20 == 0:
    #         action = actions[0]
    #     else:
    #         action = actions[1]
    #     print action
    #     reward = p.act(action)
    #     if stop_requested == True:
    #         break
    return


def signal_handler(signal_, frame_):
    print 'You pressed Ctrl+C !'
    stop_requested = True
    return


def run():
    train_treads = []
    for i in range(2):
        train_treads.append(threading.Thread(target=flappy_bird, args=(i,)))

    signal.signal(signal.SIGINT, signal_handler)

    for t in train_treads:
        t.start()

    print 'Press Ctrl+C to stop'
    signal.pause()

    print 'Now saving data....'
    for t in train_treads:
        t.join()

    return


if __name__ == '__main__':
    run()

Windows Dockerfile example?

Do you guys know if you can run the dockerfile from a windows host? Not really an issue I guess but not sure where to ask / put this request?

Thanks

Agents

Hey, can you give a list of agents that we can use and a bit of docs on how to incorporate the agent to this project