microsoft/malmo-challenge

Task and example code for the Malmo Collaborative AI Challenge

The Malmo Collaborative AI Challenge

This repository contains the task definition and example code for the Malmo Collaborative AI Challenge. This challenge is organized to encourage research in collaborative AI - to work towards AI agents that learn to collaborate to solve problems and achieve goals. You can find additional details, including terms and conditions, prizes and information on how to participate at the Challenge Homepage.

---

Notes for challenge participants: Once you and your team decide to participate in the challenge, please make sure to register your team at our Registration Page. On the registration form, you need to provide a link to the GitHub repository that will contain your solution. We recommend that you fork this repository (learn how), and provide address of the forked repo. You can then update your submission as you make progress on the challenge task. We will consider the version of the code on branch master at the time of the submission deadline as your challenge submission. Your submission needs to contain code in working order, a 1-page description of your approach, and a 1-minute video that shows off your agent. Please see the challenge terms and conditions for further details.

---

Jump to:

• Installation

  • Prerequisites
  • Minimal installation
  • Optional extensions

• Getting started

  • Play the challenge task
  • Run your first experiment

• Next steps

  • Run an experiment in Docker on Azure
  • Compare your results again other teams
  • Resources

Installation

Prerequisites

• Python 2.7+ (recommended) or 3.5+
• Project Malmo - we recommend downloading the Malmo-0.21.0 release and installing dependencies for Windows, Linux or MacOS. Test your Malmo installation by launching Minecraft with Malmo and launching an agent.

Minimal installation

pip install -e git+https://github.com/Microsoft/malmo-challenge#egg=malmopy

or

git clone https://github.com/Microsoft/malmo-challenge
cd malmo-challenge
pip install -e .

Optional extensions

Some of the example code uses additional dependencies to provide 'extra' functionality. These can be installed using:

pip install -e '.[extra1, extra2]'

For example to install gym and chainer:

pip install -e '.[gym]'

Or to install all extras:

pip install -e '.[all]'

The following extras are available:

• gym: OpenAI Gym is an interface to a wide range of reinforcement learning environments. Installing this extra enables the Atari example agents in samples/atari to train on the gym environments. Note that OpenAI gym atari environments are currently not available on Windows.
• tensorflow: TensorFlow is a popular deep learning framework developed by Google. In our examples it enables visualizations through TensorBoard.

Getting started

Play the challenge task

The challenge task takes the form of a mini game, called Pig Chase. Learn about the game, and try playing it yourself on our Pig Chase Challenge page.

Run your first experiment

See how to run your first baseline experiment on the Pig Chase Challenge page.

Next steps

Run an experiment in Docker on Azure

Docker is a virtualization platform that makes it easy to deploy software with all its dependencies. We use docker to run experiments locally or in the cloud. Details on how to run an example experiment using docker are in the docker README.

Compare your results against other teams:

We provide you a leaderboard website where you can compare your results against the other participants.

Resources

• Malmo Platform Tutorial
• Azure Portal
• Docker Documentation
• Docker Machine on Azure
• CNTK Tutorials
• CNTK Documentation
• Chainer Documentation
• TensorBoard Documentation

malmo-challenge's Issues

Extend human-agent script to load and run a saved model

Useful for qualitative evaluation

Add public week-on-week leaderboard

Provide trained baseline models to challenge participants

Provide the following models: DQN trained on grayscale top-down view + DQN trained on grayscale first-person view

Is the pig supposed to move?

From playing the game, I see that the pig moves as pigs move in Minecraft, independent of what actions the agents take, or not take. In some cases, it may be convenient for both agents to wait, not taking any action, until the pig moves itself into a place that is close and easy to catch. Is this allowed?

This repo is missing important files

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

Extend training visualization

For diagnostics, add P(pig caught) and P(exit)

Extend dqn script to allow continued training from a saved model

from common import... is not found

I went through the install and I already had Malmo installed, but when I tried to run code using that line it said import not found. How do I install this?

ImportError: Module use of python27.dll conflicts with this version of Python

Hi, I used Python 3.5 to install all the things. However, when I try to run baseline file, there is such an error.

PigChaseSymbolicStateBuilder returns inconsistent states

Hi,

We found that PigChaseSymbolicStateBuilder sometimes returns wrong state information, either agents (or pig) disappear, or the agents at wrong places after executing a moving action.

For example the below figure. At the second last state, the pig and the two agents are there, but in the last state only Agent 2 appears (Agent 1 and the pig disappear). Is this a bug?

Best,
Mengchen

Deadlock in pig_chase_baseline.py

When I run pig_chase_baseline.py, it seems like the agents are stuck in a deadlock after serval actions taken. Both agents are standing still after 4~5 turns.

Docker connection issues.

The set-up is as follows:

• two docker containers on a server
• training a model on a different server using the malmo instances in the docker containers runing on the first server.

The two containers are using the malmo:latest Dockerfile with no modification. I am launching the containers and mapping the application port to the host port of the server like this:
docker run -p 10011:10000 malmo:latest
docker run -p 10012:10000 malmo:latest
also tried to the same result:
docker run -p host_ip:10011:10000 malmo:latest
docker run -p host_ip:10012:10000 malmo:latest

From a separate server I am trying to train a model and connect it the same way as if the malmo instances would have been local, but specifying the host ip of the dokcers and the host ports. So my clients list for the Challenger and TrainingAgent would look like this: [('10.18.28.14', 10011), ('10.18.28.14', 10012)].

The issue I have is that while the initial handshake between the two instances does happen, I eventually get some errors when one of the malmo instances tries to connect to some port it shouldn't have access to.

Also, this setup is working fine when the dockers are local, on the same server as the model I am training.

Here is the sample output of the first docker:

[15:26:31] [Thread-15/INFO] [STDOUT]: [com.microsoft.Malmo.Utils.TCPInputPoller:run:191]: Listening for messages on port 10000
[15:26:31] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: DORMANT
[15:26:31] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: DORMANT
[15:26:44] [Thread-17/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 10.18.28.17:
[15:26:44] [Thread-17/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: MALMO_REQUEST_CLIENT:0.21.0:20000:
[15:26:44] [Thread-17/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine:isReserved:154]: ==== RES:  - -1500910004819
[15:26:44] [Thread-17/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOOK
[15:26:44] [Thread-19/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 10.18.28.17:
[15:26:44] [Thread-19/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: <?xml version="1.0" encoding="UTF-8" standalone="no" ?><MissionInit xmlns="http://ProjectMalmo.microsoft.com" PlatformVersion="0.21.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://ProjectMalmo.microsoft.com MissionInit.xsd"><Mission><About><Summary>Catch the pig!</Summary></About><ModSettings><MsPerTick>1</MsPerTick></ModSettings><ServerSection><ServerInitialConditions><Time><StartTime>6000</StartTime><AllowPassageOfTime>false</AllowPassageOfTime></Time><Weather>clear</Weather><AllowSpawning>false</AllowSpawning></ServerInitialConditions><ServerHandlers><FlatWorldGenerator destroyAfterUse="true" forceReset="false" generatorString="3;minecraft:bedrock,2*minecraft:dirt,minecraft:grass;1;village" seed=""/><DrawingDecorator><DrawCuboid type="air" x1="-10" x2="10" y1="4" y2="45" z1="-10" z2="10"/><DrawLine type="sand" x1="1" x2="7" y1="3" y2="3" z1="0" z2="0"/><DrawLine type="fence" x1="1" x2="7" y1="4" y2="4" z1="0" z2="0"/><DrawLine type="sand" x1="1" x2="7" y1="3" y2="3" z1="
[15:26:45] [Thread-19/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOOK
[15:26:45] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$DormantEpisode:checkForMissionCommand:764]: Mission received: Catch the pig!
[15:26:45] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: CREATING_HANDLERS
[15:26:45] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: CREATING_HANDLERS
[15:26:45] [Thread-20/INFO] [STDOUT]: [com.microsoft.Malmo.Utils.TCPInputPoller:run:191]: Listening for messages on port 10656
[15:26:45] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: EVALUATING_WORLD_REQUIREMENTS
[15:26:45] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: EVALUATING_WORLD_REQUIREMENTS
[15:26:45] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: CREATING_NEW_WORLD
[15:26:45] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: CREATING_NEW_WORLD
[15:26:46] [Server thread/INFO]: Starting integrated minecraft server version 1.8
[15:26:46] [Server thread/INFO]: Generating keypair
[15:26:46] [Server thread/INFO]: Converting map!
[15:26:46] [Server thread/INFO]: Scanning folders...
[15:26:46] [Server thread/INFO]: Total conversion count is 0
[15:26:46] [Server thread/INFO] [FML]: Injecting existing block and item data into this server instance
[15:26:46] [Server thread/INFO] [FML]: Applying holder lookups
[15:26:46] [Server thread/INFO] [FML]: Holder lookups applied
[15:26:46] [Server thread/INFO] [FML]: Loading dimension 0 (TEMP_10000_48960987-6f36-4612-893f-a25dca36a97f) (net.minecraft.server.integrated.IntegratedServer@5e783ed9)
[15:26:48] [Server thread/INFO] [FML]: Loading dimension 1 (TEMP_10000_48960987-6f36-4612-893f-a25dca36a97f) (net.minecraft.server.integrated.IntegratedServer@5e783ed9)
[15:26:48] [Server thread/INFO] [FML]: Loading dimension -1 (TEMP_10000_48960987-6f36-4612-893f-a25dca36a97f) (net.minecraft.server.integrated.IntegratedServer@5e783ed9)
[15:26:48] [Server thread/INFO]: Preparing start region for level 0
[15:26:48] [Server thread/INFO]: Changing view distance to 6, from 10
[15:26:49] [Thread-23/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 10.18.28.17:
[15:26:49] [Thread-23/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: MALMO_FIND_SERVER
[15:26:49] [Thread-23/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMONOSERVERYET
[15:26:50] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: SERVER request state: WAITING_FOR_MOD_READY
[15:26:50] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: WAITING_FOR_SERVER_READY
[15:26:50] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: WAITING_FOR_SERVER_READY
[15:26:50] [Client thread/INFO]: Started on 39584
[15:26:50] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: SERVER enter state: WAITING_FOR_MOD_READY
[15:26:50] [Netty Local Client IO #0/INFO] [FML]: Server protocol version 2
[15:26:50] [Netty Server IO #1/INFO] [FML]: Client protocol version 2
[15:26:50] [Netty Server IO #1/INFO] [FML]: Client attempting to join with 4 mods : [email protected],[email protected],[email protected],[email protected]
[15:26:50] [Netty Local Client IO #0/INFO] [FML]: [Netty Local Client IO #0] Client side modded connection established
[15:26:50] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: SERVER request state: DORMANT
[15:26:50] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: SERVER enter state: DORMANT
[15:26:50] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.Server.ServerStateMachine$DormantEpisode:execute:468]: INCOMING MISSION: Received MissionInit directly through ServerStateMachine constructor.
[15:26:50] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.Server.ServerStateMachine$DormantEpisode:onReceiveMissionInit:499]: Mission received: Catch the pig!
[15:26:50] [Server thread/INFO]: Received mission: ?9Catch the pig!
[15:26:50] [Server thread/INFO]: Source: ?a10.18.28.17
[15:26:50] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: SERVER request state: BUILDING_WORLD
[15:26:50] [Server thread/INFO] [FML]: [Server thread] Server side modded connection established
[15:26:50] [Server thread/INFO]: Agent_1[local:E:6515ccef] logged in with entity id 0 at (647.5, 4.0, 446.5)
[15:26:50] [Server thread/INFO]: Agent_1 joined the game
[15:26:50] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: SERVER enter state: BUILDING_WORLD
[15:26:50] [Thread-26/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 10.18.28.17:
[15:26:50] [Thread-26/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: MALMO_FIND_SERVER
[15:26:50] [Thread-26/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOS10.18.28.14:39584
[15:26:51] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: SERVER request state: WAITING_FOR_AGENTS_TO_ASSEMBLE
[15:26:51] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: SERVER enter state: WAITING_FOR_AGENTS_TO_ASSEMBLE
[15:26:51] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.Server.ServerStateMachine$WaitingForAgentsEpisode:execute:878]: Experiment requires: 
[15:26:51] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.Server.ServerStateMachine$WaitingForAgentsEpisode:execute:881]: >>>> Agent_1
[15:26:51] [Server thread/INFO] [STDOUT]: [com.microsoft.Malmo.Server.ServerStateMachine$WaitingForAgentsEpisode:execute:881]: >>>> Agent_2
[15:26:52] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$WaitingForServerEpisode:onClientTick:923]: ***Telling server we are ready - Agent_1
[15:26:52] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$WaitingForServerEpisode:onClientTick:923]: ***Telling server we are ready - Agent_1
[15:26:53] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$WaitingForServerEpisode:onClientTick:923]: ***Telling server we are ready - Agent_1

And the second, the one that errors out:

[15:26:19] [Thread-15/INFO] [STDOUT]: [com.microsoft.Malmo.Utils.TCPInputPoller:run:191]: Listening for messages on port 10000
[15:26:19] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: DORMANT
[15:26:19] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: DORMANT
[15:26:44] [Thread-17/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 10.18.28.17:
[15:26:44] [Thread-17/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: MALMO_REQUEST_CLIENT:0.21.0:20000:
[15:26:44] [Thread-17/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine:isReserved:154]: ==== RES:  - -1500910004830
[15:26:44] [Thread-17/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOOK
[15:26:49] [Thread-19/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 10.18.28.17:
[15:26:49] [Thread-19/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: MALMO_FIND_SERVER
[15:26:49] [Thread-19/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMONOSERVER
[15:26:50] [Thread-21/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 10.18.28.17:
[15:26:50] [Thread-21/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: <?xml version="1.0" encoding="UTF-8" standalone="no" ?><MissionInit xmlns="http://ProjectMalmo.microsoft.com" PlatformVersion="0.21.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://ProjectMalmo.microsoft.com MissionInit.xsd"><Mission><About><Summary>Catch the pig!</Summary></About><ModSettings><MsPerTick>1</MsPerTick></ModSettings><ServerSection><ServerInitialConditions><Time><StartTime>6000</StartTime><AllowPassageOfTime>false</AllowPassageOfTime></Time><Weather>clear</Weather><AllowSpawning>false</AllowSpawning></ServerInitialConditions><ServerHandlers><FlatWorldGenerator destroyAfterUse="true" forceReset="false" generatorString="3;minecraft:bedrock,2*minecraft:dirt,minecraft:grass;1;village" seed=""/><DrawingDecorator><DrawCuboid type="air" x1="-10" x2="10" y1="4" y2="45" z1="-10" z2="10"/><DrawLine type="sand" x1="1" x2="7" y1="3" y2="3" z1="0" z2="0"/><DrawLine type="fence" x1="1" x2="7" y1="4" y2="4" z1="0" z2="0"/><DrawLine type="sand" x1="1" x2="7" y1="3" y2="3" z1="
[15:26:51] [Thread-21/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOOK
[15:26:51] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$DormantEpisode:checkForMissionCommand:764]: Mission received: Catch the pig!
[15:26:51] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: CREATING_HANDLERS
[15:26:51] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: CREATING_HANDLERS
[15:26:51] [Thread-22/INFO] [STDOUT]: [com.microsoft.Malmo.Utils.TCPInputPoller:run:191]: Listening for messages on port 10112
[15:26:51] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: WAITING_FOR_SERVER_READY
[15:26:51] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: WAITING_FOR_SERVER_READY
[15:26:51] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$WaitingForServerEpisode:execute:992]: We should be joining 10.18.28.14:39584
[15:26:52] [Client thread/INFO]: Connecting to 10.18.28.14, 39584
[15:26:52] [Server Connector #1/ERROR]: Couldn't connect to server
java.net.ConnectException: Connection refused: /10.18.28.14:39584
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[?:1.8.0_131]
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) ~[?:1.8.0_131]
	at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:191) ~[NioSocketChannel.class:4.0.15.Final]
	at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:255) ~[AbstractNioChannel$AbstractNioUnsafe.class:4.0.15.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:502) ~[NioEventLoop.class:4.0.15.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:452) ~[NioEventLoop.class:4.0.15.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:346) ~[NioEventLoop.class:4.0.15.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101) ~[SingleThreadEventExecutor$2.class:4.0.15.Final]
	at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]
[15:26:52] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: ERROR_CANNOT_CONNECT_TO_SERVER
[15:26:52] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: ERROR_CANNOT_CONNECT_TO_SERVER
[15:26:52] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$MissionEndedEpisode:sendMissionEnded:1915]: Sending mission ended message to 10.18.28.17:10173.
[15:26:52] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: DORMANT
[15:26:52] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: DORMANT
[15:26:52] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Utils.TCPInputPoller:clearCommands:127]: JETTISONING 0 COMMANDS

Challenge Agent does not have "move -1"?

Hey,
When I try to print out the ENV_ACTIONS from common, I see that the ChanllengeAgent does not have (move -1) as one of its actions, but at the end of each episode, it will execute this (move -1), any particular reason for that?

How is the game evaluated?

Hey,
So is the performance of the agent evaluated on the total score of the Agent0 & Agent1 or only the score of Agent1 (which I assumed is the one to be controlled by our algorithm)?
Cos during some test plays I observed that everytime Agent1 makes the final the move to corner the pig, both Agent0 and Agent1 get the 25 big reward; while if it's Agent0 who makes the final move to corner the pig, only Agent0 will get the 25 while Agent1 will not.
Thanks!

agent return a strange state

python pig_chase_dqn.py

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "pig_chase_dqn.py", line 122, in agent_factory
    action = agent.act(obs, reward, agent_done, is_training=True)
  File "/home/tai/Freiburg/Malmo/malmo-challenge/ai_challenge/pig_chase/agent.py", line 73, in act
    return self.current_agent.act(new_state, reward, done, is_training)
  File "/home/tai/Freiburg/Malmo/malmo-challenge/ai_challenge/pig_chase/agent.py", line 107, in act
    me = [(j, i) for i, v in enumerate(state) for j, k in enumerate(v) if self.name in k]
TypeError: 'numpy.uint8' object is not iterabl

It is not a (9,9) matrix but a (84,1) list and all the elements are int.

How to obtain the position of the pig and position of the other agent?

Hi everyone,

I've been looking at the code of the Focused agent to see how it gets the position of the pig, but I couldn't really understand how it gets it. I know it uses a value called target, but I haven't seen where does it get the target value.

Also, I'd like to know how can I obtain the position of the two agents when I'm deciding the next action to take by one of them.

Thanks!

Tkinter import needs to happen after forking on OSX

workaround - use threads rather than forking processes

Where can I find the definition of entity "pig"?

The movement rule and frequency of pig entity aren't clear enough.

agent_loop in evaluation.py randomizes agent types

In the agent_loop in evaluation.py (with a fix for #43 used)

[...]
while episode < EVAL_EPISODES:
        # check if env needs reset
        if env.done:
            print('Episode %d (%.2f)%%' % (episode, (episode / EVAL_EPISODES) * 100.))
            
            # **** fixes #43
            if type(agent.current_agent) == RandomAgent:
                    agent_type = PigChaseEnvironment.AGENT_TYPE_1
                else:
                    agent_type = PigChaseEnvironment.AGENT_TYPE_2
            
            obs = env.reset(agent_type)
            # ****
            while obs is None:
                # this can happen if the episode ended with the first
                # action of the other agent
                print('Warning: received obs == None.')
                obs = env.reset()

            episode += 1

        # select an action
        action = agent.act(obs, reward, agent_done, is_training=True)
        # take a step
        obs, reward, agent_done = env.do(action)

Since agent.act([...],agent_done=True,[...]) resets the current_agent used, it needs to happen before env.reset(agent_type). Otherwise the appearance of the agent is based on the agent type used during the previous episode.

By simply moving the agent.act([...]) call above the if env.done, leads to an action from the previous episode to be given as the first action in the subsequent episode.

unexpected stop of the environment "NetworkDispatcher exception io.netty.handler.timeout.ReadTimeoutException"

I start several pairs of Minecraft environments in docker. Use every pair of them as a training process. However, sometimes part of the processes stop unexpectedly (others are still working). Every process is a function just like the provided baseline or dqn code.
The output error is:

Traceback (most recent call last):
  File "/usr/lib/python2.7/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python2.7/multiprocessing/process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "xxxxx", line 89, in agent_factory
    obs = env.reset(agent_type)
  File "/root/malmo-challenge/ai_challenge/pig_chase/environment.py", line 382, in reset
    return super(PigChaseEnvironment, self).reset()
  File "/root/malmo-challenge/malmopy/environment/malmo/malmo.py", line 315, in reset
    (self.MAX_START_MISSION_RETRY, e))
Exception: Unable to connect after 50 tries There are not enough clients availble in the ClientPool to start this 2 agent mission.

output of the agent 1 minecraft:

[17:20:13] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$WaitingForServerEpisode:onClientTick:923]: ***Telling server we are ready - Agent_1
[17:20:13] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$WaitingForServerEpisode:onClientTick:923]: ***Telling server we are ready - Agent_1
[17:20:13] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$WaitingForServerEpisode:onClientTick:923]: ***Telling server we are ready - Agent_1
[17:20:13] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$WaitingForServerEpisode:onClientTick:923]: ***Telling server we are ready - Agent_1
[17:20:14] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$WaitingForServerEpisode:onClientTick:923]: ***Telling server we are ready - Agent_1
[17:20:14] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$WaitingForServerEpisode:onClientTick:923]: ***Telling server we are ready - Agent_1

output of agent2 minecraft:

[16:59:48] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: IDLING
[16:59:48] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: IDLING
[16:59:49] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: MISSION_ENDED
[16:59:49] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: MISSION_ENDED
[16:59:49] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$MissionEndedEpisode:sendMissionEnded:1915]: Sending mission ended message to 172.19.0.9:10583.
[16:59:49] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: WAITING_FOR_SERVER_MISSION_END
[16:59:49] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: WAITING_FOR_SERVER_MISSION_END
[16:59:49] [Thread-1565/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 172.19.0.9:
[16:59:49] [Thread-1565/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: MALMO_FIND_SERVER
[16:59:49] [Thread-1565/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOSmalmo1:45573
[16:59:49] [Thread-1567/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 172.19.0.9:
[16:59:49] [Thread-1567/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: MALMO_REQUEST_CLIENT:0.21.0:20000:
[16:59:49] [Thread-1567/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOBUSY
[16:59:49] [Thread-1569/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 172.19.0.9:
[16:59:49] [Thread-1569/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: <?xml version="1.0" encoding="UTF-8" standalone="no" ?><MissionInit xmlns="http://ProjectMalmo.microsoft.com" PlatformVersion="0.21.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://ProjectMalmo.microsoft.com MissionInit.xsd"><Mission><About><Summary>Catch the pig!</Summary></About><ModSettings><MsPerTick>4</MsPerTick></ModSettings><ServerSection><ServerInitialConditions><Time><StartTime>6000</StartTime><AllowPassageOfTime>false</AllowPassageOfTime></Time><Weather>clear</Weather><AllowSpawning>false</AllowSpawning></ServerInitialConditions><ServerHandlers><FlatWorldGenerator destroyAfterUse="true" forceReset="false" generatorString="3;minecraft:bedrock,2*minecraft:dirt,minecraft:grass;1;village" seed=""/><DrawingDecorator><DrawCuboid type="air" x1="-10" x2="10" y1="4" y2="45" z1="-10" z2="10"/><DrawLine type="sand" x1="1" x2="7" y1="3" y2="3" z1="0" z2="0"/><DrawLine type="fence" x1="1" x2="7" y1="4" y2="4" z1="0" z2="0"/><DrawLine type="sand" x1="1" x2="7" y1="3" y2="3" z1="
[16:59:49] [Thread-1569/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOBUSY
[16:59:49] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: DORMANT
[16:59:49] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: DORMANT
[16:59:49] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Utils.TCPInputPoller:clearCommands:127]: JETTISONING 0 COMMANDS
[16:59:55] [Thread-1571/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 172.19.0.9:
[16:59:55] [Thread-1571/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: MALMO_REQUEST_CLIENT:0.21.0:20000:
[16:59:55] [Thread-1571/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine:isReserved:154]: ==== RES:  - 6363
[16:59:55] [Thread-1571/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOOK
[16:59:55] [Thread-1573/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 172.19.0.9:
[16:59:55] [Thread-1573/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: MALMO_CANCEL_REQUEST
[16:59:55] [Thread-1573/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine:isReserved:154]: ==== RES: RESERVED - 19989
[16:59:55] [Thread-1573/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOOK
[16:59:57] [Thread-1575/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:443]: Received from 172.19.0.9:
[16:59:57] [Thread-1575/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:onCommand:444]: MALMO_REQUEST_CLIENT:0.21.0:20000:
[16:59:57] [Thread-1575/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine:isReserved:154]: ==== RES:  - 18283
[16:59:57] [Thread-1575/INFO] [STDOUT]: [com.microsoft.Malmo.Client.ClientStateMachine$1:reply:428]: REPLYING WITH: MALMOOK
[17:00:10] [Netty Client IO #1/ERROR] [FML]: NetworkDispatcher exception
io.netty.handler.timeout.ReadTimeoutException
[17:00:10] [Client thread/INFO] [FML]: Applying holder lookups
[17:00:10] [Client thread/INFO] [FML]: Holder lookups applied

Add a top-down view for simplified vision experiments

Add tabular q-learning baseline

...and add a screenshot of the results here.

Method do of PigChaseEnvironment returns inconsistent type

the env.do(...) method produces both tuple and numpy.ndarray for the returned state.

Add print(type(obs)) after line 125 in evaluation.py to reproduce the following output:

Episode 6 (0.00)%
<type 'tuple'>
<type 'numpy.ndarray'>
<type 'tuple'>
<type 'numpy.ndarray'>
<type 'tuple'>
<type 'numpy.ndarray'>
<type 'tuple'>

Agent appearance not reset in evaluation.py

In evaluation.py (75d0bfb) the agent's appearance is not reset.

Command obs = env.reset() is missing suitable agent_type as input, i.e obs = env.reset(agent_type)

Trained model for submission

Hi,

if we have understood correctly, then our submission should include two already trained agents with respectively 100.000 and 500.000 actions taken? Or would it be okay to submit agents trained with less steps?

We are worried that it will be hard to manage to do that before the submission deadline.

Could you give some tips on the fastest way to train the agents?

docker run malmo, the client stucked

From /malmo-challenge/docker, After docker build malmo, I run docker run malmo. The client stucked there. I use ubuntu16.
Below is the log

username@34ISZ:~/Desktop/malmo-challenge/docker$ docker run malmo
Starting a new Gradle Daemon for this build (subsequent builds will be faster).

---

Powered By MCP:
http://modcoderpack.com/
Searge, ProfMobius, Fesh0r,
R4wk, ZeuX, IngisKahn, bspkrs
MCP Data version : 9.10

---

:deobfCompileDummyTask
:deobfProvidedDummyTask
:getVersionJson
:extractUserdev UP-TO-DATE
:extractMcpData SKIPPED
:extractMcpMappings SKIPPED
:genSrgs SKIPPED
:downloadClient SKIPPED
:downloadServer SKIPPED
:splitServerJar SKIPPED
:mergeJars SKIPPED
:deobfMcSRG SKIPPED
:getFernFlower SKIPPED
:decompileMc SKIPPED
:fixMcSources SKIPPED
:applySourcePatches SKIPPED
:remapMcSources SKIPPED
:recompileMc SKIPPED
:extractNatives SKIPPED
:getAssetIndex UP-TO-DATE
:getAssets
Current status: 271/734 36%
Current status: 504/734 68%
Current status: 658/734 89%
:makeStart SKIPPED
:setupDecompWorkspace

BUILD SUCCESSFUL

Total time: 18.355 secs
:deobfCompileDummyTask
:deobfProvidedDummyTask
:compileApiJava UP-TO-DATE
:processApiResources UP-TO-DATE
:apiClasses UP-TO-DATE
:copyModToClient
:copyModToServer
:deleteSchemas
:copySchemas
:jaxb UP-TO-DATE
:sourceMainJava UP-TO-DATE
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:jar
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE
:extractMcpData SKIPPED
:extractMcpMappings SKIPPED
:getVersionJson
:extractUserdev UP-TO-DATE
:genSrgs SKIPPED
:reobfJar
:assemble
:check UP-TO-DATE
:build

BUILD SUCCESSFUL

Total time: 2.761 secs
:deobfCompileDummyTask
:deobfProvidedDummyTask
:compileApiJava UP-TO-DATE
:processApiResources UP-TO-DATE
:apiClasses UP-TO-DATE
:copyModToClient UP-TO-DATE
:copyModToServer UP-TO-DATE
:deleteSchemas
:copySchemas
:jaxb UP-TO-DATE
:sourceMainJava UP-TO-DATE
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:jar
:getVersionJson
:extractNatives SKIPPED
:extractUserdev UP-TO-DATE
:getAssetIndex UP-TO-DATE
:getAssets
:makeStart SKIPPED
:runClient
[22:10:18] [main/INFO] [GradleStart]: Extra: []
[22:10:18] [main/INFO] [GradleStart]: Running with arguments: [--userProperties, {}, --assetsDir, /root/.gradle/caches/minecraft/assets, --assetIndex, 1.8, --accessToken{REDACTED}, --version, 1.8, --tweakClass, net.minecraftforge.fml.common.launcher.FMLTweaker]
[22:10:18] [main/INFO] [LaunchWrapper]: Loading tweak class name net.minecraftforge.fml.common.launcher.FMLTweaker
[22:10:18] [main/INFO] [LaunchWrapper]: Using primary tweak class name net.minecraftforge.fml.common.launcher.FMLTweaker
[22:10:18] [main/INFO] [LaunchWrapper]: Calling tweak class net.minecraftforge.fml.common.launcher.FMLTweaker
[22:10:18] [main/INFO] [FML]: Forge Mod Loader version 11.14.3.1543 for Minecraft 1.8 loading
[22:10:18] [main/INFO] [FML]: Java is OpenJDK 64-Bit Server VM, version 1.8.0_171, running on Linux:amd64:4.4.0-130-generic, installed at /usr/lib/jvm/java-8-openjdk-amd64/jre
[22:10:18] [main/INFO] [FML]: Managed to load a deobfuscated Minecraft name- we are in a deobfuscated environment. Skipping runtime deobfuscation
[22:10:18] [main/INFO] [FML]: Found a command line coremod : com.microsoft.Malmo.OverclockingPlugin
[22:10:18] [main/WARN] [FML]: The coremod com.microsoft.Malmo.OverclockingPlugin does not have a MCVersion annotation, it may cause issues with this version of Minecraft
[22:10:18] [main/INFO] [LaunchWrapper]: Loading tweak class name net.minecraftforge.fml.common.launcher.FMLInjectionAndSortingTweaker
[22:10:18] [main/INFO] [LaunchWrapper]: Loading tweak class name net.minecraftforge.fml.common.launcher.FMLDeobfTweaker
[22:10:18] [main/INFO] [LaunchWrapper]: Calling tweak class net.minecraftforge.fml.common.launcher.FMLInjectionAndSortingTweaker
[22:10:18] [main/INFO] [LaunchWrapper]: Calling tweak class net.minecraftforge.fml.common.launcher.FMLInjectionAndSortingTweaker
[22:10:18] [main/INFO] [LaunchWrapper]: Calling tweak class net.minecraftforge.fml.relauncher.CoreModManager$FMLPluginWrapper
[22:10:18] [main/ERROR] [FML]: The binary patch set is missing. Either you are in a development environment, or things are not going to work!
[22:10:18] [main/ERROR] [FML]: FML appears to be missing any signature data. This is not a good thing
[22:10:18] [main/INFO] [LaunchWrapper]: Calling tweak class net.minecraftforge.fml.relauncher.CoreModManager$FMLPluginWrapper
[22:10:18] [main/INFO] [LaunchWrapper]: Calling tweak class net.minecraftforge.fml.relauncher.CoreModManager$FMLPluginWrapper
[22:10:18] [main/INFO] [LaunchWrapper]: Calling tweak class net.minecraftforge.fml.common.launcher.FMLDeobfTweaker
[22:10:19] [main/INFO] [LaunchWrapper]: Loading tweak class name net.minecraftforge.fml.common.launcher.TerminalTweaker
[22:10:19] [main/INFO] [LaunchWrapper]: Calling tweak class net.minecraftforge.fml.common.launcher.TerminalTweaker
[22:10:19] [main/INFO] [LaunchWrapper]: Launching wrapped minecraft {net.minecraft.client.main.Main}
[22:10:19] [main/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:transform:52]: MALMO: Attempting to transform MinecraftServer
[22:10:19] [main/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:overclockRenderer:147]: MALMO: Found Minecraft, attempting to transform it
[22:10:19] [main/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:overclockRenderer:153]: MALMO: Found Minecraft.runGameLoop() method, attempting to transform it
[22:10:19] [main/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:overclockRenderer:168]: MALMO: Hooked into call to Minecraft.updateDisplay()
[22:10:19] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:transform:52]: MALMO: Attempting to transform MinecraftServer
[22:10:19] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:overclockServer:112]: MALMO: Found MinecraftServer, attempting to transform it
[22:10:19] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:overclockServer:118]: MALMO: Found MinecraftServer.run() method, attempting to transform it
[22:10:19] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:overclockServer:126]: MALMO: Transforming LDC
[22:10:19] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:overclockServer:126]: MALMO: Transforming LDC
[22:10:19] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:overclockServer:126]: MALMO: Transforming LDC
[22:10:19] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.OverclockingClassTransformer:overclockServer:126]: MALMO: Transforming LDC
[22:10:19] [Client thread/INFO]: Setting user: Player516
[22:10:20] [Client thread/INFO]: LWJGL Version: 2.9.1
[22:10:21] [Client thread/INFO] [FML]: Could not load splash.properties, will create a default one
[22:10:21] [Client thread/INFO] [STDOUT]: [net.minecraftforge.fml.client.SplashProgress:start:-1]: ---- Minecraft Crash Report ----
// Hey, that tickles! Hehehe!

Time: 7/10/18 10:10 PM
Description: Loading screen debug info

This is just a prompt for computer specs to be printed. THIS IS NOT A ERROR

A detailed walkthrough of the error, its code path and all known details is as follows:

-- System Details --
Details:
Minecraft Version: 1.8
Operating System: Linux (amd64) version 4.4.0-130-generic
Java Version: 1.8.0_171, Oracle Corporation
Java VM Version: OpenJDK 64-Bit Server VM (mixed mode), Oracle Corporation
Memory: 313767720 bytes (299 MB) / 458227712 bytes (437 MB) up to 1908932608 bytes (1820 MB)
JVM Flags: 1 total; -Xmx2G
IntCache: cache: 0, tcache: 0, allocated: 0, tallocated: 0
FML:
Loaded coremods (and transformers):
OverclockingPlugin (unknown)
com.microsoft.Malmo.OverclockingClassTransformer
GL info: ' Vendor: 'VMware, Inc.' Version: '3.0 Mesa 18.0.5' Renderer: 'llvmpipe (LLVM 6.0, 256 bits)'
[22:10:21] [Client thread/INFO] [MinecraftForge]: Attempting early MinecraftForge initialization
[22:10:21] [Client thread/INFO] [FML]: MinecraftForge v11.14.3.1543 Initialized
[22:10:21] [Client thread/INFO] [FML]: Replaced 204 ore recipies
[22:10:21] [Client thread/INFO] [MinecraftForge]: Completed early MinecraftForge initialization
[22:10:21] [Client thread/INFO] [FML]: Found 0 mods from the command line. Injecting into mod discoverer
[22:10:21] [Client thread/INFO] [FML]: Searching /root/Malmo/Minecraft/run/mods for mods
[22:10:22] [Client thread/INFO] [FML]: Forge Mod Loader has identified 4 mods to load
[22:10:22] [Client thread/INFO] [FML]: Attempting connection with missing mods [mcp, FML, Forge, malmomod] at CLIENT
[22:10:22] [Client thread/INFO] [FML]: Attempting connection with missing mods [mcp, FML, Forge, malmomod] at SERVER
[22:10:22] [Client thread/INFO]: Reloading ResourceManager: Default, FMLFileResourcePack:Forge Mod Loader, FMLFileResourcePack:Minecraft Forge, FMLFileResourcePack:Microsoft Malmo Mod
[22:10:22] [Client thread/WARN]: ResourcePack: ignored non-lowercase namespace: MalmoMod in /root/Malmo/Minecraft/build/libs/MalmoMod-0.21.0.jar
[22:10:22] [Client thread/WARN]: ResourcePack: ignored non-lowercase namespace: MalmoMod in /root/Malmo/Minecraft/build/libs/MalmoMod-0.21.0.jar
[22:10:22] [Client thread/WARN]: ResourcePack: ignored non-lowercase namespace: MalmoMod in /root/Malmo/Minecraft/build/libs/MalmoMod-0.21.0.jar
[22:10:22] [Client thread/INFO] [FML]: Processing ObjectHolder annotations
[22:10:22] [Client thread/INFO] [FML]: Found 384 ObjectHolder annotations
[22:10:22] [Client thread/INFO] [FML]: Identifying ItemStackHolder annotations
[22:10:22] [Client thread/INFO] [FML]: Found 0 ItemStackHolder annotations
[22:10:22] [Client thread/INFO] [FML]: Configured a dormant chunk cache size of 0
[22:10:22] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.Utils.SchemaHelper:testSchemaVersionNumbers:157]: Testing schemas against internal version number: 0.21
[22:10:22] [Client thread/INFO] [FML]: Applying holder lookups
[22:10:22] [Client thread/INFO] [FML]: Holder lookups applied
[22:10:22] [Client thread/INFO] [FML]: Injecting itemstacks
[22:10:22] [Client thread/INFO] [FML]: Itemstack injection complete
[22:10:22] [Client thread/INFO] [STDOUT]: [tv.twitch.StandardCoreAPI::16]: If on Windows, make sure to provide all of the necessary dll's as specified in the twitchsdk README. Also, make sure to set the PATH environment variable to point to the directory containing the dll's.
[22:10:22] [Client thread/ERROR]: Couldn't initialize twitch stream
[22:10:22] [Sound Library Loader/INFO]: Starting up SoundSystem...
[22:10:22] [Thread-7/INFO]: Initializing LWJGL OpenAL
[22:10:22] [Thread-7/INFO]: (The LWJGL binding of OpenAL. For more information, see http://www.lwjgl.org)
ALSA lib confmisc.c:768:(parse_card) cannot find card '0'
ALSA lib conf.c:4292:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4292:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1251:(snd_func_refer) error evaluating name
ALSA lib conf.c:4292:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4771:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM default
AL lib: (EE) alsa_open_playback: Could not open playback device 'default': No such file or directory
[22:10:23] [Thread-7/ERROR]: Error in class 'LibraryLWJGLOpenAL'
[22:10:23] [Thread-7/ERROR]: Unable to initialize OpenAL. Probable cause: OpenAL not supported.
[22:10:23] [Thread-7/WARN]: ERROR MESSAGE:
[22:10:23] [Thread-7/INFO]: Could not locate OpenAL library.
[22:10:23] [Thread-7/WARN]: STACK TRACE:
[22:10:23] [Thread-7/INFO]: org.lwjgl.openal.AL.create(AL.java:151)
[22:10:23] [Thread-7/INFO]: org.lwjgl.openal.AL.create(AL.java:102)
[22:10:23] [Thread-7/INFO]: org.lwjgl.openal.AL.create(AL.java:201)
[22:10:23] [Thread-7/INFO]: paulscode.sound.libraries.LibraryLWJGLOpenAL.init(LibraryLWJGLOpenAL.java:164)
[22:10:23] [Thread-7/INFO]: paulscode.sound.SoundSystem.CommandNewLibrary(SoundSystem.java:1576)
[22:10:23] [Thread-7/INFO]: paulscode.sound.SoundSystem.CommandQueue(SoundSystem.java:2572)
[22:10:23] [Thread-7/INFO]: paulscode.sound.CommandThread.run(CommandThread.java:121)
[22:10:23] [Sound Library Loader/WARN]: ERROR MESSAGE:
[22:10:23] [Sound Library Loader/INFO]: Could not locate OpenAL library.
[22:10:23] [Sound Library Loader/INFO]: Starting up SoundSystem...
[22:10:23] [Client thread/INFO] [FML]: Max texture size: 8192
[22:10:23] [Client thread/INFO]: Created: 16x16 textures-atlas
[22:10:23] [Thread-9/INFO]: Switching to No Sound
[22:10:23] [Thread-9/INFO]: (Silent Mode)
[22:10:23] [Sound Library Loader/INFO]: Sound engine started
[22:10:23] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: WAITING_FOR_MOD_READY
[22:10:23] [Client thread/INFO] [FML]: Injecting itemstacks
[22:10:23] [Client thread/INFO] [FML]: Itemstack injection complete
[22:10:23] [Client thread/INFO] [FML]: Forge Mod Loader has successfully loaded 4 mods
[22:10:23] [Client thread/INFO]: Reloading ResourceManager: Default, FMLFileResourcePack:Forge Mod Loader, FMLFileResourcePack:Minecraft Forge, FMLFileResourcePack:Microsoft Malmo Mod
[22:10:23] [Client thread/WARN]: ResourcePack: ignored non-lowercase namespace: MalmoMod in /root/Malmo/Minecraft/build/libs/MalmoMod-0.21.0.jar
[22:10:23] [Client thread/WARN]: ResourcePack: ignored non-lowercase namespace: MalmoMod in /root/Malmo/Minecraft/build/libs/MalmoMod-0.21.0.jar
[22:10:23] [Client thread/WARN]: ResourcePack: ignored non-lowercase namespace: MalmoMod in /root/Malmo/Minecraft/build/libs/MalmoMod-0.21.0.jar
[22:10:24] [Client thread/INFO]: SoundSystem shutting down...
[22:10:24] [Client thread/WARN]: Author: Paul Lamb, www.paulscode.com
[22:10:24] [Sound Library Loader/INFO]: Starting up SoundSystem...
[22:10:24] [Thread-11/INFO]: Initializing LWJGL OpenAL
[22:10:24] [Thread-11/INFO]: (The LWJGL binding of OpenAL. For more information, see http://www.lwjgl.org)
ALSA lib confmisc.c:768:(parse_card) cannot find card '0'
ALSA lib conf.c:4292:(_snd_config_evaluate) function snd_func_card_driver returned error: No such file or directory
ALSA lib confmisc.c:392:(snd_func_concat) error evaluating strings
ALSA lib conf.c:4292:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1251:(snd_func_refer) error evaluating name
ALSA lib conf.c:4292:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4771:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2266:(snd_pcm_open_noupdate) Unknown PCM default
AL lib: (EE) alsa_open_playback: Could not open playback device 'default': No such file or directory
[22:10:24] [Thread-11/ERROR]: Error in class 'LibraryLWJGLOpenAL'
[22:10:24] [Thread-11/ERROR]: Unable to initialize OpenAL. Probable cause: OpenAL not supported.
[22:10:24] [Thread-11/WARN]: ERROR MESSAGE:
[22:10:24] [Thread-11/INFO]: Could not locate OpenAL library.
[22:10:24] [Thread-11/WARN]: STACK TRACE:
[22:10:24] [Thread-11/INFO]: org.lwjgl.openal.AL.create(AL.java:151)
[22:10:24] [Thread-11/INFO]: org.lwjgl.openal.AL.create(AL.java:102)
[22:10:24] [Thread-11/INFO]: org.lwjgl.openal.AL.create(AL.java:201)
[22:10:24] [Thread-11/INFO]: paulscode.sound.libraries.LibraryLWJGLOpenAL.init(LibraryLWJGLOpenAL.java:164)
[22:10:24] [Thread-11/INFO]: paulscode.sound.SoundSystem.CommandNewLibrary(SoundSystem.java:1576)
[22:10:24] [Thread-11/INFO]: paulscode.sound.SoundSystem.CommandQueue(SoundSystem.java:2572)
[22:10:24] [Thread-11/INFO]: paulscode.sound.CommandThread.run(CommandThread.java:121)
[22:10:24] [Client thread/INFO] [FML]: Max texture size: 8192
[22:10:24] [Sound Library Loader/WARN]: ERROR MESSAGE:
[22:10:24] [Sound Library Loader/INFO]: Could not locate OpenAL library.
[22:10:24] [Sound Library Loader/INFO]: Starting up SoundSystem...
[22:10:24] [Client thread/INFO]: Created: 512x512 textures-atlas
[22:10:25] [Thread-13/INFO]: Switching to No Sound
[22:10:25] [Thread-13/INFO]: (Silent Mode)
[22:10:25] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: WAITING_FOR_MOD_READY
[22:10:25] [Thread-15/INFO] [STDOUT]: [com.microsoft.Malmo.Utils.TCPInputPoller:run:191]: Listening for messages on port 10000
[22:10:25] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:queueStateChange:143]: CLIENT request state: DORMANT
[22:10:25] [Sound Library Loader/INFO]: Sound engine started
[22:10:25] [Client thread/INFO] [STDOUT]: [com.microsoft.Malmo.StateMachine:setState:101]: CLIENT enter state: DORMANT

Using PigChaseSymbolicStateBuilder for the submission agent

We can do that right? Use the symbolic state for our agent.

Add a docker example to run training off of local code

Our agent will be evaluated by playing with all other teams' agents, or just the challenge agent.

Please clarify, thanks!

no module named malmopython

When execute pig_chase_baseline.py

OpenCV found, setting as default backend.
Traceback (most recent call last):
  File "pig_chase_baseline.py", line 38, in <module>
    from environment import PigChaseEnvironment, PigChaseSymbolicStateBuilder
  File "/home/tyler/Software/Malmo/malmo-challenge/ai_challenge/pig_chase/environment.py", line 27, in <module>
    from MalmoPython import MissionSpec
ImportError: No module named MalmoPython

But nothing happened for malmo examples.

Cannot run pig_chase_human_vs_agent.py or pig_chase_baseline.py

Hello, after a fresh install of Malmo and cloning the challenge repo, I cannot seem to get the challenge working. I have both instances of malmo up and running and when i try to run either of the two above scripts I get:

Traceback (most recent call last):
File "pig_chase_baseline.py", line 37, in
from agent import PigChaseChallengeAgent, FocusedAgent
File "/home/zach/malmo-challenge/ai_challenge/pig_chase/agent.py", line 23, in
from tkinter import ttk, Canvas, W
File "/usr/local/lib/python2.7/dist-packages/tkinter/init.py", line 6, in
from Tkinter import (_cnfmerge, _default_root, _flatten, _join, _setit,
ImportError: cannot import name _splitdict

Not sure how this error is occurring. Has anyone run into this?

The process has forked and you cannot use this CoreFoundation functionality safely.

I ran the

python pig_chase_human_vs_agent.py

on OSX then it turns out the following result.

The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().Break on THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC() to debug.

It seems that the OSX do not allow multi-process gui?

Add official evaluation script

Input: trained agent (git repo + git commit id + json config file that details agent class + parameters)
Output: performance in terms of score@100k and score@500k, where score is the actual malmo pig chase game score, @100k means that the model was saved after 100,000 interaction steps.

Evaluation procedure: the evaluation script should run the trained model for 500 episodes and reports the mean + stderr performance achieved during evaluation.