Having troubles with your submission? Check the FAQ and recent discussion for possible solutions.
This repository provides instructions for how to submit to the Obstacle Tower Challenge.
Your goal in the Obstacle Tower is to have your agent traverse the floors of a procedurally generated tower and climb to the highest level possible. Each level is progressively more difficult, and you'll be tested against a towers generated with random seeds your agent hasn't seen before and thus will need to generalize from the 100 provided tower seeds.
Before submitting to the challenge, you will want to train an agent to advance through the Obstacle Tower.
The first step is to clone this repository:
git clone [email protected]:Unity-Technologies/obstacle-tower-challenge.git
Next, install the following dependencies:
- Python dependencies
pip install -r requirements.txt
- Obstacle Tower (Your OS) Download the link for your OS here
and unzip in the
obstacle-tower-challengefolder from the cloned repository.
Finally, you can run the environment using the included agent (in run.py) with random actions:
python run.py
Note: Your Obstacle Tower build must be located at ./ObstacleTower/obstacletower.XYZ from the base of the
cloned repository, where XYZ represents the appropriate file extension for your operating system's Obstacle Tower
build.
Once you've set up your environment, you'll need to train your agent. We've provided a guide for using Google's Dopamine library to train an agent on Google Cloud Platform.
Before making your challenge submission, you may want to test your agent using a similar environment to the one used for the official challenge evaluation. Your agent and the Obstacle Tower environment will be run in separate Docker containers which can communicate over the local network.
- Docker See instructions here
- aicrowd-repo2docker
pip install aicrowd-repo2docker
# or
pip install -r requirements.txt
- Obstacle Tower (Linux) Download the linux build for docker evaluation here
and unzip in the
obstacle-tower-challengefolder from the cloned repository.
We've provided a build script that uses aicrowd-repo2docker to build an image obstacle_tower_challenge:latest from your repository. Ensure Docker is running on your machine, then run:
./build.sh
Now that you've built a Docker image with your agent script and the Obstacle Tower environment binary, you can run both the agent and the environment within a separate container:
# Start the container running your agent script.
docker run \
--rm \
--env OTC_EVALUATION_ENABLED=true \
--network=host \
-it obstacle_tower_challenge:latest ./run.sh
# In another terminal window, execute the environment.
docker run \
--rm \
--env OTC_EVALUATION_ENABLED=true \
--env OTC_DEMO_EVALUATION=true \
--network=host \
-it obstacle_tower_challenge:latest ./env.sh
To use GPU, add the tag --runtime=nvidia after docker run.
The environment script should output the evaluation state as it advances, recording overall state as well as the progress within each episode for seeds 101-105:
{"state":"PENDING","floor_number_avg":0.0,"reward_avg":-1.0,"episodes":[],"last_update":"2019-02-09T00:17:15Z"}
{"state":"IN_PROGRESS","floor_number_avg":0.0,"reward_avg":-1.0,"episodes":[{"state":"IN_PROGRESS","seed":101,"floor_number":0,"reward":0.0,"step_count":0}],"last_update":"2019-02-09T00:17:16Z"}
...To submit to the challenge you'll need to ensure you've set up an appropriate repository structure, create a private git repository at https://gitlab.aicrowd.com with the contents of your submission, and push a git tag corresponding to the version of your repository you'd like to submit.
Each repository should have a aicrowd.json file with the following fields:
{
"challenge_id" : "unity-obstacle-tower-challenge-2019",
"grader_id": "unity-obstacle-tower-challenge-2019",
"authors" : ["aicrowd-user"],
"description" : "Random Obstacle Tower agent",
"gpu": false,
"debug": false
}This file is used to identify your submission as a part of the Obstacle Tower Challenge. You must use the challenge_id and grader_id specified above in the submission. The gpu field specifies whether or not your model will require a GPU for evaluation. You can set the debug field to true if you want to view logs of your submission for debugging purposes (more information here).
You can specify your software environment by using all the available configuration options of repo2docker.
For example, to use Anaconda configuration files you can include an environment.yml file:
conda env export --no-build > environment.yml
It is important to include --no-build flag, which is important for allowing your Anaconda config to be replicable cross-platform.
The evaluator will use /home/aicrowd/run.sh as the entrypoint. Please remember to have a run.sh at the root which can instantiate any necessary environment variables and execute your code. This repository includes a sample run.sh file.
To make a submission, you will have to create a private repository on https://gitlab.aicrowd.com.
You will have to add your SSH Keys to your GitLab account by following the instructions here. If you do not have SSH Keys, you will first need to generate one.
Then you can create a submission by making a tag push to your repository, adding the correct git remote and pushing to the remote:
cd obstacle-tower-challenge
# Add AICrowd git remote endpoint
git remote add aicrowd [email protected]:<YOUR_AICROWD_USER_NAME>/obstacle-tower-challenge.git
git push aicrowd master
# Create a tag for your submission and push
git tag -am "submission-v0.1" submission-v0.1
git push aicrowd master
git push aicrowd submission-v0.1
# Note : If the contents of your repository (latest commit hash) does not change,
# then pushing a new tag will not trigger a new evaluation.
# Note : Only tag which begin with "submission-" will trigger an evaluation
You now should be able to see the details of your submission at : gitlab.aicrowd.com/<YOUR_AICROWD_USER_NAME>/obstacle-tower-challenge/issues
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent