torchcraft/stardata

Starcraft AI Research Dataset

TorchCraft

A bridge between Torch and StarCraft.

A general overview of the system (and motivations for using TorchCraft) can be found in:

Synnaeve, G., Nardelli, N., Auvolat, A., Chintala, S., Lacroix, T., Lin, Z., Richoux, F. and Usunier, N., 2016. TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games - arXiv:1611.00625.

Installation

Please follow the installation instructions.

You will need to have a license of StarCraft: Brood War.

TorchCraft is a BWAPI module that sends StarCraft data out over a ZMQ connection. This lets you parse StarCraft data and interact with BWAPI from anywhere. The TorchCraft client should be installed from C++, Python, or Lua. We provide off the shelf solutions for Python and Lua:

Requirements:

• zstd-devel 1.1.4
• zeromq 4+

Remember to init submodules: git submodule update --init --recursive
Python setup: pip install pybind11 && pip install .
Lua setup (depends on torch7): luarocks make *.rockspec

We provide an example C++ CMake project in examples/cpp/

The hardest part of installing the server is actually setting up starcraft. We detail three ways of doing this:

• Use the standard Windows + StarCraft route, perhaps with a VM
• Use WINE
• Use OpenBW

Running TorchCraft

See examples/ for a list of scripts that you can try. In all cases, the $server_ip is the ip address of the machine running StarCraft.

StarCraft Server

Do whichever was installed:

Windows / WINE

1. Open bwapi.ini and set ai = PATH\TO\BWEnv.dll (or copy ours from config/bwapi.in)
2. Start ChaosLauncher, enable BWAPI DLL Injector [Release].
3. Press Start.

WINE

1. Open bwapi.ini and set ai = PATH\TO\BWEnv.dll (or copy
2. wine bwheadless.exe -e $STARCRAFT/StarCraft.exe -l $STARCRAFT/bwapi-data/BWAPI.dll --headful as detailed in the WINE docs

OpenBW

Something like: OPENBW_ENABLE_UI=0 BWAPI_CONFIG_AI__RACE=Terran BWAPI_CONFIG_AI__AI="BWEnv/build/BWEnv.so" BWAPI_CONFIG_AUTO_MENU__AUTO_MENU="SINGLE_PLAYER" BWAPI_CONFIG_AUTO_MENU__MAP=maps/micro/m5v5_c_far.scm BWAPI_CONFIG_AUTO_MENU__GAME_TYPE="USE MAP SETTINGS" TORCHCRAFT_PORT=11111 BWAPILauncher

TorchCraft Client

$ cd examples
$ th lua/simple_dll.lua -t $server_ip
$ python py/example.py -t $server_ip

Other Documentation

• Lua Replayer

Citation

Please cite the arXiv paper if you use TorchCraft in your work:

@article{synnaeve2016torchcraft,
  title={TorchCraft: a Library for Machine Learning Research on Real-Time Strategy Games},
  author={Synnaeve, Gabriel and Nardelli, Nantas and Auvolat, Alex and Chintala, Soumith and Lacroix, Timoth{\'e}e and Lin, Zeming and Richoux, Florian and Usunier, Nicolas},
  journal={arXiv preprint arXiv:1611.00625},
  year={2016}
}

Contributing

See Contributing

Also check out the code structure and PR process

stardata's Issues

Dataset availability

I tried to access the amazon bucket with this awesome dataset as it would be really valuable for me in my school work and research. However, I got "All access to this object has been disabled" response. Is the dataset not available anymore? Is it somehow possible to get the original replay files?

How to extract unit information?

Hello,

For each match in StarData, I need to extract the game state information. Specifically, the attributes of all of the units on the map. However, the recent decision to make StarCraft free has made installation of the correct versions (BW, BWAPI, TorchCraft, etc) on Ubuntu 16 (via Wine) very complicated. Is there an easy way to extract unit information from the .rep files or the .tcr files without needing StarCraft (given that I don't need to watch the replays)?

There are a few hacked map replays

Recently, I'm implementing an AI using this dataset, and I found that there are a few replays that contain units whose unit types are not listed in the torchcraft/Constants/unittypes

I found that, surprisingly, they are Protoss Hero Archon and Terran Hero Battle Cruiser. These replays are recorded in the hacked maps, and I believe that these replays must be excluded from your dataset. I think that you should check this type of corrupted data. I only checked the case of PvT match-up in the Fighting Spirit.

Following are replays I found:
16/bwrep_8dtu8.rep (see 12m 30s )
3/bwrep_u5hyn.rep (see 5min 40s)
7/bwrep_cgrrb.rep (see 8min 30s)

I really appreciate your effort to make this dataset. Thank you.

fatal error: 'TH/TH.h' file not found

Hi, when trying to install, I get this error:

In file included from py/pyreplayer.cpp:4:
    include/replayer.h:13:10: fatal error: 'TH/TH.h' file not found
    #include <TH/TH.h>

It seems the TH.h file is missing from the folder?

The sequence of commands I ran to install:
git clone https://github.com/TorchCraft/StarData
git submodule update --init
cd TorchCraft
pip install .

Would appreciate any help!

.tcr replay parsing issue - "Corrupted replay: actions nPlayer > 9"

I am having an issue parsing the data with the python replayer module. I am able to parse the file linked in a previous issue about parsing (https://www.dropbox.com/s/xow9cfg9r0rheuy/bwrep_00r3w.tcr?dl=0), but am not able to parse any of the files contained in the 20 chunked replay tar.gzs. I am getting the following exception for file I try to parse:

Corrupted replay: actions nPlayer > 9

May you provide a BT seed please?

Thank you for sharing this precious data!
But due to our restricted network, it's hard to download this dataset from china.

I really wish you can provide a seed, so I can download it through BT.

Thank you again!

What data is saved

I went through the sources codes to try to find how exactly the data is saved, so I suspect it's somewhere around here:

https://github.com/TorchCraft/StarData/blob/master/extract_units.cpp#L81

I see it saves unit position and type into the output stream.

But that does not match what is written in the whitepaper:

The saved frame data contains full game state, including unit dynamic characteristics (about 30 per unit) and bullets.

https://arxiv.org/pdf/1708.02139.pdf

Am I missing something?

How to parse the .tcr replay file

Thank you for publishing the starcraft data.
I'm a developer of broodwar bot and have great interest in deep learning. I have download your data in dumped_replays and decompress the data, got lots of .tcr file, but I don't find any furthur guidance of how to analyze the dumped file. I have deploy the torchcraft, and I wonder if you can give me some hint about the processing procedure.
Thank you very much!

Connect to State in a Replay

Hello!

The python example looks very helpful for connecting to a State and gathering relevant information if I have a functional client and I'm stepping through a game, but is there a way to do this when iterating through replays? Can I initialize a State object in the first frame of a Replay somehow, and then step through the replay with that?

Thanks! (and sorry for contributing so many issues to the project)