davechurchill/starcraftaitournamentmanager

Tournament Manager Software for StarCraft AI Competitions

starcraftaitournamentmanager's Issues

vcredists easy install script typos

Two of the vcredist installer names have a dot instead of an underscore in the name.

>intall_all_redist.bat
>vs2008_vcredist_x86.exe /q /norestart
>vs2008_sp1_vcredist_x86.exe /q /norestart
>vs2010_vcredist_x86.exe /q /norestart
>vs2012_vcredist_x86.exe /q /norestart
>vs2013_vcredist_x86.exe /q /norestart
>vs2015_vcredist_x86.exe /q /norestart
'vs2015_vcredist_x86.exe' is not recognized as an internal or external command,
operable program or batch file.
>vs2017_vcredist_x86.exe /q /norestart
'vs2017_vcredist_x86.exe' is not recognized as an internal or external command,
operable program or batch file.

Client gets stuck in loop on: CMD /C taskkill /T /F /IM StarCraft.exe

I have the server and client set up on local machine, and another client in a Win10VM.

The clients see the server, and can start starcraft.

Problem: The clients (on either machine) when hitting the CMD /C taskkill /T /F /IM StarCraft.exe cannot continue further. BW does not close for them, freezing the tournament.

"run_proxy.bat" may not be executed with a low but persistent possibility

When we run our bot on clusters, we see a low but persistent crash rate of our bot, about 1 out of 50 games. After printing out the log, we found that the "run_proxy.bat" was not executed for those crashed games(but this file does exist, just not executed). We run a python model in "run_proxy.bat", which is essential to our bot.

We think this may caused by java Runtime environment in line 146 of /src/utility/WindowsCommandTools.java
Process proc = Runtime.getRuntime().exec(windowsCommandPrefix + command); Too frequent commands may invalidate this one.

We did a test to simulate this situation, the code is published in https://github.com/LukeZhuang/test_proxy. We use Java Runtime to run the test "run_proxy.bat" for 200 times and wait for 10 seconds after each 2 times. After checking "output.txt", we found that "run_proxy.bat" was in fact executed for 100 times, just half of them was executed.

Potential Solution:
We found that adding sleep command after excuting Java Runtime can solve this problem. For example:
Process proc = Runtime.getRuntime().exec(windowsCommandPrefix + command);
Thread.sleep(10);
Then the crash rate is 0 after adding that.
Probably it creates a latency so it make sure that each command will not be overrided.
After adding this to our test simulation, we found that "run_proxy.bat" was executed for 160 times, more than before, but still not 200 times. So, we think we can conclude that this problem is caused by too frequent commands.

Suggestions for the AIIDE competition and/or ladder

I have finally got around to listening to the Undermind podcasts and noticed that in one episode, you asked long-time bot developers for suggestions for StarcraftAITournamentManager for AIIDE. Pasting the relevant discussion from Discord:

I have some suggestions for what's missing from current tournament setups:

1. Rules/logic for Current Working Dir and run_proxy.bat

Refer to the e-mail conversation I had with Dave and Rick about this at the end of Oct '18.
Clarify in the Rules page whether the new TM software will set the current working dir to the Starcraft dir (e.g. C:\TM\Starcraft) or the AI dir (e.g. C:\TM\Starcraft\bwapi-data\AI), for DLL bots and non-DLL bots like EXEs/JARs. Afaik, sc-docker always sets the current working dir to the Starcraft dir (not the AI dir) then starts bwapi-data/AI/.dll or bwapi-data/AI/run_proxy.bat. Should non-DLL bots submit a run_proxy.bat file, and if so, should it change the working dir at all (if so, to what absolute path or what relative path?)? Or for non-DLL bots, perhaps authors should submit a run_proxy.bat file that either doesn't set/change it, or sets it to a particular environment var if it exists (e.g. AIIDE_STARCRAFT_DIR), otherwise the bot is allowed to set it to whatever they like (so that it will work on their personal dev/test environments and AIIDE without editing). Even UAlbertaBot is confusing - its' run_proxy.bat does "cd c:\TM\Starcraft\bwapi-data\AI" then "c:\TM\Starcraft\bwapi-data\AI\UAlbertaBot.exe" then its' config file sets the "ReadDirectory" setting to "../read/" and "WriteDirectory" to "../write/". Even though the Rules page says to use "bwapi-data/read" and "bwapi-data/write". If this isn't fixed or clarified (ideally fix StarcraftAITournamentManager to follow the Rules page, which is also how sc-docker works), we will keep seeing broken bots and teething problems in the various competitions. It is a pity this single issue has caused so many broken/weak bots in past AIIDE & CIG competitions.

PurpleWave / Cheese Maven: I think StarCraft directory is intended to be the one true working directory
just that implementations of run_proxy.bat in tournaments (namely, CIG) have not achieved that consistently

Afaik, the latest dev version of StarcraftAITournamentManager sets it to the .../Starcraft/bwapi-data/AI folder (not the .../Starcraft folder), so unless that is fixed, the run_proxy.bat file must be set up to include a pushd or cd command that changes the working dir (to e.g. C:\TM\Starcraft or ../..) in order to get the bot to work properly (e.g. find BWTA2 map data at the correct location, and so that File I/O uses the correct paths to the read and write folders etc).

The main change would be to edit ClientCommands.java and change the line:
WindowsCommandTools.RunWindowsCommand("cd " + ClientSettings.Instance().ClientStarcraftDir + "bwapi-data\\AI & "+ ClientSettings.Instance().ClientStarcraftDir + "bwapi-data\\AI\\run_proxy.bat", false, false);
to:
WindowsCommandTools.RunWindowsCommand("cd " + ClientSettings.Instance().ClientStarcraftDir & "+ ClientSettings.Instance().ClientStarcraftDir + "bwapi-data\\AI\\run_proxy.bat", false, false);
... and also change UAlbertaBot accordingly if you still want UAlbertaBot to work. I don't know whether any other old AIIDE bots would need updating (possibly just some of their run_proxy.bat files). On the AIIDE competition website (maybe on the Rules page) I also suggest showing a sample run_proxy.bat file for a JAR bot and a sample run_proxy.bat for an EXE bot.

2. Ensure the Tournament Module allows some permissions for some versions of BWAPI

Update: this point has been moved into #32.

After the bug in point 7 is fixed, rebuild the TournamentModule.dll's (for BWAPI 4.2.0 and 4.1.2 at least - I wouldn't bother fixing 4.0.1 Beta, and I wouldn't bother changing 3.7.4 which doesn't allow any permissions) to allow bots to do setLatCom(false) and setCommandOptimizationLevel(a value other than 1, i.e. especially 0 or 2, but perhaps some other authors might want 3 or 4). They don't affect the opponent bot at all - it only affects what info BWAPI returns to you, and some bots rely on them. The default should probably stay as 1, but I think bots should be allowed to set it to 0 and other values if they want to, even though a value of 0 may increase the bot's APM and replay file sizes, because some bots rely on them and currently there is no way for a bot to check what the command optimization level is. And anyway, currently, BWAPI version 4.2.0/4.1.2/4.0.1 Beta bots have always done this until now, due to the bug, so I don't think it's a problem. I suggest to also allow leaveGame(), so that bots can leave the game to indicate they are resigning/surrendering in lost positions, or in bugged states to attempt to avoid crashing later (and this may help address bwapi/bwapi#719
). This might also help increase the overall game rate, so long as Starcraft or bots don't/rarely crash when leaving, because bot-vs-bot endgames can often take a lot of time if the winning bot is weak/slow. Personally, I don't care whether sendText/printf/setTextSize are allowed, so I would probably disallow them on anti-collusion grounds and in case they help to increase performance, but I think the others should be disallowed.

Note: until the bug noticed in point 7 is fixed, the Tournament Module for BWAPI versions 4.2.0, 4.1.2 and 4.0.1 Beta actually allows all permissions (even for DLL bots, not just for non-DLL bots). This point should be addressed after point 7 is fixed.

3. Game Speed slider setting in game lobbies

Update: this issue has been moved to #37.

At least a few months before the competition (so we have time for sufficient training/testing), tell us in the Rules page whether the Game Speed slider in the game lobbies will be set to Normal or Fastest (i.e. LF3 or LF6 respectively). I added a feature request to BWAPI to make organizers' management easier for this (bwapi/bwapi#807). If you want to use "Fastest", I suggest you ask as early as possible whether there are any objections from the big teams like Facebook and Sumsung and other serious competitors, because bots that rely on ML training might need a lot of time/resources to be re-trained.

4. Ability to disable bots (not just ignore their results)

Ability to disqualify a bot (manually at least, perhaps also automatically based on some thresholds) so that it doesn't play any of the remaining games, e.g. if it crashes/times out too much. I know there's already a feature to exclude a bot from the results summaries, but afaik it still plays all the remaining games. I could be wrong. I don't want this ability for my testing - it's just an idea you might find helpful.

5. Better checks for why some games fail to start

Better checks for whether the TM actually handed control to each individual bot for frame 0 and whether each individual bot actually returned control to the TM on frame 0, so that it is clear whether games that didn't start properly are a problem before each bot was started (i.e. not either of the bots' fault), or a problem after each bot tried to start. I haven't investigated whether the following is how StarcraftAITournamentManager already works, but please check the following cases:

• If HeroBot started ok and returned control to the TM on frame 0, but the TM never got as far as starting the call(s) to MegaBot's binaries for frame 0, HeroBot should count it as a win, and MegaBot's "total games played" shouldn't count that game. Or if you want symmetry, count is as a loss for MegaBot.
• If HeroBot started ok and returned control to the TM on frame 0, and the TM started the call(s) to MegaBot's binaries for frame 0 but the TM never got as far as passing control to MegaBot for frame 0 (e.g. missing mandatory DLL dependencies, so MegaBot didn't get as far as entering its onStart() function), HeroBot should count it as a win, and it's up to you whether it should be counted as a loss for MegaBot (if you want symmetry) or MegaBot's "total games played" shouldn't count that game. In this case, it's unclear whether to blame MegaBot or an environmental problem caused by the TM or Starcraft/bots from old games (e.g. out of memory from a old processes that are still running for whatever reason).
• If HeroBot started ok and returned control to the TM on frame 0, and the TM passed control to MegaBot for frame 0 but MegaBot didn't return control back to the TM for frame 0 (this happened a lot for some bots like MegaBot), HeroBot should be assigned a win (rather than ignoring the game entirely), and MegaBot should be assigned a loss.

I don't know whether there's any way in the TM to distinguish between the last two cases though. In all cases, try to distinguish the various cases as much as feasible in a field that stores the reason for the result for that bot.

6. Request a new official BWAPI 4.x release

Update: this point has been moved into #34.

Request a new official BWAPI 4.x release (4.2.0 was released most recently, so it would be something like 4.3.0) and build the corresponding TournamentModule.dll (& add it & its source code to GitHub). A few bots are probably now using a more recent dev version of BWAPI than 4.2.0 (post-4.2.0 includes improvements to BWAPI's latency compensation logic, among other things).

7. TournamentModule.dll allows all permissions for BWAPI version 4 (e.g. 4.2.0, 4.1.2, 4.0.1 Beta)!

Update: this point has been moved into #32.

Due to a change to a BWAPI function's argument type as part of the first release of version 4 that was not correspondingly migrated into the Tournament Module (i.e. the TournamentModule::onAction function's first argument type was changed from an into to a BWAPI::Tournament::ActionID), permissions like enableFlag(CompleteMapInformation), setLocalSpeed(42) are not enforced by TournamentModule.dll, because the default implementation of the function that is defined in BWAPI is being used, which always returns true. This is why the Sling bot was able to successfully do setLocalSpeed(42) and setFrameSkip(0) until Sling's source code was patched, and the Tyr/TyrProtoss bot was able to successfully do setLocalSpeed(10) in the AIIDE and CIG competitions, which slowed down the runs. If any bot that uses BWAPI 4 had cheated by doing things like enableFlag(CompleteMapInformation), the tournament module wouldn't have stopped them. Note that this is true for non-DLL bots and for DLL bots (originally I thought it only affected non-DLL bots, but after investigating and finding the cause of the bug I realized it affects DLL bots too).

8. Add a Win Percentage Per Map Cycle Graph

In the summary results, add a graph for "Win Percentage Per Map Cycle Graph". So if there are 10 maps, rather than ~100 points the X axis, there would only be 100/10 = 10. For each point on the X axis, it would only consider the games for that particular cycle through the set of maps (not consider the earlier cycles too). This avoids the noise caused by bots being better on some maps than others, and makes it easier to identify turning points in the progress of a bot's learning.

9. Add a Win Percentage Over Time by Map Cycle Graph

In the summary results, add a graph for "Win Percentage Over Time by Map Cycle Graph". For each point on the X axis, it would only consider the games for that map cycle and the earlier cycles combined. This avoids the noise caused by bots being better on some maps than others, and shows the rankings as-at the end of each map cycle.

10. Add an ability to pick race randomly out of Terran/Protoss/Zerg in the game lobby

I am not suggesting to forbid picking "Random" race in the game lobby - just to allow bots to specify that the TM should pick randomly out of Terran/Protoss/Zerg from the drop-down in the game lobby. This makes the management easier for authors so they can easily write and update a single bot that supports playing as different races (rather than needing to set up & update one bot that always plays as Terran like tscmoo (Terran), one bot that always plays as Protoss like tscmoop, one bot that always plays as Zerg like tscmooz). Note: in terms of game results records and File I/O, it is important to distinguish between picking "Random" race from the drop-down and picking a specific race out of Terran/Protoss/Zerg, because the meta is quite different when the opponent knows your race at the very start of the game - they don't need to scout your race or common proxy locations to help rule out some of the various possible early game attacks like 4-6 pool, offensive sunken rush, cannon rush, BBS or bunker rush etc. Some human players despise (& I get the feeling some bot authors dislike) that the meta is different against Random race players that exploit the fact that the opponent does not know their race, compared with knowing the opponent's race. I know that Dave doesn't see this as a concern for AIIDE until a bot actually relies on it, but personally I think this feature would be handy for future-proofing in case it becomes a problem sooner than expected. No big deal.

11. Add support for BWAPI 3.7.5

Add support for BWAPI 3.7.5. I wouldn't use it, but some of the GOAL-based bots in particular, might.

12. Add remaining SSCAIT & CIG maps with BWTA2/BWTA data

Add the map files and BWTA2 & BWTA data for the SSCAIT and CIG maps. E.g. create "maps/sscai" and "maps/cig" folders for the map files within each ZIP file in the "server/required" folder that corresponds to a particular BWAPI version, and similarly for the "bwapi-data/BWTA" and "bwapi-data/BWTA2" folders. This will make it easier for the CIG organizers to run their competitions, and would be very convenient for bot authors that want to train their bot. I know there is occasionally talk of changing the map pool for some comps/ladders, but I wouldn't hold my breath. The map files and BWTA2 & BWTA data for the remaining CIG maps can be downloaded from here. The map files for SSCAIT can be downloaded from here but I am not aware of an official source for their BWTA2/BWTA data. I could send you the files I have, or I could fork & add what I have. I probably downloaded them from another bot, or I made a bot generate them.

13. Separate storage

It's quite time-consuming to manage the "Required_BWAPI_.zip" files, e.g. to add a support for a new BWAPI version or add maps to all supported BWAPI versions. Most of the contents don't seem to vary according to the BWAPI version, so a feature that might make it easier for organizers and bot authors to manage in future is to separate the common files to a separate ZIP file. I.E. almost everything except the BWAPI-related DLLs, bwapi.ini, "Multiple Instance Hack.bat", TournamentModule.dll, Broodwar.map, bspatch.exe, starcraft.bsdiff, maybe injectory.x86.exe (not sure whether this depends on BWAPI version), and SNP_DirectIP.snp. I suggest that almost all of the files that are used on client VMs except the actual bot files/data are installed on client VMs. Some automation could be added that recreates the bwapi-data folder etc and copies the files into place each game, rather than downloaded every time a client downloads a bot. This avoids wasting a lot of time downloading a lot of data from the server every game. Some folders should be re-created every game though, e.g. the bwapi-data folder, and maybe the characters folder, and maybe older BWAPI versions like 3.7.4 or 3.7.5 have some special files/folders (not sure). Likewise, the files associated with a particular version of BWAPI could all be installed somewhere on the client VMs and copied into place every game, depending on the BWAPI version used by the bot.

14. Add dependencies for BWAPI/bot-related libraries

Install onto the client VMs' Starcraft folder the DLLs etc for BWAPI-related libraries that bots commonly depend on such as the various versions of BWAPI bridge (bwapi_bridge1_2.dll, bwapi_bridge2_3.dll, bwapi_bridge2_4.dll, bwapi_bridge2_5.dll, bwapi_bridge2_6.dll), QtCore4.dll, QtGui4.dll, and the files sc-docker has at https://github.com/Games-and-Simulations/sc-docker/tree/master/docker/dlls. And config files used by some bots like Aiur to configure where the correct locations of the read and write folders are (directories.txt, file_settings.txt) to stop them updating files in the Starcraft folder itself. This would really help to more easily set up and manage client VMs and avoid a lot of headaches trying to get bots to start / work properly due to missing DLL dependencies or File I/O configuration issues. I can add the files I have to a fork if you like.

Header links don't work without JS handlers

Minor issue: The header links like "Bots" and "Detailed Results" lack an src attribute and thus you can't do things like middle-click or control-click them.

Questions about Cherrypi

We installed CUDA9.0 and corresponding CuDNN as you suggested and we tried to run Cherrypi again. It still doesn't work and stuck like this.

And if we run Cherrypi.exe straightly, there is an error printed on the screen.

We don't know what's wrong with it. we appreciate it if you could tell us what to do next.

ManagerClientThread Object Streams could not initialize

Any idea what may be causing the following Java exception when running the TM?

    at server.ServerClientThread.setupConnectionStreams(ServerClientThread.java:76)
   at server.ServerClientThread.run(ServerClientThread.java:44)

java.io.StreamCorruptedException: invalid stream header: 0300002F
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:857)
at java.io.ObjectInputStream.(ObjectInputStream.java:349)
at server.ServerClientThread.setupConnectionStreams(ServerClientThread.java:71)
at server.ServerClientThread.run(ServerClientThread.java:44)

problems of psexec

I use the psexec script in the document to launch client at different machine,
but it seems only launch the process at back-end, when I remote login the target machine, I can not see the client is running and the client exit after a short time.....

load other dll at run time

Hi,

I wonder what's the standard way to load other dlls at run time using dll-bots?
add ./AI directory to PATH or specify the dll path in code?

Thanks

TM overwrites bwapi.ini

The TM spits out a hard coded BWAPI.ini, so if you want to do something like have the sound turned on, you can't edit the bwapi.ini file in the required folders. If you do, the clients will still get stuck with the hard coded bwapi from the TM (this prevents changing any other settings for the .ini file too).

Game not started counted incorrectly

If a game doesn't start (rare, but happens, the away client sometimes can't join the game) I believe it is counted as a crash for the bot that reports first, even though both report crashes (because in a normal game we assume whoever reports first crashed first).

I believe the proper solution is to reschedule the game.

Missing source code for TournamentModule.dll for BWAPI version 4.2.0

The source code is missing for TournamentModule.dll for BWAPI version 4.2.0, i.e. the file bwapi-data/TournamentModule.dll contained in server/required/Required_BWAPI_420.zip. For the other versions it is located at e.g. src/tournamentmodule/BWAPI_412

Please could it be added? The reasons are for transparency and because I would like to build my own variant of TournamentModule.dll for BWAPI 4.2.0 that runs on Windows XP SP3 (FYI, BWAPI 4.2.0 doesn't work on Windows XP SP3 and I've already proposed a fix to BWAPI in bwapi/bwapi#733). Thanks

support of machine learning library

because it is much more efficient to use other language(ie. python) to write some machine learning code on top of some library(ie. tensorflow), but the official AIIDE tournament machine seems do not support such python library.
so I has several questions:

1. do you have plan to do such things in future?
2. is the submitted AI folder has space limitation? for example, can i put a pre-config python version in the AI folder which may occupy about 30m.
3. other recommended alternative?

@davechurchill @richard-kelly

Thanks

Add option to set the random seed via seed_override

Add an option to set the random seed.

Background: From BWAPI version 4.0.1 Beta onwards, there is a seed_override option in bwapi.ini that allows users to fix the random seed at the start of a game (including multi-player). Here's the corresponding comments about this option in the default bwapi.ini (pasted from BWAPI 4.2.0):

; Random seed override. This uses a fixed seed at the start of the game so that if played out the exact same way,
; the same occurrences will happen every time. This value must be a decimal integer.
;
; When this key is commented out, Starcraft will use the system time as a seed. This is the default behaviour.
;
; Note: This option affects both single AND multi-player modes (for game hosts only). This means that hosting a multi-player
; game with this option enabled will distribute this fixed seed to all other players in the game.
;seed_override = 123456789

I'm creating this issue on behalf of @davechurchill who requested on Facebook that I create this issue. I don't plan to use this option personally, but others might find it useful.

Is there some way let StarcraftAITournamentManager enable CompleteMapInformation?

StarcraftAITournamentManager is a excellent tools to run and visualize game result with multiplayer.
It is necessary to generate training data for my Bot learning. But I can not get the groundtruth value without CompleteMapInformation flag. I am tired of using chaoslauncher to set diffrent version BWAPI's opponent one by one.
Wish you tell me how to generate groundtruth values while StarcraftAITournamentManager run. I know it can get the CompleteMapInformation refer to bwapi/bwapi#594

Add bot packs from all AIIDE competitions

Right now you have the bots from 2014, 2015, 2016 and 2017 included. It would be nice if you add the bots from all the other previous competitions so people don't have to download them manually. 🙂

Can't upload bot

I attempted to upload PurpleWave.zip, which contained PurpleWave.jar and BWAPI.dll. But the upload fails, with returning a HTTP 500 error

PurpleWave.zip

Precompiled JARs

Hello,

The Tournament Manager documentation at http://www.cs.mun.ca/~dchurchill/starcraftaicomp/tm.shtml#download and at https://www.youtube.com/watch?v=tl-nansNbsA mention that precompiled server and client JARs are available from this repository. This no longer seems to be the case.

Are the precompiled JARs available anywhere at the moment?

Use the latest Microsoft Visual C++ x86 redistributables

Update the installer for the Microsoft Visual C++ x86 redistributables (README.md has links to http://www.cs.mun.ca/~dchurchill/starcraftaicomp/all_vcredist_x86.zip) to use the latest versions of the redistributables. The reasons are to ensure StarcraftAITournamentManager is compatible with BWAPI 4.4.0, and with bots that use BWAPI < 4.4.0 but updated compiler versions, and because it may fix some crashes and security vulnerabilities for old bots (Microsoft updated some old redists to fix some crashes in MFC).

client occasionally getting stuck in Status: SENDING

Attaching some screenshots. As you can see from the first screenshot, the tournament manager client got stuck in "SENDING" status and the StarCraft process is still running. It happens often enough and requires manual intervention. What could be causing the issue?

File I/O not synchronized immediately between rounds

I'm using the tournament manager in a way where the game schedule has every following game to also be in a new round, e.g.
...
game 15, round 0
game 16, round 1
game 17, round 2
...
Hoping that the TM would synchronize the File I/O after each round. It looks like that doesn't always work though.
For example, I tested with the following games: BotA vs BotB, BotB vs BotA, BotA vs BotB and in the third game, BotA didn't seem to have access to the second game.
What could the issue be? Is File I/O really synchronized after each round or could there be any delays?

some confusing problems

We tried to run the Starcraft AI Tournament Manager and we encountered some problems which make us confused.

1. We added some bots of AIIDE2018 to the competition and modified server_settings.json as stated, but some bots didn’t do any action in some specific games( such as Locutus vs. CSE, Locutus didn’t do anything) when they act normal in other games.
2. When the game of Cherrypi begins, the starcraft will crash and has no response.

4.4 Tournament Module mistimes client bots

BWAPI 4.4 provides very different timing information for client and module bots. This bug causes the current 4.4 Tournament Module to miscalculate the amount of time a client bot has spent running, usually a vast overestimate.

Let's say there's a bot that spends 1ms on each onUnitShow and 20ms on each onFrame. On a frame where two units show, and the bot takes 22ms to handle all its events. If the tournament manager adds up getLastEventTime()s for each event:

• If it's a module bot, the tournament manager sees the bot take 22ms
• If it's a client bot, the tournament manager sees the bot take 66ms

Empirically I've observed this happening to PurpleWave on the AIIDE ladder. In one game where it got disqualified after 7:29 game minutes and under 4 real-time minutes:

• Bot says: 19 frames went over 55ms
• JBWAPI says: 33 frames went over 42ms
• Tournament Module says: 320 frames went over 55ms

Add support for BWAPI 4.4.0 (but not 4.3.0 because it crashes a lot)

Update 2019-03-24: the original plan was to add support for BWAPI 4.3.0 and that is how this issue was originally written, but the plan changed to only add support for BWAPI 4.4.0, not 4.3.0 too. So I have edited this issue to replace references of BWAPI 4.3.0 with just BWAPI 4.4.0 (even though the commit history will show that 4.3.0 was added then 4.4.0 added then 4.3.0 removed).

Add support for BWAPI 4.4.0 by adding Required_BWAPI_440.zip. The only differences for Required_BWAPI_440.zip compared with Required_BWAPI_420.zip are the updated TournamentModule.dll that needs to be built (using the updated ExampleTournamentModule.vcxproj file from BWAPI, and also add the source code to a separate folder), and the updated official BWAPI.dll & updated official SNP_DirectIP.snp. Also, remove Multiple Instance Hack.bat, bspatch.exe, starcraft.bsdiff because they were removed from BWAPI several versions ago (since version 4.1.2).

For the source code for TournamentModule.dll, I will combine the updated ExampleTournamentModule.vcxproj file from BWAPI 4.4.0 with the TM source code for BWAPI 4.2.0 from after #32 was fixed, so that those permissions fixes are also applied to BWAPI 4.4.0.

To ensure compatibility with BWAPI 4.4.0, I recommend installing the latest version of the x86 "Microsoft Visual C++ Redistributable for Visual Studio 2017". https://support.microsoft.com/en-au/help/2977003/the-latest-supported-visual-c-downloads provides a download link to whatever the latest version is. The links on that page change every time they release a new version. Currently (as-at 2019-03-24), the version of the redist is 14.16.27027, although it is actually displayed in Add/Remove Programs as 14.16.27027.1.

replay is twisted

very strange issue.

I use TM to play games. I firstly notice that some replay is strange, seems like bugs in my bot.
so I create a one-one match on two VM, using the same injectory.x86.exe command to start both bots, I watch my bots playing through the game until the end. but when I load the replay to see some detail info,
I found that after about 20 minutes playing, my bot's behavior is totally different from what I saw before, it seems like another bot take over this.
I also log some detail information, like how many unit I own at this time, and it also totally different from what the replay shows. I has tested many time and this situation is continuously happening.

I don't has any clue about why this can happen, has anyone encounter this issue before?
Thanks

Possibly Starcraft UDP Connection Issue?

I tried to use one machine to run both server/run_server.bat and client/run_client.bat, and a second machine to start a client. The server was able recognize it, alongside the client created on the machine which started the server. The server tried to start a game, one client (non-server side) then launched an instance of the Starcraft game, and the game crashed/was terminated after seeing the scene where the game lets you to choose which connection type (Battle.net, UDP, etc.). The other client (server side) also launched the game, and was stuck at the scene where the game lets you to choose game mode (single player, multiplayer, etc.). I wonder if the UDP connection (supposedly the type to use here) is not set up properly, or was this caused by something else?

1 vs N tournament

Hi,

I notice that when the tournament type set to 1vsN, the schedule of the tournament is not paralleling even though I have enough machines.
for example, there are three bots : A, B, C the schedule is sequential :
A-B
A-C
not A-B and A-C holding on four different machines.

as far as I know, paralleling the game for each round may satisfy current AIIDE rules, so is there some settings which I missed to achieve this...

Bugfixes to Tournament Module permissions for BWAPI versions 4.2.0 & 4.1.2

From #31 points 7 then 2. I.E.:

#31 point 7 is that there is a bug in the Tournament Module that causes it to allows all permissions for BWAPI version 4 (e.g. 4.2.0, 4.1.2, 4.0.1 Beta)!

Due to a change to a BWAPI function's argument type as part of the first release of version 4 that was not correspondingly migrated into the Tournament Module (i.e. the TournamentModule::onAction function's first argument type was changed from an into to a BWAPI::Tournament::ActionID), permissions like enableFlag(CompleteMapInformation), setLocalSpeed(42) are not enforced by TournamentModule.dll, because the default implementation of the function that is defined in BWAPI is being used, which always returns true. This is why the Sling bot was able to successfully do setLocalSpeed(42) and setFrameSkip(0) until Sling's source code was patched, and the Tyr/TyrProtoss bot was able to successfully do setLocalSpeed(10) in the AIIDE and CIG competitions, which slowed down the runs. If any bot that uses BWAPI 4 had cheated by doing things like enableFlag(CompleteMapInformation), the tournament module wouldn't have stopped them. Note that this is true for non-DLL bots and for DLL bots (originally I thought it only affected non-DLL bots, but after investigating and finding the cause of the bug I realized it affects DLL bots too).

#31 point 2 should be addressed after point 7 is fixed. I.E. ensure the Tournament Module allows some permissions for some versions of BWAPI.

After the bug in point 7 is fixed, rebuild the TournamentModule.dll's (for BWAPI 4.2.0 and 4.1.2 at least - I wouldn't bother fixing 4.0.1 Beta, and I wouldn't bother changing 3.7.4 which doesn't allow any permissions) to allow bots to do setLatCom(false) and setCommandOptimizationLevel(a value other than 1, i.e. especially 0 or 2, but perhaps some other authors might want 3 or 4). They don't affect the opponent bot at all - it only affects what info BWAPI returns to you, and some bots rely on them. The default should probably stay as 1, but I think bots should be allowed to set it to 0 and other values if they want to, even though a value of 0 may increase the bot's APM and replay file sizes, because some bots rely on them and currently there is no way for a bot to check what the command optimization level is. And anyway, currently, BWAPI version 4.2.0/4.1.2/4.0.1 Beta bots have always done this until now, due to the bug, so I don't think it's a problem. I suggest to also allow leaveGame(), so that bots can leave the game to indicate they are resigning/surrendering in lost positions, or in bugged states to attempt to avoid crashing later (and this may help address bwapi/bwapi#719
). This might also help increase the overall game rate, so long as Starcraft or bots don't/rarely crash when leaving, because bot-vs-bot endgames can often take a lot of time if the winning bot is weak/slow. Personally, I don't care whether sendText/printf/setTextSize are allowed, so I would probably disallow them on anti-collusion grounds and in case they help to increase performance, but I think the others should be disallowed.

Clients do not update State to RUNNING on low-end machines with 1 CPU core

I set up 2 Windows 10 VMs with 4 GB of RAM and 1 CPU.
I also tried 4 VMs that way.

In both cases there would be one VM hosting a server and one client, while the rest hosts clients. All VMs reside on the same host machine with 12 Cores and 32 GB of RAM. The virtual disks are located on a SSD.

Running with 1 Core and 4 GB RAM caused the client to not set the state to RUNNING, but left it at STARTING after starting the Starcraft Game. The game then runs, but no data is transferred to the server. After about 5 to 10 minutes the client recovers, sets the state to RUNNING and eventually continues with tournament execution.

When setting CPU count to 2 everything runs just fine.

It's not a big issue, but I could run double the amount of VMs if this would work. I assume it has something to do with threads that get paused and only recover when the process eventually receives processing time.

I will have a look into this when I have more time this week.

Game Speed slider setting in game lobbies

From #31 point 3. I.E.:

At least a few months before the competition (so we have time for sufficient training/testing), tell us in the Rules page whether the Game Speed slider in the game lobbies will be set to Normal or Fastest (i.e. LF3 or LF6 respectively). I added a feature request to BWAPI to make organizers' management easier for this (bwapi/bwapi#807). If you want to use "Fastest", I suggest you ask as early as possible whether there are any objections from the big teams like Facebook and Sumsung and other serious competitors, because bots that rely on ML training might need a lot of time/resources to be re-trained.

In the Undermind podcast Episode 18 in the part from 51:03-54:32, Dave (with some comments from Antiga) replied to my question about whether AIIDE 2019 will set the Game Speed slider in the game lobbies to "Normal" or "Fastest". Just need to clarify the Rules page similarly to how CIG worded their rules (here), depending on whether you want "Normal" or "Fastest", then this issue can be closed.

Background: In GitHub on StarcraftAITournamentManager's "ladder" branch, it's configurable and in the default config file, the "lobbyGameSpeed" setting is set to "Normal". If last year's bots are re-entered (e.g. SAIDA and/or CherryPi if they don't get updated - note that Dave mentioned that CherryPi used trained models), AIIDE 2019 may want to stick with "Normal" in case last year's bot versions don't work properly using "Fastest". E.g. CIG 2018 used "Fastest" and Overkill (re-entered without any update from the previous year) seemed to perform badly due to using "Fastest" (refer to the last comment at http://satirist.org/ai/starcraft/blog/archives/678-CIG-2018-Overkill-was-broken.html and http://satirist.org/ai/starcraft/blog/archives/686-CIG-2018-not-only-Overkill-was-broken.html).