ggtracker / sc2reader Goto Github PK
View Code? Open in Web Editor NEWExtracts gameplay information from Starcraft II replay files
Home Page: http://pypi.python.org/pypi/sc2reader
License: MIT License
Extracts gameplay information from Starcraft II replay files
Home Page: http://pypi.python.org/pypi/sc2reader
License: MIT License
There are sometimes message events(other players send message) in the replay files. If we can remove them would be better for recoding video.
I don't know is there any way to encode the struct from sc2reader.load_replay back to .replay Binary?
See GraylinKim#148 for the original
Via the new Battle.net API for Starcraft.
I got a report that Stats's Warp Gate was too early in this replay by about 10 seconds
https://lotv.spawningtool.com/45980/
I looked at the replay and code, and it appears that spawningtool couldn't find the chronoboost events with this logic (that has worked int he past)
I thought it might be a data issue, so I went through and exported new balance data to capture events
https://github.com/ggtracker/sc2reader/blob/upstream/sc2reader/data/HOWTO.md
but I noticed that only "Wings of Liberty" was an export option, and the abilities file that I could generate was only 186 lines long (whereas the last one was 408 lines long) https://github.com/ggtracker/sc2reader/blob/329e044fecda0b6c090bdc16568314b8ae752bbc/sc2reader/data/LotV/70154_abilities.csv
and it was indeed missing chronoboost.
I guess my first ask is whether someone else can try the export process in the doc above to see if you can get more results than I can.
If not, then I'm open to ideas about what went wrong and how we fix this
Replays should be analyzed with the new LOTV real time, instead of Blizz time. [bug]
To reproduce:
Expected:
Actual:
File "C:\Users\cyber\AppData\Local\Programs\Python\Python38\lib\site-packages\sc2reader\decoders.py", line 111, in read_string
return self.read_bytes(count).decode(encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 27: invalid start byte
Replays include MMR info. It would be good to load it as a player attribute
The the current new_units.py
and the shell commands listed in the HOWTO
do not preserve the command index of the ability when populating ability_lookup.csv
. This means that the abilities with the same ability link but different command indices end up mapping to the same ability id.
This can be easily verified by loading a replay (with game events) where the player builds a supply depot and a barracks. The supply depot command event will have the correct ability name, but the barracks command event will not.
https://lotv.spawningtool.com/56283/
In this game, there are UnitBornEvents for a Colossus and Phoenix near the end of the game (back-calculated to 9:32) by Vibe. If you watch the game, Vibe didn't built either: both were hallucinated. However, neither were marked as hallucinations by sc2reader.
Interesting, these appear to be the anomaly because 2 earlier Phoenixes are correctly marked as hallucinations, so I'm not sure what the difference is.
#108 (comment) and drop all legacy Python support.
Hello,
I would like to understand the relationship between this project here and blizzards own https://github.com/Blizzard/s2protocol
It seems both sc2replay
and s2protocol
use mpyq in order to open the replay files and both are able to decode the binary contents into Python structures.
From s2protocols acknowledgements it seems to me sc2reader was there first.
From my personal point of view the difference is that sc2reader is actually usable by normal people, while s2protocol is missing even the most basic examples explaining how to use the project and how to understand the replays file contents. Also sc2reader seems to have more contributers.
But it seems maintaining two different binary decoders for one and the same data is quite some overhead. And having blizzard as a company maintaining the "ugly" binary part sounds quite nice to me.
So my question would be, has there ever been a try to merge these two projects in some way? maybe by just importing s2protocol for the binary parsing part and then "just" providing a much nicer user facing interface?
File is 9fd34a64b68c8d8150f59a3c6d87e227ab0cc3d7fe5b8f9d8e403de173fdce29.s2gs, obtainable from depot.
Not sure if it's the EU or US depot, though. Marian, can you tell?
http://ec2-54-243-214-243.compute-1.amazonaws.com/resque/statuses/67895980fad2012fbe8522000a1dea5f
First of all, thanks for maintaining this code. It's a great tool, now that SC2 is becoming the testbed for RTS AI.
There seems to be a bug when printing UnitBornEvents. I loaded a replay (that I link to) and tried to print all the events with a for loop:
replay1 = sc2reader.load_replay('solar1.SC2Replay', load_map=True)
for event in replay1.events:
print(event)
and the first two events (PlayerSetupEvents) print just fine, but once they come to the first UnitBornEvents (the ones that concern mineral fields, for example), I get the following TypeError
:
TypeError Traceback (most recent call last)
in ()
1 for event in replay1.events:
----> 2 print(event)
~/.virtualenvs/StarCraft2_venv/lib/python3.5/site-packages/sc2reader/events/tracker.py in str(self)
292
293 def str(self):
--> 294 return self._str_prefix() + "{0: >15} - Unit born {1}".format(self.unit_upkeeper, self.unit)
295
296
TypeError: non-empty format string passed to object.format
Checking the variables that are to be formatted into the string, it seems that self.unit_upkeeper
is None
for these particular UnitBornEvents
. It may be a bug.
Thanks!
I get this error whenever I try to load a Replay. I tried reinstalling and installing directly from GitHub but the error persists. Any ideas on what could be causing this?
File "<stdin>", line 1, in <module>
File "/home/stefan/Documents/ML/env/lib/python3.6/site-packages/sc2reader/factories/sc2factory.py", line 85, in load_replay
return self.load(Replay, source, options, **new_options)
File "/home/stefan/Documents/ML/env/lib/python3.6/site-packages/sc2reader/factories/sc2factory.py", line 137, in load
return self._load(cls, resource, filename=filename, options=options)
File "/home/stefan/Documents/ML/env/lib/python3.6/site-packages/sc2reader/factories/sc2factory.py", line 146, in _load
obj = cls(resource, filename=filename, factory=self, **options)
File "/home/stefan/Documents/ML/env/lib/python3.6/site-packages/sc2reader/resources.py", line 312, in __init__
engine.run(self)
File "/home/stefan/Documents/ML/env/lib/python3.6/site-packages/sc2reader/engine/engine.py", line 177, in run
for new_event in (event_handler(event, replay) or []):
File "/home/stefan/Documents/ML/env/lib/python3.6/site-packages/sc2reader/engine/plugins/context.py", line 57, in handleTargetUnitCommandEvent
self.last_target_ability_event[event.player.pid] = event
AttributeError: 'NoneType' object has no attribute 'pid'
https://github.com/ggtracker/sc2reader/releases is empty but the old repo used releases.
Hi there
When I tried load a replay by following line
replay = (AlphaStar_MaNa_Game_1.SC2Replay'), it shows index error. This replay is the match between Mana and AlphaStar, it can be downloaded from deepmind. Also, other replays between alphastar and master players cannot be loaded.
I think it is because these matches had some mods?
Thank you
Hey there! Great work!
I just stumbled over this repo while trying to gather some statistics from my own replays.
Looking at #78 I was wondering if you guys would consider setting up automatic testing using for example https://travis-ci.org/ or so. It's free for open source projects.
Continuous testing means, upon certain events (e.g. for every new pull request or once every day) a machine in the cloud downloads and installs your repo and executes the tests.
This makes for example reviewing new PRs very easy for you, since you can be sure a new feature does not break any existing functionality and you can concentrate more on the new feature itself.
Something like a daily check can ensure that your project still works, even if for half a year no person executed the tests. This can prevent you from bad surprises, where you look for the source of an error for hours, while actually some 3rd party library changed and you did not notice.
Also I've seen you support 5 Python versions: 2.7, 3.4 3.5, 3.6 and 3.7. automated testing could install and execute the tests on all of these Pythons for you every day and make sure everything still works, even for 2.7.
I do not ask this just for the fun of it. I was reading through the code and had the impression some stuff could be improved, but I would not like to bother you guys with PRs, as long as there are no tests, which can show, that I did not break anything.
I just forked from https://github.com/ggtracker/sc2reader (the upstream branch) on Windows 10 (Python 3.6.8) and ran test_all.py
The error message is as follows:
morestuff
morestuff
....Unable to change type of None [0] to 686 [frame 2135]; unit type not found in build 38215
test_replays/3.0.0.38215/fourth.SC2Replay
Release String: 3.0.0.38215
Player 1 - Owl (Terran)
Player 2 - dremax (Zerg)
12948 Owl Missing ability 3B20 from Build
Owl 169.6294859359851
.Fetching remote resource: http://EU.depot.battle.net:1119/5984441b6318a28781f01f3efb54c7614af7eb286ae1c068d0f06e471fb40370.s2ma
.DOING 1
Unable to change type of None [240001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [280001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [480001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [4C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [500001] to CollapsiblePurifierTowerDiagonal [frame 0]; unit type not found in build base
Unable to change type of None [780001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [7C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [B40001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [B80001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [DC0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [E00001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [11C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [1200001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [1440001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [1480001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [1580001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [15C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2440001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2480001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [24C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2500001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2540001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2580001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [25C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2600001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2640001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2680001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [26C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2700001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2740001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2780001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [27C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2800001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2840001] to CollapsiblePurifierTowerDiagonal [frame 0]; unit type not found in build base
DOING 2
test_replays/3.1.0/2.SC2Replay
Release String: 3.1.0.39576
Player 1 - OneScotch (Terran)
Player 2 - TubbyJeff (Zerg)
18538 OneScotch Missing ability 47E0 from Build
18976 OneScotch Missing ability 4800 from Build
DOING 3
DOING 4
.Unknown attribute id: 5005
Unknown attribute id: 5005
Unknown attribute id: 3016
Unknown attribute id: 3016
Unknown attribute id: 5002
Unknown attribute id: 5002
Unknown attribute id: 5004
Unknown attribute id: 5004
Unknown attribute id: 5003
Unknown attribute id: 5003
Unknown attribute id: 5001
Unknown attribute id: 5001
Unknown attribute id: 3017
Unknown attribute id: 3017
Unknown attribute id: 5000
Unknown attribute id: 5000
.Unknown attribute id: 5005
.......
Unable to change type of None [2440001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2480001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [24C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2500001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2540001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2580001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [25C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2600001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2640001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2680001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [26C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2700001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2740001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2780001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [27C0001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2800001] to ShakurasVespeneGeyser [frame 0]; unit type not found in build base
Unable to change type of None [2840001] to CollapsiblePurifierTowerDiagonal [frame 0]; unit type not found in build base
Unable to change type of Pylon [5840001] to PylonOvercharged [frame 10843]; unit type not found in build base
.......
test_replays/3.4.0/issueYY.SC2Replay
Release String: 3.4.0.44401
Player 1 - bacon (Protoss)
Player 2 - FlyingCake (Terran)
14 bacon Missing ability 1540 from Build
23 FlyingCake Missing ability 12C0 from Build
184 FlyingCake Missing ability 12C0 from Build
392 FlyingCake Missing ability F81 from Build
417 bacon Missing ability 14A1 from Build
489 bacon Missing ability 1540 from Build
508 FlyingCake Missing ability 12C0 from Build
860 bacon Missing ability 14A3 from Build
888 FlyingCake Missing ability F83 from Build
914 bacon Missing ability 1540 from Build
932 FlyingCake Missing ability F83 from Build
951 FlyingCake Missing ability 12C0 from Build
1102 bacon Missing ability 14A2 from Build
1132 FlyingCake Missing ability 12C0 from Build
1227 bacon Missing ability 1540 from Build
........
Bad pid (4) for event <class 'sc2reader.events.tracker.PlayerStatsEvent'> at 11.20 [10880].
Bad pid (4) for event <class 'sc2reader.events.tracker.PlayerStatsEvent'> at 11.30 [11040].
Bad pid (4) for event <class 'sc2reader.events.tracker.PlayerStatsEvent'> at 11.40 [11200].
Bad pid (4) for event <class 'sc2reader.events.tracker.PlayerStatsEvent'> at 11.50 [11360].
Bad pid (4) for event <class 'sc2reader.events.tracker.PlayerStatsEvent'> at 12.00 [11520].
Bad pid (4) for event <class 'sc2reader.events.tracker.PlayerStatsEvent'> at 12.10 [11680].
Bad pid (4) for event <class 'sc2reader.events.tracker.PlayerStatsEvent'> at 12.20 [11840].
Bad pid (4) for event <class 'sc2reader.events.tracker.PlayerStatsEvent'> at 12.30 [12000].
Bad pid (4) for event <class 'sc2reader.events.tracker.PlayerStatsEvent'> at 12.37 [12114].
x.............
======================================================================
ERROR: test_lotv_creepTracker (__main__.TestReplays)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_replays/test_all.py", line 437, in test_lotv_creepTracker
replay =factory.load_replay(replayfilename,engine=pluginEngine,load_map= True)
File "D:\Anaconda3\envs\StarCraft\lib\site-packages\sc2reader-1.3.1-py3.6.egg\sc2reader\factories\sc2factory.py", line 85, in load_replay
return self.load(Replay, source, options, **new_options)
File "D:\Anaconda3\envs\StarCraft\lib\site-packages\sc2reader-1.3.1-py3.6.egg\sc2reader\factories\sc2factory.py", line 137, in load
return self._load(cls, resource, filename=filename, options=options)
File "D:\Anaconda3\envs\StarCraft\lib\site-packages\sc2reader-1.3.1-py3.6.egg\sc2reader\factories\sc2factory.py", line 146, in _load
obj = cls(resource, filename=filename, factory=self, **options)
File "D:\Anaconda3\envs\StarCraft\lib\site-packages\sc2reader-1.3.1-py3.6.egg\sc2reader\resources.py", line 288, in __init__
self.load_map()
File "D:\Anaconda3\envs\StarCraft\lib\site-packages\sc2reader-1.3.1-py3.6.egg\sc2reader\resources.py", line 405, in load_map
self.map = self.factory.load_map(self.map_file, **self.opt)
File "D:\Anaconda3\envs\StarCraft\lib\site-packages\sc2reader-1.3.1-py3.6.egg\sc2reader\factories\sc2factory.py", line 101, in load_map
return self.load(Map, source, options, **new_options)
File "D:\Anaconda3\envs\StarCraft\lib\site-packages\sc2reader-1.3.1-py3.6.egg\sc2reader\factories\sc2factory.py", line 136, in load
resource, filename = self._load_resource(source, options=options)
File "D:\Anaconda3\envs\StarCraft\lib\site-packages\sc2reader-1.3.1-py3.6.egg\sc2reader\factories\sc2factory.py", line 195, in _load_resource
contents = self.load_remote_resource_contents(resource, **options)
File "D:\Anaconda3\envs\StarCraft\lib\site-packages\sc2reader-1.3.1-py3.6.egg\sc2reader\factories\sc2factory.py", line 179, in load_remote_resource_contents
return urlopen(resource).read()
File "D:\Anaconda3\envs\StarCraft\lib\urllib\request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "D:\Anaconda3\envs\StarCraft\lib\urllib\request.py", line 532, in open
response = meth(req, response)
File "D:\Anaconda3\envs\StarCraft\lib\urllib\request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "D:\Anaconda3\envs\StarCraft\lib\urllib\request.py", line 570, in error
return self._call_chain(*args)
File "D:\Anaconda3\envs\StarCraft\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "D:\Anaconda3\envs\StarCraft\lib\urllib\request.py", line 650, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 400: Bad Request
----------------------------------------------------------------------
Ran 67 tests in 294.889s
FAILED (errors=2, expected failures=3)
I was just wondering if it is possible to find which server a game was played on (Americas/EU/Korea) through replay data. Ideally, I would like to be able to determine the server for any game (including custom games, arcade games, and campaign), but I would be satisfied if I was able to get this information just for ladder games, vs AI, and co-op. If this is not possible through replay data then is there any other way that one might obtain this information?
The following output is from a new test case in ggpyjobs.
The s2gs is originally from http://eu.depot.battle.net:1119/49e7be5d753cc21c236cad4f757540fd283ea65df534566698dfcf454ad5e7f5.s2gs
======================================================================
ERROR: test_s2gs_parse_lioor (sc2parse.tests.SC2ReaderToEsdbTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/david/Dropbox/Programming/esdb/esdb/vendor/ggpyjobs/sc2parse/tests.py", line 156, in test_s2gs_parse_lioor
s2gs = self.get_parsed_s2gs(11)
File "/Users/david/Dropbox/Programming/esdb/esdb/vendor/ggpyjobs/sc2parse/tests.py", line 68, in get_parsed_s2gs
summary = ggfactory.load_game_summary(s2gsfile)
File "/Users/david/Dropbox/Programming/esdb/esdb/vendor/ggpyjobs/src/sc2reader/sc2reader/factories.py", line 92, in load_game_summary
return self.load(GameSummary, source, options, **new_options)
File "/Users/david/Dropbox/Programming/esdb/esdb/vendor/ggpyjobs/src/sc2reader/sc2reader/factories.py", line 137, in load
return self._load(cls, resource, filename=filename, options=options)
File "/Users/david/Dropbox/Programming/esdb/esdb/vendor/ggpyjobs/src/sc2reader/sc2reader/factories.py", line 147, in _load
obj = cls(resource, filename=filename, factory=self, **options)
File "/Users/david/Dropbox/Programming/esdb/esdb/vendor/ggpyjobs/src/sc2reader/sc2reader/resources.py", line 721, in __init__
self.load_translations()
File "/Users/david/Dropbox/Programming/esdb/esdb/vendor/ggpyjobs/src/sc2reader/sc2reader/resources.py", line 803, in load_translations
translation[uid] = sheets[sheet][item]
IndexError: list index out of range
Trying to load replays at load_level=2 and sc2reader errors out with the following stack trace:
Traceback (most recent call last):
File "main.py", line 443, in
main(sys.argv[1:])
File "main.py", line 185, in main
replay_list = list(replays)
File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/factories/sc2factory.py", line 146, in load_all
yield self._load(cls, resource, filename=filename, options=options)
File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/factories/sc2factory.py", line 151, in _load
obj = cls(resource, filename=filename, factory=self, **options)
File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/resources.py", line 301, in init
self._read_data(data_file, self._get_reader(data_file))
File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/resources.py", line 893, in _read_data
self.raw_data[data_file] = reader(data, self)
File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/readers.py", line 274, in call
for i in range(data.read_bits(5))
File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/readers.py", line 274, in
for i in range(data.read_bits(5))
File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/readers.py", line 197, in
for i in range(
File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/decoders.py", line 210, in read_uint32
data = self._buffer.read_uint32()
File "/home/steve/workspace/trueskill/venv/lib/python3.7/site-packages/sc2reader/decoders.py", line 91, in read_uint32
return self._unpack_int(self.read(4))[0]
struct.error: unpack requires a buffer of 4 bytes
Script runs successfully when load_level=0
Here's the relevant output of sc2parse (only the errors):
pastebin
Here are archives of the offending replays:
sc2reader_replays_1.zip
sc2reader_replays_2.zip
sc2reader_replays_3.zip
sc2reader_replays_4.zip
sc2reader_replays_5.zip
http://ggtracker.com/matches/2719434
There are vikings after the 30:00 mark but they don't show up in the graph,
because OwnershpTracker thinks nobody owns them, because
>>> replay.player[2].selection[32000][10].objects[1].name
'VikingAssault'
>>> replay.player[2].selection[32000][10].objects[1].race
'Neutral'
I see the issue has something to do with the unit list naming conventions in sc2reader/data/init.py, but I don't grok the code well enough to know which way to fix it.
And I see that graylinkim/sc2reader has moved forward to a newer approach, however unit_info.json may have the same issue.
Is there a way to link the playerId and UserId together? For replay.game.events they use the network "UserId" instead of the playerId, and I'm interested in tying the replay.game.events to a specific Player.
During unpacking of a replay following error is present:
Problem seems to be present for Zerg and Protoss races.
It needs to be noted that there were couple of recent changes to the game:
The most often used ability would probably be Battery Overcharge on the Protoss side.
There is another log below the one shown here which consists of different ability ID's being missing some of which are repetetive.
[2020-09-03 19:48:00,719][context][37][ERROR] - Y:\Projects\SC2Science\DEMOS\Input\11506451_1566325439_628104.SC2Replay
[2020-09-03 19:48:00,719][context][38][ERROR] - Release String: 4.10.1.75800
[2020-09-03 19:48:00,720][context][45][ERROR] - Player 1 - Nickname_1 (Protoss)
[2020-09-03 19:48:00,728][context][45][ERROR] - Player 2 - Nickname_2 (Terran)
[2020-09-03 19:48:00,728][context][52][ERROR] - 69 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,734][context][52][ERROR] - 2904 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,736][context][52][ERROR] - 3601 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,740][context][52][ERROR] - 6106 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,741][context][52][ERROR] - 7751 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,749][context][52][ERROR] - 8646 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,751][context][52][ERROR] - 9215 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,753][context][52][ERROR] - 10024 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,757][context][52][ERROR] - 11121 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,758][context][52][ERROR] - 11802 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,767][context][52][ERROR] - 13529 Nickname_1 Missing ability 59A0 from Build
[2020-09-03 19:48:00,768][context][52][ERROR] - 13699 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,775][context][52][ERROR] - 17440 Nickname_1 Missing ability 3460 from Build
[2020-09-03 19:48:00,785][context][52][ERROR] - 17954 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,786][context][52][ERROR] - 18247 Nickname_1 Missing ability 59A0 from Build
[2020-09-03 19:48:00,788][context][52][ERROR] - 19039 Nickname_1 Missing ability 3460 from Build
[2020-09-03 19:48:00,795][context][52][ERROR] - 20752 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,812][context][52][ERROR] - 24663 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,817][context][52][ERROR] - 24767 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,819][context][52][ERROR] - 24973 Nickname_1 Missing ability 5980 from Build
[2020-09-03 19:48:00,821][context][52][ERROR] - 25063 Nickname_1 Missing ability 59A0 from Build
[2020-09-03 19:48:00,825][context][52][ERROR] - 26491 Nickname_1 Missing ability 5980 from Build
Different error codes displayed below. Some of them are repetetive.
[2020-09-03 19:48:00,121][context][38][ERROR] - Release String: 4.10.1.75800
[2020-09-03 19:48:00,122][context][45][ERROR] - Player 1 - Nickname_1 (Zerg) Jimmy
[2020-09-03 19:48:00,129][context][45][ERROR] - Player 2 - Nickname_2 (Protoss) Hoplite
[2020-09-03 19:48:00,130][context][52][ERROR] - 957 Nickname_2 Missing ability 5980 from Build
[2020-09-03 19:48:00,134][context][52][ERROR] - 3378 Nickname_2 Missing ability 5980 from Build
[2020-09-03 19:48:00,138][context][52][ERROR] - 4678 Nickname_2 Missing ability 5980 from Build
[2020-09-03 19:48:00,139][context][52][ERROR] - 5131 Nickname_2 Missing ability 5980 from Build
[2020-09-03 19:48:00,141][context][52][ERROR] - 5829 Nickname_2 Missing ability 4340 from Build
[2020-09-03 19:48:00,149][context][52][ERROR] - 5940 Nickname_2 Missing ability 4341 from Build
[2020-09-03 19:48:00,151][context][52][ERROR] - 6088 Nickname_2 Missing ability 5980 from Build
[2020-09-03 19:48:00,153][context][52][ERROR] - 6942 Nickname_2 Missing ability 4340 from Build
[2020-09-03 19:48:00,155][context][52][ERROR] - 7096 Nickname_2 Missing ability 4341 from Build
[2020-09-03 19:48:00,157][context][52][ERROR] - 7703 Nickname_2 Missing ability 4340 from Build
[2020-09-03 19:48:00,158][context][52][ERROR] - 7791 Nickname_2 Missing ability 4341 from Build
[2020-09-03 19:48:00,165][context][52][ERROR] - 8193 Nickname_2 Missing ability 4340 from Build
[2020-09-03 19:48:00,167][context][52][ERROR] - 8612 Nickname_2 Missing ability 4341 from Build
[2020-09-03 19:48:00,168][context][52][ERROR] - 8748 Nickname_2 Missing ability 5980 from Build
[2020-09-03 19:48:00,170][context][52][ERROR] - 9877 Nickname_2 Missing ability 4340 from Build
[2020-09-03 19:48:00,173][context][52][ERROR] - 10857 Nickname_2 Missing ability 5980 from Build
[2020-09-03 19:48:00,174][context][52][ERROR] - 11334 Nickname_1 Missing ability 26A0 from Build
[2020-09-03 19:48:00,176][context][52][ERROR] - 11570 Nickname_2 Missing ability 4340 from Build
[2020-09-03 19:48:00,183][context][52][ERROR] - 11735 Nickname_2 Missing ability 4341 from Build
[2020-09-03 19:48:00,185][context][52][ERROR] - 12377 Nickname_2 Missing ability 4340 from Build
[2020-09-03 19:48:00,186][context][52][ERROR] - 12446 Nickname_1 Missing ability 2320 from Build
[2020-09-03 19:48:00,187][context][52][ERROR] - 12683 Nickname_2 Missing ability 4340 from Build
[2020-09-03 19:48:00,188][context][52][ERROR] - 12956 Nickname_1 Missing ability 2320 from Build
[2020-09-03 19:48:00,190][context][52][ERROR] - 13267 Nickname_1 Missing ability 2320 from Build
[2020-09-03 19:48:00,190][context][52][ERROR] - 13309 Nickname_2 Missing ability 4340 from Build
[2020-09-03 19:48:00,191][context][52][ERROR] - 13509 Nickname_1 Missing ability 2320 from Build
[2020-09-03 19:48:00,198][context][52][ERROR] - 13598 Nickname_2 Missing ability 4340 from Build
This is something totally unexpected:
[2020-09-03 19:48:00,488][objects][75][INFO] - Unknown attribute id: 5005
[2020-09-03 19:48:00,489][objects][75][INFO] - Unknown attribute id: 5005
[2020-09-03 19:48:00,489][objects][75][INFO] - Unknown attribute id: 3020
[2020-09-03 19:48:00,490][objects][75][INFO] - Unknown attribute id: 3020
[2020-09-03 19:48:00,490][objects][75][INFO] - Unknown attribute id: 3016
[2020-09-03 19:48:00,491][objects][75][INFO] - Unknown attribute id: 3016
[2020-09-03 19:48:00,491][objects][75][INFO] - Unknown attribute id: 5002
[2020-09-03 19:48:00,492][objects][75][INFO] - Unknown attribute id: 5002
[2020-09-03 19:48:00,499][objects][75][INFO] - Unknown attribute id: 3019
[2020-09-03 19:48:00,499][objects][75][INFO] - Unknown attribute id: 3019
[2020-09-03 19:48:00,500][objects][75][INFO] - Unknown attribute id: 5004
[2020-09-03 19:48:00,500][objects][75][INFO] - Unknown attribute id: 5004
[2020-09-03 19:48:00,501][objects][75][INFO] - Unknown attribute id: 5003
[2020-09-03 19:48:00,501][objects][75][INFO] - Unknown attribute id: 5003
[2020-09-03 19:48:00,502][objects][75][INFO] - Unknown attribute id: 5001
[2020-09-03 19:48:00,502][objects][75][INFO] - Unknown attribute id: 5001
[2020-09-03 19:48:00,503][objects][75][INFO] - Unknown attribute id: 3017
[2020-09-03 19:48:00,503][objects][75][INFO] - Unknown attribute id: 3017
[2020-09-03 19:48:00,504][objects][75][INFO] - Unknown attribute id: 5000
[2020-09-03 19:48:00,504][objects][75][INFO] - Unknown attribute id: 5000
When using the APMTracker
plugin, it became clear to me that the events that the plugin parses the attribute player
is None
.
Code to reproduce:
class TestPlugin(object):
name = 'TestPlugin'
def handleCommandEvent(self, event, replay):
print(event.player)
replay_file = '/home/rcortini/soft/sc2reader/test_replays/4.7.0.70154/1.SC2Replay'
replay = sc2reader.load_replay(
replay_file,
engine=sc2reader.engine.GameEngine(plugins=[TestPlugin()]))
which outputs a bunch of None
s.
By the way, this also means that the APMTracker
plugin does not work, and that it is very difficult to write functional plugins.
I'd submit a fix myself but I lack the skills to do so
sc2reader/sc2reader/resources.py
Line 184 in 5f189ae
I think here it is a typo that misspells "expansion".
P macro mechanic should be updated to reflect new usage. [feature]
Efficiency: the % of chronoboosted time that buildings are being utilized, e.g. for research, production, or warpgate cooldown.
I could not get the unittests to pass without pip install mpyq.
Is mpyq used at runtime or is it only used in the unittests?
I'd like to know if the unit mapping (unit name to unit type number) are somewhere listed.
The bot APIs for StarCraft 2 use the IDs from the stableid.json file: https://github.com/BurnySc2/python-sc2/blob/develop/sc2/ids/unit_typeid.py#L26
However, when parsing replays with a debugger attached, I can see that the type CommandCenter
has type 39
, however in the stableid.json it is listed as id 18
.
Is there a list somewhere so I can create a mapping between these two?
Super low priority!
Here's an example of a match against the AI:
http://ggtracker.com/matches/523185
s2gs file is:
us.depot.battle.net:1119/982866c4bc5337daffdd2243ad832bb2154d499317fa4b353edba90a80f53564.s2gs
The computer opponent info is not showing up, at least not in the usual way.
Haven't debugged it enough to know if the problem is truly in the sc2reader layer or the layers above it.
The UpdateTargetPointCommandEvent is not being caught, despite there being some code/attempt to do so. I have a fix I implemented on my local copy, but I wanted to make the issue known. Will likely create a pull request with the changes. They are small and shouldn't break anything existing.
As of today there is no way to access the mineral/vespene cost of upgrades and research.
Initiating an upgrade is represented as a BasicCommandEvent, which exposes no details about the ability that was triggered (compare this to UnitInitEvent which exposes the related unit and it's cost).
Is there a way to access the resource cost of the upgrade, or is this feature to be implemeted?
If you would share how/where the cost is fetched for a unit, then I'd be more than happy to implement a similar feature for upgrades.
Bumped into this error
File "/home/kevin/sc2reader/sc2reader/factories/sc2factory.py", line 85, in load_replay
return self.load(Replay, source, options, **new_options)
File "/home/kevin/sc2reader/sc2reader/factories/sc2factory.py", line 143, in load
return self._load(cls, resource, filename=filename, options=options)
File "/home/kevin/sc2reader/sc2reader/factories/sc2factory.py", line 152, in _load
obj = cls(resource, filename=filename, factory=self, **options)
File "/home/kevin/sc2reader/sc2reader/resources.py", line 302, in __init__
self.load_all_details()
File "/home/kevin/sc2reader/sc2reader/resources.py", line 446, in load_all_details
self.load_details()
File "/home/kevin/sc2reader/sc2reader/resources.py", line 435, in load_details
self.length.seconds / GAME_SPEED_FACTOR[self.expansion][self.speed]
KeyError: ''
on this replay. I got the same error on other replays, though I'm not sure it applies to all of them. The entire replay pack is at https://deepmind.com/research/open-source/alphastar-resources
hello guys,
i'm fresh of python and can you tell me after i load the replay, there is not output, where can i see the data of the replay?
cheers,
@miguelgondu wrote this fantastic tutorial on using sc2reader
I want to integrate this more directly into the documentation around, but I'm not quite sure where to wedge it in amongst the README, GitHub wiki, docs, etc.
What do you guys think?
Z macro mechanics should be broken into 2 stats, 'consistency' and 'efficiency'. [feature]
Consistency: % of game time hatches spend injected.
Efficiency: % of total injected time hatches have injects queued.
I just forked from https://github.com/ggtracker/sc2reader (the upstream branch) on Windows 10 (Python 3.6.4) and ran test_all.py
This generates a ton of logs, much of which look like errors, such as:
Whoa! 'CreepTracker' object has no attribute 'creepTracker'
...
Unable to change type of None [4C80001] to SpecialNexus [frame 8345]; unit type not found in build base
...
Conflicting results for Team 1: {'Loss', 'Win', None}
...
Bad upkeep_pid (4) for event <class 'sc2reader.events.tracker.UnitBornEvent'> at 00.00 [0].
....etc.... and at the end there are 4 tracebacks and a status message:
======================================================================
ERROR: test_33_shift_click_calldown_mule (__main__.TestReplays)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_replays/test_all.py", line 512, in test_33_shift_click_calldown_mule
self.assertEqual(len(filter(efilter, replay.events)), 29)
TypeError: object of type 'filter' has no len()
======================================================================
ERROR: test_33_shift_click_spawn_larva (__main__.TestReplays)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_replays/test_all.py", line 518, in test_33_shift_click_spawn_larva
self.assertEqual(len(filter(efilter, replay.events)), 23)
TypeError: object of type 'filter' has no len()
======================================================================
ERROR: test_creepTracker (__main__.TestReplays)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_replays/test_all.py", line 387, in test_creepTracker
assert replay.player[player_id].max_creep_spread[1] >0
AttributeError: 'Participant' object has no attribute 'max_creep_spread'
======================================================================
ERROR: test_lotv_creepTracker (__main__.TestReplays)
----------------------------------------------------------------------
Traceback (most recent call last):
File "test_replays/test_all.py", line 434, in test_lotv_creepTracker
assert replay.player[player_id].max_creep_spread >0
AttributeError: 'Participant' object has no attribute 'max_creep_spread'
----------------------------------------------------------------------
Ran 60 tests in 30.223s
FAILED (errors=4, expected failures=3)
You can find the entire output here: https://pastebin.com/BFCSdSN2
I made a post about this in our Google Group and it was recommended that I log a bug: https://groups.google.com/d/msg/sc2reader/3bTRTrVoGm0/c-Ol8kgNDQAJ
Let me know if you'd like more troubleshooting info!
For example, with 0effa8abed1a61db2f377c213c65334fa62626e2dcceef94d67bb51dc8585560.s2gs,
Traceback (most recent call last):
File "/Users/david/Dropbox/Programming/esdb/esdb/vendor/ggpyjobs/sc2parse/tests.py", line 144, in test_s2gs_parse
s2gs = self.get_parsed_s2gs(3)
File "/Users/david/Dropbox/Programming/esdb/esdb/vendor/ggpyjobs/sc2parse/tests.py", line 65, in get_parsed_s2gs
summary = sc2reader.load_game_summary(s2gsfile)
File "/Users/david/Dropbox/Programming/sc2reader_ggtracker/sc2reader/sc2reader/factories.py", line 92, in load_game_summary
return self.load(GameSummary, source, options, **new_options)
File "/Users/david/Dropbox/Programming/sc2reader_ggtracker/sc2reader/sc2reader/factories.py", line 137, in load
return self._load(cls, resource, filename=filename, options=options)
File "/Users/david/Dropbox/Programming/sc2reader_ggtracker/sc2reader/sc2reader/factories.py", line 147, in _load
obj = cls(resource, filename=filename, factory=self, **options)
File "/Users/david/Dropbox/Programming/sc2reader_ggtracker/sc2reader/sc2reader/resources.py", line 716, in __init__
self.load_player_builds()
File "/Users/david/Dropbox/Programming/sc2reader_ggtracker/sc2reader/sc2reader/resources.py", line 904, in load_player_builds
order_name = self.translations['enUS'][build_item[0][1]]
KeyError: 83886080
If i order just one unit at a time (say, Marine in a Barrack) i have two events:
But when I order multiple marines in a queue i just get one BasicCommandEvent "TrainMarine" and then multiple UnitBornEvents, when every marine is done.
Is it possible to catch "train" event when i order every unit in a queue?
See GraylinKim#125 for the original
Using RFDaemoniac/sc2-map-analyzer as a reference. As we figure out what can be done smaller chunks should be broken out into new feature issues.
graylinkim/sc2reader used coveralls.io for test coverage reports. consider adding the same for this repo
See discussion #82 where @dneise proposed using https://github.com/python/black
I don't have strong feelings about whether we should globally reformat the codebase or if we should just apply it progressively as we're making changes, but we should document and verify code if we decide to implement this.
Loading replays I get
unpack requires a buffer of 4 bytes
on line 91 in decoders.py.
Using the latest version of sc2reader available on PIP, sc2reader chokes when opening co-op replays that use factions besides the standard multiplayer ones. I did not encounter this error previously on version 1.5.0.
Here is a sample replay.
Here is a simple test script that uses that replay to demonstrate the issue:
#!/usr/bin/env python3
import sc2reader
replay = sc2reader.load_replay('./Miner Evacuation (64).SC2Replay', load_level=2)
It gives me the following error when executed:
Traceback (most recent call last):
File "./test.py", line 5, in <module>
replay = sc2reader.load_replay('./Miner Evacuation (64).SC2Replay', load_level=2)
File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/factories/sc2factory.py", line 85, in load_replay
return self.load(Replay, source, options, **new_options)
File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/factories/sc2factory.py", line 143, in load
return self._load(cls, resource, filename=filename, options=options)
File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/factories/sc2factory.py", line 152, in _load
obj = cls(resource, filename=filename, factory=self, **options)
File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/resources.py", line 301, in __init__
self._read_data(data_file, self._get_reader(data_file))
File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/resources.py", line 902, in _read_data
self.raw_data[data_file] = reader(data, self)
File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/readers.py", line 303, in __call__
result = [
File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/readers.py", line 304, in <listcomp>
Attribute(
File "/Users/jcfields/Library/Python/3.8/lib/python/site-packages/sc2reader/objects.py", line 80, in __init__
self.value = lookup[value.strip("\x00 ")[::-1]]
KeyError: 'InfT'
The problem seems to be that the "InfT" (Infested Terran) race used on the Miner Evacuation mission is not in the "attributes.json" file (which only has the standard Terran, Zerg, Protoss, and Random), so its presence in the replay file causes a key look-up error.
I looped through all of my co-op replays and found the following non-standard races that cause read errors:
I don't know if this is an exhaustive list, but it's a pool of over 1,000 co-op replays.
I would guess that either adding these races to "attributes.json" or implementing some sort of fallback to prevent a fatal error if a non-standard race can't be identified would fix this issue.
Edit: Using sc2reader 1.6.0, Python 3.8.5, Mac OS 10.14.6.
Under Documents > Starcraft II > Replays, I am unable to parse replays
The example (in the readme) suggests to write os.environ['SC2READER_CACHE_MAX_SIZE'] = 100
. This does not work (at least for me, Python 3.6.9) : environment variables are supposed to be strings (throws a TypeError).
When set to string, this causes an error in sc2reader because of a comparison between a string and an integer (sc2factory.py
line 315).
https://github.com/ggtracker/sc2reader#installation says 2.6 is supported! My vote would be to drop all legacy Python support.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.