saltierl/carball

📈 A Rocket League replay decompiling and analysis library

carball's Issues

Nan data will incorrectly assign zero distance from the ball

This messes up the distance from the ball stats and messes up how we calculate who hit the ball

Add in type and style checking to carball

https://github.com/dbader/pytest-mypy

http://mypy-lang.org/

https://pypi.org/project/flake8/

Stats reset after a player rejoins a game

Player rejoins a game after he disconnected. All his previous stats got wiped. Only new stats after he rejoins are counted.

The stats in the replay shown are zero after the rejoin because he actually did nothing.

Should: not clear old data.

Details:

00:00 - 04:00 Active
04:00 - 04:19 Inactive / DC
04:19 - 05:00 Rejoin / Idle

Player "Glitza"

Replay:
https://cdn.discordapp.com/attachments/493849514680254468/503512015629975553/ISSUE_PLAYER_REJOIN.replay

Get replay playlist properly

Don't use teamsize.

Add in test for boost after goal score

We should make sure that we are not counting boost after a goal is scored.

handle 'user_color_old', looks like an ID like primary and accent colors

carball/carball/json_parser/player.py

Lines 187 to 192 in 9c73c74

	# TODO handle 'user_color_old', looks like an ID like primary and accent colors
	if 'user_color_new' in attribute['value']:
	# rgb integer, 0xAARRGGBB, banners and avatar borders have different default values
	user_color[item_name] = attribute['value']['user_color_new']
	self.paint.append({
	'car': paint.get('body', None),

---

This issue was generated by todo based on a TODO comment in 9c73c74. It's been assigned to @Longi94 because they committed the code.

Create xGoals using ML

https://www.americansocceranalysis.com/explanation/

Divide average_boost_level by 255

USG%

https://www.basketball-reference.com/about/glossary.html

Players with the same name or named ["ball", "game", "zTTL"]

There are issues if players have specific names.
It messes up the Dataframe analysis.data_frame

Can't parse at all:

• Two players with the same name (https://cdn.discordapp.com/attachments/493849514680254468/503509429224865812/PLAYERNAME_TWO_PLAYERS_NAMED_SAME.replay)
• Player named "ball" (https://cdn.discordapp.com/attachments/493849514680254468/503509417577152512/PLAYERNAME_BALL.replay)

Actually "can parse" but series of the df just gets extended, no new series for the player

• Player named "game" (https://cdn.discordapp.com/attachments/493849514680254468/503509425882005506/PLAYERNAME_GAME.replay)
• Player named "zTTL" (https://cdn.discordapp.com/attachments/493849514680254468/503509431342989313/PLAYERNAME_ZTTL.replay)

A parsed replay sanitiser that looks through fields and ensures it parsed properly

should be able to specify fields that are a) must have, b) strong warnings, and c) weak warnings

Potential crash with replay goals not being found

Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2897, in get_loc return self._engine.get_loc(key) File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/hashtable_class_helper.pxi", line 1607, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'boost' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 346, in run_http_function result = _function_handler.invoke_user_function(flask.request) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 217, in invoke_user_function return call_user_function(request_or_event) File "/env/local/lib/python3.7/site-packages/google/cloud/functions/worker.py", line 210, in call_user_function return self._user_function(request_or_event) File "/user_code/main.py", line 109, in parse_replay raise e File "/user_code/main.py", line 105, in parse_replay game_manager, proto_location, pandas_location = decompile_replay(replay_location) File "/user_code/main.py", line 41, in decompile_replay proto_manager = analyze_replay_file(filename, rattletrap_path=dir_path) File "/env/local/lib/python3.7/site-packages/carball/decompile_replays.py", line 53, in analyze_replay_file analysis.create_analysis() File "/env/local/lib/python3.7/site-packages/carball/analysis/analysis_manager.py", line 59, in create_analysis data_frame, kickoff_frames, first_touch_frames) File "/env/local/lib/python3.7/site-packages/carball/analysis/analysis_manager.py", line 83, in perform_full_analysis self.events_creator.create_events(game, proto_game, player_map, data_frame, kickoff_frames, first_touch_frames) File "/env/local/lib/python3.7/site-packages/carball/analysis/events/event_creator.py", line 33, in create_events self.calculate_kickoff_stats(game, proto_game, player_map, data_frame, kickoff_frames, first_touch_frames) File "/env/local/lib/python3.7/site-packages/carball/analysis/events/event_creator.py", line 40, in calculate_kickoff_stats kickoffs = BaseKickoff.get_kickoffs_from_game(game, proto_game, self.id_creator, player_map, data_frame, kickoff_frames, first_touch_frames) File "/env/local/lib/python3.7/site-packages/carball/analysis/events/kickoff_detection/kickoff_analysis.py", line 44, in get_kickoffs_from_game kickoff_player = BaseKickoff.get_player_stats(cur_kickoff, player, data_frame, frame, end_frame) File "/env/local/lib/python3.7/site-packages/carball/analysis/events/kickoff_detection/kickoff_analysis.py", line 63, in get_player_stats kickoff_player.boost = data_frame[player.name]['boost'][end_frame] File "/env/local/lib/python3.7/site-packages/pandas/core/frame.py", line 2980, in __getitem__ indexer = self.columns.get_loc(key) File "/env/local/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2899, in get_loc return self._engine.get_loc(self._maybe_cast_indexer(key)) File "pandas/_libs/index.pyx", line 107, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/index.pyx", line 131, in pandas._libs.index.IndexEngine.get_loc File "pandas/_libs/hashtable_class_helper.pxi", line 1607, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas/_libs/hashtable_class_helper.pxi", line 1614, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'boost'

Create a list of possessions per player

This will essentially be consecutive hits.

Maybe needs to be more sensitive to shots

A shot that I missed should have been considered a shot.

Or maybe we have missedshots that are more sensitive

Give bots a unique id to track them between each game

Each bot should have its own unique id.

Per goal frame stats

Add boost steals

which is large boost taken on opponent side
while ball is on opponent side

Add curve goal detection

These are goals scored in the curve mutator mode but would not make it in as a straight shot.

Big boost pads on non standard maps

Big boost pads only work on standard maps due to the hard coded pad positions. Add support for non-standard maps. See https://gist.github.com/Longi94/5be73367d0682056ee3c2ecbcf70226c (I used RLBot to get the coordinates).

Loadout not being grabbed / set correctly

It's really obvious if you look at any player page, as it 500s. Most replays do not have loadouts correctly set.

Add breakaway goals / counterattack goals

These would goals that start in the defending half and then move to the attacking half either without contention or happen really quickly before a defense can setup

Player not found when trying to get player hits

https://cdn.discordapp.com/attachments/492424177886953503/492857814058795018/52C8E1C84818EB64476A1B8923345DAA.replay

Encoding issue with names

Carball throws an error about latin-1 encoding issues in replays with Unicode names.

Add demo/bump count and "usefull demos/bumps"

physical := demo/bump
I'd love to see a statistic that shows how many physicals a player/team did.

With the recognition of a physical you could also calculate if it was useful (or bad). My suggestions for this:

Offense: Easier to calculate I guess. A goal and a physical happens in a time frame. The physical stopped a player that could have potentially saved the shot.

if( goalHappened() && physicalHappened())
return canSave(player.positions[before], shot) && !canSave(player.position[after], shot)

Defense: You would need a prediction routine probably, that simulates a timeframe if a bump wouldn't have happen. Probably to hard, but suprise me :P

A bad demo is when you bring someone in position to save a ball he couldn't have saved without it.
You could modify my example from earlier. It occurs when a shot was made and saved with a demo in a time window (probably how long it takes to drive field end to end without boost).

if( shotHappened() && demoHappened() && saveHappend())
return canSave(player.positions[after], shot) && !canSave(player.position[before], shot)

Would be really great to see something like this :)

carball 0.3.0 breaks on the replay host joining late (presumably)

https://cdn.discordapp.com/attachments/412423088521740288/490247165046685696/1E9B0E044A80E084534F6381517D5D82.replay

carball.analyze_replay_file('replays/1E9B0E044A80E084534F6381517D5D82.replay', 'replays/1E9B0E044A80E084534F6381517D5D82.replay.json')
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/home/matthew/PycharmProjects/ReplayAnalysis/3.6/lib/python3.6/site-packages/carball/decompile_replays.py", line 68, in analyze_replay_file
    game = Game(loaded_json=_json)
  File "/home/matthew/PycharmProjects/ReplayAnalysis/3.6/lib/python3.6/site-packages/carball/json_parser/game.py", line 64, in __init__
    self.all_data = self.parse_replay()
  File "/home/matthew/PycharmProjects/ReplayAnalysis/3.6/lib/python3.6/site-packages/carball/json_parser/game.py", line 341, in parse_replay
    player_ball_data[player_actor_id][frame_number] = data_dict
KeyError: 5

Track spawn kills

output file

What does the output file look like once you have run carball? and can you show and an example?

Added dribbling carry time

Use bot logic for dribble detection to detect carry time of the ball for dribbles.

Kickoff calculation might fail on replays with network errors

Some replays might have a network error that can cause kickoffs to not have a first touch.
e.g. https://calculated.gg/replays/1681268811E8EDB9F2498C8DA76F5C8B

Add total game time to game_metadata.proto

Should be ['game']['delta'].sum() on non-goal frames.

handle 'user_color_old', looks like an ID like primary and accent colors

carball/carball/json_parser/player.py

Lines 187 to 192 in f39636d

	# TODO handle 'user_color_old', looks like an ID like primary and accent colors
	if 'user_color_new' in attribute['value']:
	# rgb integer, 0xAARRGGBB, banners and avatar borders have different default values
	user_color[item_name] = attribute['value']['user_color_new']
	self.paint.append({
	'car': paint.get('body', None),

---

This issue was generated by todo based on a TODO comment in f39636d when #174 was merged. cc @SaltieRL.

No data dataframe created for a player

Player 0 in the supplied replay file does not get a data dataframe generated for them.

Relevant replay from here because GitHub doesn't like .replays

https://cdn.discordapp.com/attachments/351743823430418432/489670995137789952/kbm_steering_only_left.replay

A second replay showing the issue.
https://cdn.discordapp.com/attachments/174200808475197441/489674423259889674/kbm_steering_right_at_start.replay

Analysis crashed when no players are in game.

Bug: installation ImportError: cannot import name 'game_metadata_pb2'

Hi,

I get the following error trying to install carball from source, both pip install -e . and python init.py creates the same error. I can install with pip install carball

I am macos and python 3.6

Traceback (most recent call last):
  File "init.py", line 17, in <module>
    initialize_project()
  File "init.py", line 14, in initialize_project
    initalize_rattletrap()
  File "init.py", line 2, in initalize_rattletrap
    from carball.rattletrap.check_rattletrap_version import update_rattletrap
  File "/xxx/xxx/toolboxes/carball/carball/__init__.py", line 2, in <module>
    from carball.decompile_replays import decompile_replay
  File "/xxx/xxx/toolboxes/carball/carball/decompile_replays.py", line 4, in <module>
    from carball.analysis.analysis_manager import AnalysisManager
  File "/xxx/xxx/toolboxes/carball/carball/analysis/analysis_manager.py", line 14, in <module>
    from ..analysis.cleaner.cleaner import clean_replay
  File "/xxx/xxx/toolboxes/carball/carball/analysis/cleaner/cleaner.py", line 5, in <module>
    from ...analysis.cleaner.frame_cleaner import remove_invalid_players
  File "/xxx/xxx/toolboxes/carball/carball/analysis/cleaner/frame_cleaner.py", line 5, in <module>
    from ...generated.api import game_pb2
  File "/xxx/xxx/toolboxes/carball/carball/generated/api/game_pb2.py", line 16, in <module>
    from api.metadata import game_metadata_pb2 as api_dot_metadata_dot_game__metadata__pb2
ImportError: cannot import name 'game_metadata_pb2'

Trouble installing package

I'm trying to install carball to an anaconda environment using the instructions in the readme. I keep getting this error:

ERROR: Could not find a version that satisfies the requirement python (from versions: none)
ERROR: No matching distribution found for python

I'm using python 3.7,
can anybody help me?

Not finding hit for goal frame

https://cdn.discordapp.com/attachments/423255214402043905/492987349270003743/B09E3A84459B195EE1416FB689F69410.replay

Detect parties

Use PartyLeader (https://media.discordapp.net/attachments/467421611436277772/482570991894134786/Selection_101.png)

Kickoff goals

Goals that occur < 20 seconds after kickoff

Create list of pressure groups

This will take in a list of possessions and filter into team possessions which will filter into pressure group.
A pressure group is where the ball is on a half for more than 5 seconds and does not leave the half for more then 3 seconds

Duplicated replays when uploaded from multiple players

When two players from the same match (same team for sure) upload a replay of the same match, both files are kept and displayed on the players.

Experienced with Bakkes Mod auto upload

E.g. https://calculated.gg/players/76561198026480940/match_history?page=1&limit=10 28/9/18, uploaded by 0312Penner (my mate) and Ravi(me)

Parse into JSON.

Hey,

I'm not familiar at all with Python and their structure. So this might be more a Python related question than a Carball related question. Hard to tell:

My goal is to parse a replay into a JSON that includes the analysis etc...
Currently I'm able to load the replay perfectly and let it parse it. Now I want to write that data to a .json file. Looked things up and found the json.dumps but it's providing me the following error:
TypeError: Object of type 'Game' is not JSON serializable

I pretty much understand the error but not sure how I would go about on parsing that into JSON.
Any help or clue towards the correct answer is very welcome.

Kind regards,
Pollie

We can gzip .replay files for minor gains

This is like a 1% gain but that can build up as we approach space limits

Type error when parsing certain replay

Replay that is failing:
87B20C2E45C89031C1E19295E6BD8AF4.zip

Seems that there is an issue with certain rotation values.

carball\json_parser\actor_parsing.py
106     data_dict['rot_x'] *= -2 * math.pi / 65535

TypeError: unsupported operand type(s) for *=: 'NoneType' and 'float'

I will take a look at correcting this issue.

Calculate where ball crosses goal line

A minimal parser to get data on failing replays

A way to incrementally add/reparse stats

Pandas KeyError during kickoff analysis

Attempted to analyze this replay: https://cdn.discordapp.com/attachments/594899039355666432/599450806428762113/21DF74C045EEF30284D1339ECEC561D4.replay but carball crashed.

Happy to provide any other info you'd need!

Python stack trace:

Traceback (most recent call last):
  File "stats_init.py", line 10, in <module>
    carball.analyze_replay_file(init_replay_file_path)
  File "<project-folder>\venv\lib\site-packages\carball\decompile_replays.py", line 52, in analyze_replay_file
    analysis.create_analysis()
  File "<project-folder>\venv\lib\site-packages\carball\analysis\analysis_manager.py", line 59, in create_analysis
    data_frame, kickoff_frames, first_touch_frames)
  File "<project-folder>\venv\lib\site-packages\carball\analysis\analysis_manager.py", line 83, in perform_full_analysis
    self.events_creator.create_events(game, proto_game, player_map, data_frame, kickoff_frames, first_touch_frames)
  File "<project-folder>\venv\lib\site-packages\carball\analysis\events\event_creator.py", line 32, in create_events
    self.calculate_kickoff_stats(game, proto_game, player_map, data_frame, kickoff_frames, first_touch_frames)
  File "<project-folder>\venv\lib\site-packages\carball\analysis\events\event_creator.py", line 38, in calculate_kickoff_stats
    kickoffs = BaseKickoff.get_kickoffs_from_game(game, proto_game, self.id_creator, player_map, data_frame, kickoff_frames, first_touch_frames)
  File "<project-folder>\venv\lib\site-packages\carball\analysis\events\kickoff_detection\kickoff_analysis.py", line 40, in get_kickoffs_from_game
    BaseKickoff.set_jumps(kPlayer, player, data_frame, frame, end_frame)
  File "<project-folder>\venv\lib\site-packages\carball\analysis\events\kickoff_detection\kickoff_analysis.py", line 56, in set_jumps
    if collected_boost_df[f] == True:
  File "<project-folder>\venv\lib\site-packages\pandas\core\series.py", line 868, in __getitem__
    result = self.index.get_value(self, key)
  File "<project-folder>\venv\lib\site-packages\pandas\core\indexes\base.py", line 4375, in get_value
    tz=getattr(series.dtype, 'tz', None))
  File "pandas\_libs\index.pyx", line 81, in pandas._libs.index.IndexEngine.get_value
  File "pandas\_libs\index.pyx", line 89, in pandas._libs.index.IndexEngine.get_value
  File "pandas\_libs\index.pyx", line 132, in pandas._libs.index.IndexEngine.get_loc
  File "pandas\_libs\hashtable_class_helper.pxi", line 987, in pandas._libs.hashtable.Int64HashTable.get_item
  File "pandas\_libs\hashtable_class_helper.pxi", line 993, in pandas._libs.hashtable.Int64HashTable.get_item
KeyError: 9434

Crash when rattletrap update was interrupted

Traceback (most recent call last):
  File "C:\Users\laptop3\AppData\Local\Programs\Python\Python37-32\Scripts\carball-script.py", line 11, in <module>
    load_entry_point('carball==0.6.30', 'console_scripts', 'carball')()
  File "c:\users\laptop3\appdata\local\programs\python\python37-32\lib\site-packages\carball\command_line.py", line 56, in main
    manager = carball.analyze_replay_file(args.input)
  File "c:\users\laptop3\appdata\local\programs\python\python37-32\lib\site-packages\carball\decompile_replays.py", line 43, in analyze_replay_file
    _json = decompile_replay(replay_path, output_path=output_path, overwrite=overwrite, rattletrap_path=rattletrap_path)
  File "c:\users\laptop3\appdata\local\programs\python\python37-32\lib\site-packages\carball\decompile_replays.py", line 25, in decompile_replay
    return run_rattletrap.decompile_replay(replay_path, output_path, overwrite, rattletrap_path)
  File "c:\users\laptop3\appdata\local\programs\python\python37-32\lib\site-packages\carball\rattletrap\run_rattletrap.py", line 82, in decompile_replay
    command = create_rattletrap_command(replay_path, output_path, overwrite, rattletrap_path)
  File "c:\users\laptop3\appdata\local\programs\python\python37-32\lib\site-packages\carball\rattletrap\run_rattletrap.py", line 35, in create_rattletrap_command
    binary = get_binary_for_platform(pt.system(), binaries)
  File "c:\users\laptop3\appdata\local\programs\python\python37-32\lib\site-packages\carball\rattletrap\rattletrap_utils.py", line 30, in get_binary_for_platform
    binary = [f for f in binaries if f.endswith('.exe')][0]
IndexError: list index out of range

Per possession stats

Incorrectly give team pinches to only 1 player

Team pinches should be given to 2 players

Currently it is only 1 hit and assigned to the closest player