odota/core

Open source Dota 2 data platform

core's Issues

Match/player commenting

Add Disqus to match/player pages?

Add team sums to match page

See title

Do something with hero kill data

We have hero kill data from combat log but don't visualize it.

There are also lots of edge cases that should be cleaned up.

These include druid spirit bear, kills by summons, and crediting kills when a creep/tower last hits.

add internal item names to constants

"item_arcane_ring" "Arcane Ring"
"item_armlet" "Armlet"
"item_assault" "Assault Cuirass"
"item_basher" "Skull Basher"
"item_belt_of_strength" "Belt of Strength"
"item_bfury" "Battle Fury"
"item_black_king_bar" "Black King Bar"
"item_blade_mail" "Blade Mail"
"item_blades_of_attack" "Blades of Attack"
"item_blade_of_alacrity" "Blade of Alacrity"
"item_blink" "Blink Dagger"
"item_bloodstone" "Bloodstone"
"item_boots" "Boots of Speed"
"item_boots_of_elves" "Boots of Elvenskin"
"item_bottle" "Bottle"
"item_bracer" "Bracer"
"item_branches" "Iron Branch"
"item_broadsword" "Broadsword"
"item_buckler" "Buckler"
"item_butterfly" "Butterfly"
"item_chainmail" "Chainmail"
"item_circlet" "Circlet"
"item_clarity" "Clarity"
"item_claymore" "Claymore"
"item_cloak" "Cloak"
"item_courier" "Animal Courier"
"item_flying_courier" "Flying Courier"
"item_cyclone" "Eul's Scepter of Divinity"
"item_dagon" "Dagon"
"item_demon_edge" "Demon Edge"
"item_desolator" "Desolator"
"item_diffusal_blade" "Diffusal Blade"
"item_diffusal_blade_2" "Diffusal Blade"
"item_dust" "Dust of Appearance"
"item_eagle" "Eaglesong"
"item_energy_booster" "Energy Booster"
"item_ethereal_blade" "Ethereal Blade"
"item_flask" "Healing Salve"
"item_force_staff" "Force Staff"
"item_gauntlets" "Gauntlets of Strength"
"item_gem" "Gem of True Sight"
"item_ghost" "Ghost Scepter"
"item_gloves" "Gloves of Haste"
"item_greater_crit" "Daedalus"
"item_hand_of_midas" "Hand of Midas"
"item_headdress" "Headdress"
"item_heart" "Heart of Tarrasque"
"item_helm_of_iron_will" "Helm of Iron Will"
"item_helm_of_the_dominator" "Helm of the Dominator"
"item_hood_of_defiance" "Hood of Defiance"
"item_hyperstone" "Hyperstone"
"item_invis_sword" "Shadow Blade"
"item_javelin" "Javelin"
"item_lesser_crit" "Crystalys"
"item_lifesteal" "Morbid Mask"
"item_sphere" "Linken's Sphere"
"item_maelstrom" "Maelstrom"
"item_magic_stick" "Magic Stick"
"item_magic_wand" "Magic Wand"
"item_manta" "Manta Style"
"item_mantle" "Mantle of Intelligence"
"item_mask_of_madness" "Mask of Madness"
"item_mekansm" "Mekansm"
"item_mithril_hammer" "Mithril Hammer"
"item_mjollnir" "Mjollnir"
"item_monkey_king_bar" "Monkey King Bar"
"item_mystic_staff" "Mystic Staff"
"item_necronomicon" "Necronomicon"
"item_necronomicon_2" "Necronomicon"
"item_necronomicon_3" "Necronomicon"
"item_null_talisman" "Null Talisman"
"item_oblivion_staff" "Oblivion Staff"
"item_ogre_axe" "Ogre Club"
"item_orchid" "Orchid Malevolence"
"item_pers" "Perseverance"
"item_phase_boots" "Phase Boots"
"item_pipe" "Pipe of Insight"
"item_platemail" "Platemail"
"item_point_booster" "Point Booster"
"item_poor_mans_shield" "Poor Man's Shield"
"item_power_treads" "Power Treads"
"item_power_treads_str" "Strength"
"item_power_treads_agi" "Agility"
"item_power_treads_int" "Intelligence"
"item_quarterstaff" "Quarterstaff"
"item_quelling_blade" "Quelling Blade"
"item_radiance" "Radiance"
"item_rapier" "Divine Rapier"
"item_reaver" "Reaver"
"item_refresher" "Refresher Orb"
"item_aegis" "Aegis of the Immortal"
"item_cheese" "Cheese"
"item_relic" "Sacred Relic"
"item_ring_of_basilius" "Ring of Basilius"
"item_ring_of_health" "Ring of Health"
"item_ring_of_protection" "Ring of Protection"
"item_ring_of_regen" "Ring of Regen"
"item_robe" "Robe of the Magi"
"item_sange" "Sange"
"item_sange_and_yasha" "Sange and Yasha"
"item_satanic" "Satanic"
"item_sheepstick" "Scythe of Vyse"
"item_shivas_guard" "Shiva's Guard"
"item_skadi" "Eye of Skadi"
"item_slippers" "Slippers of Agility"
"item_sobi_mask" "Sage's Mask"
"item_soul_booster" "Soul Booster"
"item_soul_ring" "Soul Ring"
"item_staff_of_wizardry" "Staff of Wizardry"
"item_stout_shield" "Stout Shield"
"item_talisman_of_evasion" "Talisman of Evasion"
"item_tango" "Tango"
"item_tpscroll" "Town Portal Scroll"
"item_travel_boots" "Boots of Travel"
"item_ultimate_orb" "Ultimate Orb"
"item_ultimate_scepter" "Aghanim's Scepter"
"item_urn_of_shadows" "Urn of Shadows"
"item_recipe_urn_of_shadows" "Urn of Shadows Recipe"
"item_vanguard" "Vanguard"
"item_vitality_booster" "Vitality Booster"
"item_vladmir" "Vladmir's Offering"
"item_void_stone" "Void Stone"
"item_ward_observer" "Observer Ward"
"item_ward_sentry" "Sentry Ward"
"item_wraith_band" "Wraith Band"
"item_yasha" "Yasha"
"item_arcane_boots" "Arcane Boots"
"item_orb_of_venom" "Orb of Venom"
"item_ancient_janggo" "Drum of Endurance"
"item_medallion_of_courage" "Medallion of Courage"
"item_smoke_of_deceit" "Smoke of Deceit"
"item_veil_of_discord" "Veil of Discord"

Change poll() to use a queue and use timers to roughly ensure 1 API request per second.

Group combat log entries by hero units with hero

Entity names can usually be mapped to heroes by the hero name within the unit name.

However, this can be tricky as sometimes it is the third and sometimes the third/fourth sections of the name delimited by "_".

Maybe we need to bruteforce try both.

Prettify the chat log

Add patch info

We probably don't want to store this in the db.

We can define constants for the unix timestamp each patch landed.

Then we compute patch info on the fly.

Redis parse queue

https://github.com/learnboost/kue

Minimize parser processing

To reduce our dependency on the parser we should do as little processing in the parser as possible and just dump data to stdout, which Node can then process and insert into db.

Use object to manage constants update

Should build most of the constants object using parallel's callback with results, and fill the rest from file.

Switch out the current array for an object.

Add chat log history

formatting and on-demand login

Formatting code with ctrl+shift+b.

Only logging into steam client on-demand (provides framework for swapping accounts)

Add ability data

Also switch to using db to store constants

implement cool dynatables/pagination

See title

Track down gold reason numbers

There's a new field in each gold loss/gain entry that gives the reason for that change.

However, I can't find a reference to map those numbers to human-readable reasons. A few can be guessed, but we'll need to do some detective work or the others.

Probably we need to dump out the log and see what else was going on when that gold changed.

fix chat times before 0:00:00

I wrote a custom time function already, it can probably be shared.

Put compressed files on s3

Files uploaded to S3 should be compressed to save space.

Organize view dependencies

We should probably separate our js/css dependencies into reusable block partials and include them as necessary in our template files. There should always be only one reference to a particular file (e.g. jquery, datatable, bootstrap, etc.)

Add rune/ward spawn/placement and usage info

Make timelines look better

Currently, the hero picture is getting covered by items, and the tick on the timeline axis is getting cut off.

Persist sessions so that when the app restarts people don't have to relog

Use this: https://github.com/kcbanner/connect-mongo

Add 500 errors and page for when jade throws errors

Add a callback to res.render: http://expressjs.com/api.html#res.render

Add an env variable that if defined, means it's production and we should hide errors.

Add tower/building info

Multiple accounts to handle >100 replays/day

I've already rewritten the code to support on-the-fly relogging, but it needs to be implemented and somebody needs to make more test accounts :(

Do sums clientside

I think I have a better way of doing this--we can define a class summable on tables we want to sum, then write javascript to sum those tables.

separate parsing from backend

Separate match data into multiple sub pages

heatmaps

http://www.patrick-wied.at/static/heatmapjs/

We'll need to gather the data from the replay. We can do just hero position at intervals, deaths, or something

advanced querying

Allow users to sort/filter their matches.

Filter matches by significant game modes only.

Detect no stats recorded matches and disregard.

Get hero information over time

This code snippet will get hero information at the current time in the match. As far as I can tell, if m_hReplicatingOtherHeroModel equals that constant, it's an illusion.

        import com.google.common.base.Predicate;

        Predicate<Entity> isHeroDT = new Predicate<Entity>() {
            @Override public boolean apply(Entity e) {
                return e.getDtClass().getDtName().indexOf("DT_DOTA_Unit_Hero") == 0;
            }
        };

       Iterator<Entity> heroes = match.getEntities().getAllByPredicate(isHeroDT);

       while(heroes.hasNext()) {
            //Does this work for all matches?
            int num = heroes.next().getProperty("m_hReplicatingOtherHeroModel");
            //IS THIS A CONSTANT?
            if(num == 2097151){
                  count++;
            }
       }

Info returned from each entity:

-- Entity [index=1316, serial=224, dtClass=DT_DOTA_Unit_Hero_Broodmother]
m_vecOrigin = (22.0, 29.0)
m_angRotation[1] = 98.43751
m_cellX = 177
m_cellY = 107
m_iHealth = 651
m_vecOrigin[2] = 0.0
m_cellZ = 130
m_cellbits = 7
m_nNewSequenceParity = 1
m_nResetEventsParity = 1
m_nMuzzleFlashParity = 0
m_flMana = 161.01965
m_nUnitState = 50331648
m_iTaggedAsVisibleByTeam = 26
m_anglediff = 0
m_fFlags = 0
m_NetworkActivity = 422
m_NetworkSequenceIndex = 2
m_nTotalDamageTaken = 0
m_iCurrentXP = 2321
bcc_localdata.m_hMyWeapons.0016 = 2097151
bcc_localdata.m_hMyWeapons.0017 = 2097151
bcc_localdata.m_hMyWeapons.0018 = 2097151
bcc_localdata.m_hMyWeapons.0019 = 2097151
bcc_localdata.m_hMyWeapons.0020 = 2097151
bcc_localdata.m_hMyWeapons.0021 = 2097151
bcc_localdata.m_hMyWeapons.0022 = 2097151
bcc_localdata.m_hMyWeapons.0023 = 2097151
bcc_localdata.m_hMyWeapons.0024 = 2097151
bcc_localdata.m_hMyWeapons.0025 = 2097151
bcc_localdata.m_hMyWeapons.0026 = 2097151
bcc_localdata.m_hMyWeapons.0027 = 2097151
bcc_localdata.m_hMyWeapons.0028 = 2097151
bcc_localdata.m_hMyWeapons.0029 = 2097151
bcc_localdata.m_hMyWeapons.0030 = 2097151
bcc_localdata.m_hMyWeapons.0031 = 2097151
bcc_localdata.m_hMyWeapons.0032 = 2097151
bcc_localdata.m_hMyWeapons.0033 = 2097151
bcc_localdata.m_hMyWeapons.0034 = 2097151
bcc_localdata.m_hMyWeapons.0035 = 2097151
bcc_localdata.m_hMyWeapons.0036 = 2097151
bcc_localdata.m_hMyWeapons.0037 = 2097151
bcc_localdata.m_hMyWeapons.0038 = 2097151
bcc_localdata.m_hMyWeapons.0039 = 2097151
bcc_localdata.m_hMyWeapons.0040 = 2097151
bcc_localdata.m_hMyWeapons.0041 = 2097151
bcc_localdata.m_hMyWeapons.0042 = 2097151
bcc_localdata.m_hMyWeapons.0043 = 2097151
bcc_localdata.m_hMyWeapons.0044 = 2097151
bcc_localdata.m_hMyWeapons.0045 = 2097151
bcc_localdata.m_hMyWeapons.0046 = 2097151
bcc_localdata.m_hMyWeapons.0047 = 2097151
bcc_localdata.m_flNextAttack = 0.0
m_hAbilities.0000 = 1795362
m_hAbilities.0001 = 845089
m_hAbilities.0002 = 445728
m_hAbilities.0003 = 1965343
m_hAbilities.0004 = 1543454
m_hAbilities.0005 = 2097151
m_hAbilities.0006 = 2097151
m_hAbilities.0007 = 2097151
m_hAbilities.0008 = 2097151
m_hAbilities.0009 = 2097151
m_hAbilities.0010 = 2097151
m_hAbilities.0011 = 2097151
m_hAbilities.0012 = 2097151
m_hAbilities.0013 = 2097151
m_hAbilities.0014 = 2097151
m_hAbilities.0015 = 2097151
m_ModifierManager.m_hModifierParent = 460068
m_Inventory.m_hItems.0000 = 121821
m_Inventory.m_hItems.0001 = 1025149
m_Inventory.m_hItems.0002 = 2097151
m_Inventory.m_hItems.0003 = 1049602
m_Inventory.m_hItems.0004 = 2097151
m_Inventory.m_hItems.0005 = 2097151
m_Inventory.m_hItems.0006 = 2097151
m_Inventory.m_hItems.0007 = 2097151
m_Inventory.m_hItems.0008 = 2097151
m_Inventory.m_hItems.0009 = 2097151
m_Inventory.m_hItems.0010 = 2097151
m_Inventory.m_hItems.0011 = 2097151
m_Inventory.m_hItems.0012 = 2097151
m_Inventory.m_hItems.0013 = 2097151
m_Inventory.m_iParity = 12
m_Inventory.m_hInventoryParent = 460068
m_Inventory.m_bStashEnabled = 1
m_Inventory.m_hTransientCastItem = 2097151
m_hMyWearables.lengthproxy.lengthprop20 = 4
m_hMyWearables.0000 = 1826074
m_hMyWearables.0001 = 1260827
m_hMyWearables.0002 = 703772
m_hMyWearables.0003 = 658717
m_hMyWearables.0004 = 0
m_hMyWearables.0005 = 0
m_hMyWearables.0006 = 0
m_hMyWearables.0007 = 0
m_hMyWearables.0008 = 0
m_hMyWearables.0009 = 0
m_hMyWearables.0010 = 0
m_hMyWearables.0011 = 0
m_hMyWearables.0012 = 0
m_hMyWearables.0013 = 0
m_hMyWearables.0014 = 0
m_hMyWearables.0015 = 0
m_hMyWearables.0016 = 0
m_hMyWearables.0017 = 0
m_hMyWearables.0018 = 0
m_hMyWearables.0019 = 0
m_flCreateTime = 0.0
m_ubInterpolationFrame = 1
bcc_localdata.m_hMyWeapons.0003 = 2097151
m_nModelIndex = 312
m_nRenderFX = 0
m_nRenderMode = 0
m_fEffects = 16
m_clrRender = -1
m_iTeamNum = 3
m_CollisionGroup = 9
m_flElasticity = 1.0
m_flShadowCastDistance = 0.0
m_hOwnerEntity = 1599490
m_hEffectEntity = 2097151
moveparent = 2097151
m_iParentAttachment = 0
m_iName =
movetype = 0
movecollide = 0
m_iTextureFrameIndex = 0
m_bSimulatedEveryTick = 1
m_bAnimatedEveryTick = 0
m_bAlternateSorting = 0
m_fadeMinDist = 0.0
m_fadeMaxDist = 0.0
m_flFadeScale = 1.0
m_nMinCPULevel = 0
m_nMaxCPULevel = 0
m_nMinGPULevel = 0
m_nMaxGPULevel = 0
m_nSkin = 0
m_nBody = 0
m_flModelScale = 0.93
m_flPlaybackRate = 1.4404297
m_bClientSideAnimation = 1
m_bClientSideFrameReset = 0
m_bClientSideRagdoll = 0
bcc_localdata.m_hMyWeapons.0004 = 2097151
bcc_localdata.m_hMyWeapons.0005 = 2097151
bcc_localdata.m_hMyWeapons.0006 = 2097151
m_hLightingOrigin = 2097151
m_flFrozen = 0.0
m_hActiveWeapon = 2097151
m_iCurrentLevel = 6
m_iCustomXPValue = 0
m_iIsControllableByPlayer = 128
m_bIsPhantom = 0
m_bIsAncient = 0
m_bStolenScepter = 0
m_bIsNeutralUnitType = 0
m_bIsSummoned = 0
m_bCanBeDominated = 1
m_bHasUpgradeableAbilities = 0
bcc_localdata.m_hMyWeapons.0000 = 2097151
m_flHealthThinkRegen = 8.253967
m_iMaxHealth = 701
m_iAttackCapabilities = 1
m_iDayTimeVisionRange = 1800
m_iNightTimeVisionRange = 800
bcc_localdata.m_hMyWeapons.0007 = 2097151
m_flMaxMana = 364.04443
m_flManaThinkRegen = 2.3809547
m_iUnitNameIndex = 61
m_iDamageMin = 55
m_iDamageMax = 61
m_iDamageBonus = 6
m_lifeState = 0
bcc_localdata.m_hMyWeapons.0008 = 2097151
bcc_localdata.m_hMyWeapons.0009 = 2097151
m_iCurShop = 6
bcc_localdata.m_hMyWeapons.0010 = 2097151
bcc_localdata.m_hMyWeapons.0011 = 2097151
m_Collision.m_nSolidType = 2
bcc_localdata.m_hMyWeapons.0001 = 2097151
m_Collision.m_nSurroundType = 0
m_Collision.m_triggerBloat = 0
bcc_localdata.m_hMyWeapons.0002 = 2097151
m_angRotation[0] = 0.70588684
m_Collision.m_usSolidFlags = 16
bcc_localdata.m_hMyWeapons.0012 = 2097151
bcc_localdata.m_hMyWeapons.0013 = 2097151
m_iBotDebugData = 0
m_bShouldDoFlyHeightVisual = 1
bcc_localdata.m_hMyWeapons.0014 = 2097151
bcc_localdata.m_hMyWeapons.0015 = 2097151
m_flRespawnTime = 0.0
m_flRespawnTimePenalty = 0.0
m_iAbilityPoints = 0
m_flStrength = 29.5
m_flAgility = 29.000004
m_flIntellect = 28.0
m_flStrengthTotal = 29.5
m_flAgilityTotal = 29.000004
m_flIntellectTotal = 28.0
m_iRecentDamage = 0
m_iPlayerID = 7
m_bReincarnating = 0
m_flSpawnedAt = 140.23433
m_hReplicatingOtherHeroModel = 2097151
m_hEconConsumableAbility = 2097151
m_bCustomKillEffect = 0

Allow downloading replays

Aggregate/over-time stats between matches/records

Things like:
Runes
Kills/Deaths/Assists
consumables avg/cumulative
item timings
HD/TD/HH (histograms)
Chat (ggs called/messages, Swearing/profanity analysis)
Grouping of heroes played (by valve groupings/primary attribute)

Use constants in parser

Read the constants file in the parser and use it to produce final output for storage in DB.

Implement mousehover for items/abilities

Add time stamps to console.log statements

Foreman logging just logs stdout. Since we don't output time stamps, the logs aren't really useful for debugging.

parsed data improvements

insert parsed data under its own key, organize parsed data better, update views to work with new format

Calendar of matches

Show a calendar of matches played on days, tinted to show win/loss

Do something with other combat log data

Data includes, damage, healing, gold lost/gained, xp gained, buff/debuff, ability use

Avoid passing full constants and match data in match view

Currently we do this to give our external js access to constants and match data. Can we avoid this to reduce bandwidth?

Improve player pages

so to start with we should just flush out the home page
let's do 1) recent matches (15, no table) 2) top 10 heroes with number matches, win, lost, kda 3) team mates all time 3) team mates this month?

Tests

This is getting bigger, we should probably start writing tests so we don't have regression bugs.

fight recaps

compute fight recaps during replay parse

Add ads!

Create scripts block to avoid including excess JS.

Steam OAuth login

Implement steam login for adding new users

break down damage/healing by inflictor and target

This can possibly help provide statistics for skillshots

handle lone druid edge case

spirit bear log entries should be grouped with druid. This can be achieve through a manual constants insert.

periodically update player display names

Function needs to be implemented.

Cache pages

To reduce processing demand we should cache mostly static pages and serve them.