fonsleenaars / tqdb Goto Github PK
View Code? Open in Web Editor NEWPython parser for the Titan Quest Anniversary Edition, Ragnarok, Atlantis, Eternal Ember DLC database.
Home Page: https://tq-db.net
License: MIT License
Python parser for the Titan Quest Anniversary Edition, Ragnarok, Atlantis, Eternal Ember DLC database.
Home Page: https://tq-db.net
License: MIT License
Hi! I think this is great possibility to have such feature on the site.
Not all people use Windows as OS and it can be mush easier to simply load any interesting data (armor, weapon) in json format from your site.
of Guile
equipment also can be in arms(worrior)
please add arms(worrior) in equipment
The monster infrequents should be relatively simple to index, because they are, as the name implies; infrequent. Only a certain type of monster, or multiple monsters, drop these items.
For the parser; this means returning the monster name, the act the monster appears in, and the chance it has it equipped. It might be interesting to also have an image of the monster, but this would require some new functionality (rendering monster images?).
For the website; the location of the Loot Table already exists so inserting the monster infrequents there should be easy.
Hi.
Please, update pyton-scripts for Titan Quest Ragnarok
Theanks
It would be great for modders to check fast the names, damage and visuals. Thanks again for your awesome work :)
Suggestion from the community forums
Originally posted by Raye_Terse:
Hi, I have some feedback on the Set List that I wanted to share.To start with, I really like the feature, and I use it frequently as a ctrl+f lookup table of sorts.
Thing is, I think there's a pretty easy way to expand the functionality of the list by a lot, especially for newbies.
If the list was sorted by level ranges first and then alphabetically within each level range, it would be a lot easier to use it for discovering items and item sets you want to use with your toon than it is right now. Pretty much any usecase where you don't already know the name of the set you're looking for is improved by this change, simply because you will always know the level of your toon. The only usecase I can think of that is impacted negatively, is one where you know the name of the set you're looking for, but not its level and you can't ctrl+f it for some reason. Personally, I think the tradeoff seems worth it.
So it would look something like this:
Level 1 - 10:
(list of all sets within range)
Level 11 - 20:
(list of all sets within range)
Level 21 - 30:
Etc.
I find a different in database with game of Blacksteel Spike
I thought this Blacksteel Spike drop in last map just only 282 Physical Damage is not very reasonable.
Not conforming to the definition of this equipment
(295 ~ 313 Physical Damage,45% Pierce Ratio) in database
(266~282 Physical Damage, 40% Pierce Ratio) in game.
I believe this new optimization thing about only rendering the stuff the user is looking at is the cause of this issue. I would normally ctrl + f on a certain item type page to find whatever stats I'm looking for, but now it gives 0 results most of the time because the item with those stats aren't being rendered.
Example: go to the shields page, and type "reflect" in the ctrl + f search bar. I got 0 results, despite there being several shields that have this stat.
However, I can easily zoom out until every item is visible and then the search works just fine.
as it is, there is no item requirements yet on all of the item on the page....help?
When going to the monsters tab some scripts hangs and I have to terminate the script (making it thus impossible to swap to loot table or the different difficulties of each monster)
Using waterfox 56.2.2 (64-bit)
Windows 10 pro 64-bit
Hey, I found spears and head gear in the Atlantis DLC with the suffix "of the Tinkerer", which makes it possible to use 2 relics on that item. This suffix is not listed on TQDB.
Some bosses/heroes listed on website as being able to drop a specific item from their loot pool don't appear to actually be able to do so if the item's level is outside of the loot table's level ranges, and it does not appear that the item drop chances listed on the website take this into account. Below, I document evidence for why I think this is the case.
Over the last two weeks, I did about 2000 Hydra chest runs, hoping to get a specific drop from it (see my username). During this time, I did not get any such drops, but I did not get any Aphrodite's Favor drops either. What follows is an estimation of the probability of Aphrodite's Favor not dropping during this time.
Per the website, each item dropped from the Hydra's chest has a 0.077% chance of being an Aphrodite's Favor drop - this may not be completely accurate, but it's good enough for an order-of-magnitude calculation. From the extracted game files, the Hydra loot table is hydra_default_55-57, and using the numSpawnMinEquation and numSpawnMaxEquation attribute values, we estimate that the chest should drop ~9 items each time (rounding down for the sake of a conservative estimate). Therefore, the chance of obtaining at least one Aphrodite's Favor drop each Hydra run is ~ 0.69%.
Given this drop rate for Aphrodite's Favor, the probability that no Aphrodite's Favors are dropped in 2000 Hydra chest runs is then ~ 0.0001% i.e. a one-in-a-million occurrence, which is what happened to me. It's possible that this could just be due to RNG, but I suspect that what's happening here is that it's actually not possible for Aphrodite's Favor to drop from Hydra. I think that this is likely because Aphrodite's Favor has an item level of 80 (according to its itemLevel attribute), and in the legendary amulet drop table used by the Hydra chest (amulet_l01), the maximum possible level of the items that are allowed to drop from this table is 69, per the maxItemLevelEquation value, and as such Aphrodite's Favor can't actually drop from the amulet_l01 table. Some evidence that this is how the min/maxItemLevelEquation work can be found in page 51 of the Grim Dawn modding guide - of course, this is assuming that the Grim Dawn engine has the same underlying mechanics as Titan Quest.
Finally, note that this is just the calculation for Aphrodite's Favor; I spotchecked several other legendary items which did not get dropped during those 2000 runs (e.g. Kraken's Fist, Fingerbone of Boreas), and found that they too, had an item level outside of the max item level specified for their corresponding drop table.
The upshot of all this is: I don't believe it's possible for the Hydra (or other Act 1 bosses) to drop Sapros and other high-level legendaries any more. It may have been possible in the past - I've seen some posts where people mention getting Sapros drops from the Hydra - but I suspect it isn't any more.
If this is correct, then I think it would be useful to filter the bosses/heroes listed on the website as having an item in their drop table down to those who are actually able to drop the item, based on the item level and the maxItemLevelEquation of the loot table the boss/hero is using.
When using throwing weapons, you have to walk up to your enemy more than your character initially does when you target an enemy. Making it pretty frustrating when targeting enemies that are out of reach of your weapon.
Hello,
Aionios doesn't have any droptable - is this thing undroppable or wasn't the droptable transferred correctly?
Best regards
Hi! I've been using your database for a really long period of time, so now, when I have a chance to thank you for this, I would like to it first of all. I think it's a greate idea to sum up every bit of information about this game, thanks for that:) However last 2 mounthes I coudn't load this site, I' ve tried different methodoes(a phone and a pc), but result was the same - google said(for this moment I was using google translator, 'cause my eng is too bad :( ): "The website www.tq-db.net can not establish a connection" just like it is on the screen below. Can you please give me some advise, because, considering the fact, that's there is no such theame here, I deceided the I am doing something wrong. If such theame exists pls sorry me for my inattention
Hi.
I really love the site, but if I want a green legs armor for an example, it is hard to know which prefixes and suffixes that there's even a chance of them having. Therefore not possible for me to plan, if the prefix I want can only occur on a ring or another item that isn't legs armor.
Is it possible to update the affixes to say which kinds of green and yellow items they can occur on? Does anyone even know?
Ps. Otherwise the site is perfect, except I don't quite understand why the Adamantine Sickle of Kronos isn't there along with all the basic non-magical pieces of armor and weapons which are the ones that can have all the affixes.
Today realised new expansion https://steamcommunity.com/games/475150/announcements/detail/1601504362724611319
Will you support this expansion?
The min/max values for the items is determined by so called 'jitter'.
Adding this to the equipment will allow users to see what kind of range they can expect on the items.
For the website, integrating this should be some sort of toggle that adds a highlighted "jitter" indicator. Something along the lines of
Toggled off
10.0 Pierce Damage
Toggled on
10.0 (-/+5.0) Pierce Damage
This would mean a value of -5.0 > +5.0 Pierce damage can be added as jitter.
@Giant2005 found a list of items (#17) that are misnamed:
Some of the items are misnamed too.
Such as the shields: The Glory of Belenus and The Boar of Witham being Battersea Shield and Witham Shield respectively;
the helmets: Gutuater Cap and Peak of Leucutius being Waterloo Ceremonial Cap and Peak of Marne respectively;
the armors: Brynhild's Belt and Teutoburg Tunic being Shieldmaiden's Belt and Celtic Weapon Coat respectively;
the bracelets: Grimhild's Serpents and The Spoils being Spakona Serpent Coils and Bormo's Arm Ring respectively;
the swords: Dagger of Basbona and Sword of the Glauberg Prince being Hochdorf Lord's Dagger and Glauberg Prince's Sword respectively;
and the boots: Esus' Travelling Boots, Fine Gallic Boots, and Völva's Hardened Wraps being Boots of Esus, Celtic Boots, and Völva's Foot Leathers respectively (As an aside, the link to Völva's Hardened Wraps on the site is broken).
Parser
The misnamed items will need to be fixed in the parser. Having the correct names should make it easier to track down where the incorrect names are coming from.
Website
The link issue will need to be fixed in the website, allowing links to have more characters now that accents are involved.
Hi, on the front end of tq-db.net, the sections "Artifacts" and "Formulas & Scrolls" are just plain text, there is no content behind them, because they are not URL's.
I kindly ask you to fix this, because I would like to preview the new scrolls, artifacts and recipes (formulas) from the second addendum "Atlantis".
Thank you in advance,
Sir-Mac
As of now I have found three "impossible" regular green items:
I've tried to send screenshots on e-mail, look for them there.
The drop tables seem to be missing a few bosses' chests. I'm not sure if this is all of them, but these are the ones I noticed:
I noticed because I preferred to farm Sandqueen Masika for Act II loot due to her faster lap time than the other bosses, and I was wondering if the drop tables were good.
I'd also like to say: Thanks you, fonsleenaars, for this service! It's really, really helpful to have up-to-date documentation on this aging game.
Is it possible (in the futur, of course ... not this week ^^)
to add FILTER function for all existing bonus.
For Example :
[Poison Resistance]
Prefixes = Alleviating, Cleansing, Curative, Defiled ................
Suffixes = (nothing)
Armor Head = Agamemnon's Crest, Arubianus Helmet .........
Armor .......
Weapons
Jewelry
.........
The item set only displays 1 of the 4 items.
Summon properties aren't displaying properly on the website yet.
Summons have a time to live, health (this is currently shown as +HP, instead of HP, because of the characterLife
property having a + value.), properties, and potentially some skills.
Whereas the affixes will be added to the database.
It should be able to check one or more affixes and that it serves as a search filter to display only the objects likely to wear them (even if it is very very rare without changing the probabilities)
https://www.tq-db.net/affix/prefixes
https://www.tq-db.net/affix/suffixes
;-)
The loot tables currently only list the rewards from bosses. Expand this list by adding quest rewards to the loot tables.
For the parser this should be relatively simple; implement the same type of parsing of loot tables that already exists for bosses.
For the website, displaying this should list the quest name (and maybe the act?), and either update the look of the loot tables (because more space will be needed), or find a way to integrate it into the current list.
Quest List:
The loot tables currently only list the rewards from bosses. Expand this list by adding quest rewards to the loot tables.
For the parser this should be relatively simple; implement the same type of parsing of loot tables that already exists for bosses.
For the website, displaying this should list the quest name (and maybe the act?), and either update the look of the loot tables (because more space will be needed), or find a way to integrate it into the current list.
I was checking out the responsiveness of your site when I noticed that when on the "Monsters" section of the website, resizing the window (to at least 880px) causes the site to spit out an error and require a refresh.
This same error occurs for other sections of the site (like Weapons/Armor) but at even smaller widths.
In any case, thanks for the site its a great resource!
I have some ideas about your project. Maybe you can realize it.
As the title states, I got a black pearl ring drop off Typhon today. I don't see Typhon listed as dropping it. Don't know if the drop rates are the same on Android as they are on PC/consoles. Especially considering this is just the original game without the expansion. Please disregard if this information is irrelevant to whatever version of TQ you are working on documenting. But since the game is also on mobile devices now, I figure this is the right place. Happy holdays and a merry Christmas!
Hi, fonsleenaars! I first want to thank you again for this excellent resource. It's super-helpful in finding the right bosses to farm as well as how to most efficiently counter them.
I noticed on the Monsters list that the list of attack names don't match up with their effects. For instance, almost every monster has a "Frost Strike" skill as their basic attack. I'm also sure the infamous Banmaru doesn't use Flame Strike and Volcanic Orb, either. Any idea where this is coming from?
"For chest loot, the chance listed is the chance of any ONE item that falls out"
How about multiply that by (numSpawnMaxEquation + numSpawnMinEquation)/2 ?
(In FixedItemLoot dbrs and let numberOfPlayers=1)
The greater artifact "Dvalinn's Simulacrum" is missing information. It does not seem to offer any information on what it actually gives. Ie, the only things in the "properties" tab are "From Arcane Formula - Dvalinn's Simulacrum" and "Grants skill: Dvalinn's Simulacrum (show details)" with the skill not having any details either.
Checklist (note: MI's [ie. Kelpie Circlet through to Bear Rider's Braces] include all difficulties):
Dvalinn's Simulacrum
The Fleeting Memory
Rhodian Sling
Electrum Sling
Teutoburg Tunic
Britomaris' Defiance
Fine Gallic Boots
Kelpie Circlet
Kelpie Vambrace
Bear Rider's Garb
Bear Rider's Leggings
Bear Rider's Bracers
Scroll of the Storm Witches
Scroll of the Sky's Rage
Scroll of the Breaking Wheel
Greater Scroll of the Storm Witches
Greater Scroll of the Djinn
Greater Scroll of the Necromancer
Greater Scroll of the Breaking Wheel
Divine Scroll of the Maddened God
Divine Scroll of the Breaking Wheel
Divine Scroll of the Behemoth
The resources contain multiple languages, adding these should be possible.
Some of the properties are currently hard-coded, these should be updated to rely on language resources instead.
After every 24hr, the big png and bundle.js takes too much time to load on wifi and metered connections on mobile. Both of them combined are >7mb.
I think the cache expires every 24hr in chrome. I tried plugins to increase the cache time but it doesn't work.
If you could increase the expiry time by one week or a month it would be great.
e.g add something like this:
30 DAYS
location ~* .(png|gif|js|css|swf)$ {
add_header Cache-Control "max-age=2592000, public"
}
Some of the issues are unfixable by the parser because the data of the actual files is incorrect. These can be communicated to the development team, but having them organized in one place is pretty useful. Since I run in to them while working on the parser, I figured I'd compile them here, that way anybody else that's on the tq-db.net website that notices something can add them, and if after digging a bit I realize it's an issue with the records and not the parser, I can add them here.
Scroll of Svefnthorn
The Greater Scroll of Svefnthorn is missing because it has the same tag as Divine Scroll of Svefnthorn.
dir: records\xpack2\item\scrolls
file: 02_svefnthorn.dbr
property: description,x2tagScrollName11
Should be: description,x2tagScrollName06
There seem to be affixes in the parser that aren't actually available for any equipment, these should be removed.
I see theres a lot of threads on internet/steam discussions about editing hero (mastery, skill points etc.), we need to use some programmes like TQVault/TQDefiler, but with Ragnarok it's not working correctly. Can u add something like this, or add a character on Helos on each difficulty levels who will do it for like 5 millions (or other large amount of money :D). It will be useful for all who like to combine with their hero.
Sorry for bad english. I translated with google.
You can add sorting by monsters from which items fall.
There will be a section on the side - "monsters". There are three tabs "normal", "epic", "legendary".
Click on the tab.
There appears a list of monsters.
Example for "normal".
(ACT I)
.
.
. Nehebkau ~ Scorpos King
.
.
(ACT II)
.
.
.
After that
Click on some monster.
A list of items(as in "weapons" -> "axe" or another tabs) that can be dropped from the monster appears.
Example for "normal" --> Nehebkau ~ Scorpos King.
"Obsidian Crescent" | "Mikros" | "Raging Bull".........
.........
........
........
Hi. I have a russian fan-site of game (titanquest.org.ua). Can you to extract Database in russian and send me? I would like to add to own site. Can you help me?
Would it be possible to update the documentation and/or source code of what's needed in order to get the parser to run correctly?
When I pull code from the branch and execute run.py, the parser finishes parsing skills and affixes correctly, but as soon as it attempts to parse "weapons, jewelry and armor", I get repeated TextureViewer.exe crashes, even though the Texture Viewer application itself seems to work fine if I try to extract textures manually.
I've checked the Windows Logs to see whether anything useful is logged in the system event logs for the TextureViewer crashes, though the logs don't seem very useful - happy to post samples if you think that will help.
Additionally, I'm also getting errors when trying to parse just "skills", which look like they're happening because I have incompatible versions of some core libraries installed. Any documentation of which versions of the libraries are being used here, or instructions on getting this to run from a clean Python install would be helpful.
run.py -c skills
Determining what files are required...
Parsing skills [####################] DONE
Writing output to files...
Traceback (most recent call last):
File "E:\Python\Anaconda\lib\site-packages\PIL\ImageFile.py", line 482, in _sa
ve
fh = fp.fileno()
AttributeError: '_idat' object has no attribute 'fileno'
During handling of the above exception, another exception occurred:
File "E:\Python\Anaconda\lib\site-packages\PIL\Image.py", line 1930, in save
save_handler(self, fp, filename)
File "E:\Python\Anaconda\lib\site-packages\PIL\PngImagePlugin.py", line 821, i
n _save
[("zip", (0, 0)+im.size, 0, rawmode)])
File "E:\Python\Anaconda\lib\site-packages\PIL\ImageFile.py", line 490, in _sa
ve
e.setimage(im.im, b)
SystemError: tile cannot extend outside image
(same as Steam forum suggestion)
Is it possible to add Affixes Loot Tables ?
For exemple with a 3st tab :
PROPERTIES | LOOT TABLE | AFFIXES (loot table)
or best =
PROPERTIES | LOOT TABLE | PREFIXES | SUFFIXES
An exemple about "Armor"
Sorry, it's in French and since april 2007, images disappeared ... but
http://forum.gamegrob.net/objets-monstres-et-base-d-equipements-f1155/les-armures-t15282.html
It's very useful for the affixes (pre- and suffixes) to list what equipment they apply to.
For the parser this means adding the equipment types (using the category names) to each affix.
For the website this means displaying this information, and allowing some basic filtering on the equipment types.
Is it possible to create DB of monsters? It would be cool.
Some more tweaking of the quest rewards is necessary.
On https://www.tq-db.net/quests some are incorrectly listing percentages, such as:
A Northern Contact (Epic, Legendary)
One Who Would Lead (All)
(sorry for my bad english)
Is it possible to add Affixes Loot Tables ?
For exemple with a 3st tab :
PROPERTIES | LOOT TABLE | AFFIXES (loot table)
https://www.tq-db.net/equipment/black-pearl-ring
and all : Jade / Pythia / Crystal / Truesilver / Black Pearl ...
Some pieces of equipment have unique/specific affixes that apply to them, regardless of their equipment category.
An example of an irregular affix is the Dvergr Greaves, which can have a torso affix:
suffixRandomizerName1,records\xpack\item\lootmagicalaffixes\suffix\tablesarmor\torsomelee_e04.dbr
prefixRandomizerName1,records\xpack\item\lootmagicalaffixes\prefix\tablesarmor\torsomelee_e04.dbr,
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.