devonjones/psrd-parser

Parser for the Pathfinder SRD

PSRD Parser

This program parses the text from the Paizo Pathfinder Reference Document website and turns the data into a series of JSON files. It then can transform those JSON files into an SQLite3 database intended for use with the Pathfinder Open Reference project.

Configuration

Any third-party libraries that need to be installed are listed in conf/requirements.txt.

You need to ensure you have a src/dir.conf file, that sets up certain environment variables and is sourced by the shell scripts. A template file is provided: src/dir.conf.template.

Check docs for known issues and notes on the database, among other things.

License

PSRD Parser is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

PSRD Parser is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with PSRD Parser. If not, see http://www.gnu.org/licenses/.

psrd-parser's Issues

Errors

Hi,
first of all thank you very much for this parser :-)

I found these errors.

./run_classes.sh

Core Rulebook: Barbarian
Core Rulebook: Bard
Core Rulebook: Cleric
Traceback (most recent call last):
  File "./class_parse.py", line 34, in <module>
    sys.exit(main())
  File "./class_parse.py", line 31, in main
    exec_main(parser, fxn, "classes")
  File "/home/petr/PSRD-Parser/src/psrd/options.py", line 34, in exec_main
    function(arg, options.output, options.book)
  File "/home/petr/PSRD-Parser/src/psrd/classes.py", line 250, in parse_core_classes
    core_class = parse_class(struct, book)
  File "/home/petr/PSRD-Parser/src/psrd/classes.py", line 211, in parse_class
    cl = stat_block_pass(cl, book)
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/__init__.py", line 47, in stat_block_pass
    newsections.append(stat_block_pass(s, book, no_sb=no_sb))
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/__init__.py", line 45, in stat_block_pass
    newsections.append(parse_stat_block(s, book, no_sb=no_sb))
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/utils.py", line 46, in parse_stat_block
    return function(sb, book)
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/animal_companion.py", line 34, in parse_animal_companion
    animal_companion_parse_function(key)(ac, value)
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/animal_companion.py", line 28, in animal_companion_parse_function
    return functions[field.lower()]
KeyError: u'starting statistics'

./run_feats.sh

Ultimate Combat: Whip Mastery
Traceback (most recent call last):
  File "./feat_parse.py", line 17, in <module>
    sys.exit(main())
  File "./feat_parse.py", line 14, in main
    exec_main(parser, parse_feats, "feats")
  File "/home/petr/PSRD-Parser/src/psrd/options.py", line 34, in exec_main
    function(arg, options.output, options.book)
  File "/home/petr/PSRD-Parser/src/psrd/feats.py", line 127, in parse_feats
    struct = quote_pass(struct)
  File "/home/petr/PSRD-Parser/src/psrd/sections.py", line 77, in quote_pass
    quote_pass(item)
  File "/home/petr/PSRD-Parser/src/psrd/sections.py", line 76, in quote_pass
    for item in section.get('sections', []):
AttributeError: StatBlockHeading instance has no attribute 'get'

./run_rules.sh

--# Description
Bestiary: Introduction
Traceback (most recent call last):
  File "./rules_parse.py", line 21, in <module>
    sys.exit(main())
  File "./rules_parse.py", line 18, in main
    exec_main(parser, fxn, "rules")
  File "/home/petr/PSRD-Parser/src/psrd/options.py", line 32, in exec_main
    function(arg, options.output, options.book, options.title)
  File "/home/petr/PSRD-Parser/src/psrd/rules.py", line 202, in parse_rules
    rules = ability_pass(rules)
  File "/home/petr/PSRD-Parser/src/psrd/sections.py", line 59, in ability_pass
    ability_pass(s)
  File "/home/petr/PSRD-Parser/src/psrd/sections.py", line 59, in ability_pass
    ability_pass(s)
  File "/home/petr/PSRD-Parser/src/psrd/sections.py", line 56, in ability_pass
    section_filter_ability_type(section)
  File "/home/petr/PSRD-Parser/src/psrd/sections.py", line 18, in section_filter_ability_type
    if section.has_key('name'):
AttributeError: StatBlockHeading instance has no attribute 'has_key'

./run_spell_lists.sh

Core Rulebook: 9th-Level Sorcerer/Wizard Spells
Traceback (most recent call last):
  File "./spell_list_parse.py", line 16, in <module>
    sys.exit(main())
  File "./spell_list_parse.py", line 13, in main
    exec_main(parser, parse_spell_lists, "spell_lists")
  File "/home/petr/PSRD-Parser/src/psrd/options.py", line 34, in exec_main
    function(arg, options.output, options.book)
  File "/home/petr/PSRD-Parser/src/psrd/spell_lists.py", line 136, in parse_spell_lists
    struct = parse_universal(filename, output, book)
  File "/home/petr/PSRD-Parser/src/psrd/universal.py", line 376, in parse_universal
    fp = open(filename)
IOError: [Errno 2] No such file or directory: '/home/petr/pfsrd-web/paizo.com/pathfinderRPG/prd/advancedClassGuide/spellLists.html'

./run_spells.sh

Advanced Class Guide: Contingent Scroll
Advanced Class Guide: Creeping Ice
Traceback (most recent call last):
  File "./spell_parse.py", line 21, in <module>
    sys.exit(main())
  File "./spell_parse.py", line 12, in main
    exec_main(parser, parse_spell, "spells")
  File "/home/petr/PSRD-Parser/src/psrd/options.py", line 34, in exec_main
    function(arg, options.output, options.book)
  File "/home/petr/PSRD-Parser/src/psrd/spells.py", line 134, in parse_spell
    struct = stat_block_pass(struct, book)
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/__init__.py", line 52, in stat_block_pass
    return parse_stat_block(section, book, no_sb=no_sb)
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/utils.py", line 46, in parse_stat_block
    return function(sb, book)
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/spell.py", line 48, in parse_spell
    spell_parse_function(key)(spell, value)
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/spell.py", line 42, in spell_parse_function
    return functions[field.lower()]
KeyError: u'or area'

./run_creatures.sh

Bestiary: Gorgon

Bestiary: Gray Ooze
Bestiary: Green Hag

Bestiary: Griffon

Bestiary: Half-Celestial
Traceback (most recent call last):
  File "./creature_parse.py", line 17, in <module>
    sys.exit(main())
  File "./creature_parse.py", line 14, in main
    exec_main(parser, parse_creature, "creatures")
  File "/home/petr/PSRD-Parser/src/psrd/options.py", line 34, in exec_main
    function(arg, options.output, options.book)
  File "/home/petr/PSRD-Parser/src/psrd/creatures.py", line 77, in parse_creature
    struct = stat_block_pass(struct, book)
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/__init__.py", line 45, in stat_block_pass
    newsections.append(parse_stat_block(s, book, no_sb=no_sb))
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/utils.py", line 46, in parse_stat_block
    return function(sb, book)
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/creature.py", line 45, in parse_creature
    parse_creature_descriptors(creature, descriptors)
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/creature.py", line 360, in parse_creature_descriptors
    parse_creature_descriptor(creature, real.pop())
  File "/home/petr/PSRD-Parser/src/psrd/stat_block/creature.py", line 408, in parse_creature_descriptor
    raise Exception('well fuck: %s' %(values))
Exception: well fuck: [u'dragon']

Could you please help me?

I would like to help, but I know nothing about Python... If you write me what to do to fix it, i will try to help you.

Thank you very much.

Best regards,
Petr