iffy/generalconference

Programming-friendly General Conference

This is a repository of programmer-friendly versions of General Conference talks of The Church of Jesus Christ of Latter-day Saints. It is not an official church project and The Church owns the copyright to the content of the talks.

I made this because I often want to do programmatic things with the talks, and don't want to write the downloader/parser for every project.

Pull requests are welcome!

Usage

First, clone this repo, then:

pip install -r requirements.txt
python download.py --help

So, to get the talks for April 2005, this would work:

python download.py 2005 4

Directory structure

The data/ directory contains directories for each language. Inside each language directory, (e.g. data/eng/), there is a directory for each General Conference.

Inside each General Conference directory (e.g. data/eng/2015-10/) you will find:

• An index.yml file with data for the whole conference. Example
• A directory for each talk

Inside each talk directory (e.g. data/eng/2015-10/014-it-works-wonderfully) you will find:

• A metadata.yml file with information about the talk. Example
• A text.md file with the content of the talk in it. Example
• Optionally, a youtube_timing.yml file with information to match the text of a talk with the timing of the YouTube video. Example

generalconference's Issues

Reference numbers in other languages

The References in other languages use 1,2,3,4 when they should use the language's numbering. For instance, in Apr, 2015 008-the-comforter for Arabic (python download.py 2015 4 --lang=ara) the talk 008-the-comforter looks like this in the "References" section:

## References

  1. ١. [متى ١١:٢٨-٣٠](https://www.lds.org/scriptures/nt/matt/11.28-30?lang=ara#27).

  2. ٢. [موصايا ١٨:٨-٩](https://www.lds.org/scriptures/bofm/mosiah/18.8-9?lang=ara#7).

(I'm not sure Github is handling RTL/LTR the same as in Sublime so I'll attach a screenshot, too)

I'm not actually sure how Markdown plays with RTL languages. Also, "## References" should not be in English.

April 1984 English incomplete

There's a broken link to Elder Perry's talk here: https://www.lds.org/general-conference/sessions/1984/04?lang=eng

When the broken link is fixed, re-download the conference.

Some author names have `\_` in them

For example: https://github.com/iffy/generalconference/blob/master/data/ita/2015-10/012-it-works-wonderfully/metadata.yml#L8

April 1988 English incomplete

There's a broken link on the final talk. Once that's fixed, re-download.

Here's the page: https://www.lds.org/general-conference/sessions/1988/04?lang=eng

no body-block

[CACHE] GET https://www.lds.org/general-conference/1982/04/the-value-of-work?lang=eng 1dc8ad45e1c4fb5b5fc2c75f9a09db78ee616a46
Traceback (most recent call last):
  File "download.py", line 313, in <module>
    args.lang)
  File "download.py", line 159, in getSingleConference
    markdown = extractTalkAsMarkdown(html, meta)
  File "download.py", line 180, in extractTalkAsMarkdown
    content = parsed.xpath('//div[@class="body-block"]')[0]
IndexError: list index out of range