iffy/generalconference

Programming-friendly General Conference

This is a repository of programmer-friendly versions of General Conference talks of The Church of Jesus Christ of Latter-day Saints. It is not an official church project and The Church owns the copyright to the content of the talks.

I made this because I often want to do programmatic things with the talks, and don't want to write the downloader/parser for every project.

Pull requests are welcome!

Usage

First, clone this repo, then:

pip install -r requirements.txt
python download.py --help

So, to get the talks for April 2005, this would work:

python download.py 2005 4

Directory structure

The data/ directory contains directories for each language. Inside each language directory, (e.g. data/eng/), there is a directory for each General Conference.

Inside each General Conference directory (e.g. data/eng/2015-10/) you will find:

• An index.yml file with data for the whole conference. Example
• A directory for each talk

Inside each talk directory (e.g. data/eng/2015-10/014-it-works-wonderfully) you will find:

• A metadata.yml file with information about the talk. Example
• A text.md file with the content of the talk in it. Example
• Optionally, a youtube_timing.yml file with information to match the text of a talk with the timing of the YouTube video. Example

generalconference's Issues

no body-block

[CACHE] GET https://www.lds.org/general-conference/1982/04/the-value-of-work?lang=eng 1dc8ad45e1c4fb5b5fc2c75f9a09db78ee616a46
Traceback (most recent call last):
  File "download.py", line 313, in <module>
    args.lang)
  File "download.py", line 159, in getSingleConference
    markdown = extractTalkAsMarkdown(html, meta)
  File "download.py", line 180, in extractTalkAsMarkdown
    content = parsed.xpath('//div[@class="body-block"]')[0]
IndexError: list index out of range

Reference numbers in other languages

The References in other languages use 1,2,3,4 when they should use the language's numbering. For instance, in Apr, 2015 008-the-comforter for Arabic (python download.py 2015 4 --lang=ara) the talk 008-the-comforter looks like this in the "References" section:

## References

  1. ١. [متى ١١:٢٨-٣٠](https://www.lds.org/scriptures/nt/matt/11.28-30?lang=ara#27).

  2. ٢. [موصايا ١٨:٨-٩](https://www.lds.org/scriptures/bofm/mosiah/18.8-9?lang=ara#7).

(I'm not sure Github is handling RTL/LTR the same as in Sublime so I'll attach a screenshot, too)

I'm not actually sure how Markdown plays with RTL languages. Also, "## References" should not be in English.

April 1984 English incomplete

There's a broken link to Elder Perry's talk here: https://www.lds.org/general-conference/sessions/1984/04?lang=eng

When the broken link is fixed, re-download the conference.

April 1988 English incomplete

There's a broken link on the final talk. Once that's fixed, re-download.

Here's the page: https://www.lds.org/general-conference/sessions/1988/04?lang=eng

Some author names have `\_` in them

For example: https://github.com/iffy/generalconference/blob/master/data/ita/2015-10/012-it-works-wonderfully/metadata.yml#L8