wangjohn/updike

An application for writing

updike

An application for writing

updike's Issues

Create IDF part in storage.

Add word dictionary to text processor.

Probably should read something into memory and keep it around (since dictionaries are relatively small).

Create an easy way to write rules for the TextProcessor

Maybe something like:

FilterBy(EndsWith, "ying").try(Word{0,-4}, 'i', 'e')
FilterBy(EndsWith, Consonant{1}, Consonant{1}, "ing").try(Word{0,-4]})
FilterBy(EndsWith, "ing").try(Word{0,-3})

FilterBy(EndsWith, "ied").try(Word{0,-1})
FilterBy(EndsWith, Consonant{1}, Consonant{1}, "ed").try(Word{0,-3})
FilterBy(EndsWith, Consonant{1}, "ed").try(Word{0,-1})
FilterBy(EndsWith, "ed").try(Word{0,-2})

FilterBy(EndsWith, "'s").try(Word{0,-2})
FilterBy(EndsWith, "s'").try(Word{0,-2})

FilterBy(EndsWith, "s").try(Word{0,-1})

Handle apostrophes in contractions

Somehow, we need to handle ' inside of words. The tricky part with this character is that it can double as part of a word and as a punctuation mark. These are most notably used in contractions like don't, he'll, or in possessives like consumer's.

It's potentially the case that we just pre-process these words and remove the '.

Create interface for pre-processing text before it gets added to storage.