skroutz/greek_stemmer

A simple Greek stemming library

GreekStemmer

A simple Greek stemmer algorithm.

This algorithm is based on this paper from George Ntais.

Installation

Add this line to your application's Gemfile:

gem 'greek_stemmer'

And then execute:

$ bundle

Or install it yourself as:

$ gem install greek_stemmer

Usage

In order to use this stemmer you should normalize input. Normalization means two things for this algorithm: detone and upcase.

  require 'greek_stemmer'

  GreekStemmer.stem("ΠΟΣΟΤΗΤΑ") # => "ΠΟΣΟΤΗΤ"

References

• Development of a Stemmer for the Greek Language

Credits

Original work: bandito

Contributing

1. Fork it ( http://github.com//greek_stemmer/fork )
2. Create your feature branch (git checkout -b my-new-feature)
3. Perform changes and run bundle exec rake update_greek_stemming_sample to update the stemming samples
4. Commit your changes (git commit -a)
5. Push to the branch (git push origin my-new-feature)
6. Create new Pull Request

License

greek_stemmer is licensed under MIT License. See LICENSE for details.

greek_stemmer's Issues

"γαλατα" vs "γαλα"

Encoding problems

Ειχαμε ενα προβλημα με τις κωδικοποιησεις:

Traceback (most recent call last):
File "E:/Python/classification/scripts/nsk_fit.py", line 31, in
stemmer = gr_stemm.GreekStemmer()
File "χ\Anaconda3\lib\site-packages\greek_stemmer_init_.py", line 35, in init
custom_rules = self.load_settings()
File "χ\Anaconda3\lib\site-packages\greek_stemmer_init_.py", line 340, in load_settings
custom_rules = yaml.load(f.read())
File "χ\Anaconda3\lib\encodings\cp1252.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 72: character maps to

Το προβλήμα λυνεται ως εξης ( στη περιπτωση μας):
γραμμη 339, file: init.py:
os.path.dirname(file), 'stemmer.yml'), 'r', encoding="utf8") as f:

"φανελακι" does not have the same stem as "φανελα"

Μονόφωτα vs Μονόφωτο

irb(main):023:0> "μονόφωτο".stem
=> "ΜΟΝΟΦΩΤ"
irb(main):024:0> "μονόφωτα".stem
=> "ΜΟΝΟΦΩ"

"μουσαμάδες" has wrong stem

GreekStemmer.stem("ΜΟΥΣΑΜΑΔΕΣ") # => "ΜΟΥΣΑΜΑΔ"

GreekStemmer.stem("ΜΟΥΣΑΜΑΣ") # => "ΜΟΥΣΑΜ"

"ρολόϊ" has the same stem as "ρολό" or "ρόλοι"

ΑΔ-ΗΣ # ΑΔ-Ω

Probably, there is a problem here: giving the words ΑΔΩ and ΑΔΗΣ, the stemming process has as output the stem ΑΔ?

"*ινος" words have bad stems

GreekStemmer.stem("ΒΡΑΔΙΝΟΣ") # => "ΒΡΑΔΙΝ" instead of "ΒΡΑΔ"

"πληκτρολογιο" does not have the same stem as "πληκτρολογια"

"τετραδα" has the same stem as "τετραδιο"

"κολάν" has the same stem as "κολικός"

"μπουφαν" has the same stem as "μπουφες"