argusdusty/ferret

An optimized substring search engine written in Go

ferret's Issues

How to search unicode text?

How there.
Thank you for awesome package. Is Ferret can be used to index and search unicode text? I mean without UnicodeToASCII conversation?

Query ResultsLimit unlimited doc is incorrect

//     ResultsLimit: Limit the results to some number of values. Set to -1 for no limit
func (IS *InvertedSuffix) Query(Word string, ResultsLimit int) ([]string, []interface{}) {
    Query := IS.Converter(Word)
    Results := make([]string, 0, ResultsLimit)
    Values := make([]interface{}, 0, ResultsLimit)

So setting ResultsLimit to -1 as documented results in make([]string, 0, -1) which results in panic: runtime error: makeslice: cap out of range at runtime.

Various

Please, add info on average & worst complexity in godoc comments for each function.

Also, in godoc comments for "Insert" function, you refer to "MakeInvertedSuffix", which is not present - supposedly that was an old name and you mean the "New" function?

Also, you have lots of deeply nested ifs in your code - you could consider using 'continue' in ifs in loops, to ease reading.

Also, you have quite a few exact repetitions of big chunks of code, I strongly advise to seriously consider extracting them to functions, or please at least comment if that's as is because of Serious Optimizations Totally Backed By Measurements And Not Premature, otherwise (paired with the issue above) they make reading the codebase highly tedious and irritating.

thanks.

examples

Example code didn't build for me until I capitalized the 'f' in 'Ferret' in the import statements. I changed:
"github.com/argusdusty/ferret"
to:
"github.com/argusdusty/Ferret"

dictionaryexample expects DOS line endings

dictionaryexample.go splits the dictionary.dat file by "\r\n" but the dictionary.dat file has only "\n" line endings. Consequently, it fails to build the word table.