Cannot index / search one file

[GoogleCodeExporter]

Cannot index / search one file

[GoogleCodeExporter]

Also, I've been using codesearch as part of a webapp at work that does forensic 
analysis of crashes (by letting us search through backtraces), and it's amazing 
:)

I'm kinda stuck right now because I cannot index some files and I'm thinking 
about using a different indexer / search system, but really codesearch is all I 
need so if someone can figure out what the problem is that would be awesome.

Thanks !!

Original comment by [email protected] on 14 Mar 2013 at 2:00

[GoogleCodeExporter]

Also, I have one line that is crazy long: 2245 characters. Maybe the problem is 
that the indexer reads line by line and has some hardcoded limit on the number 
of chars in a single line ?

Original comment by [email protected] on 14 Mar 2013 at 2:04

[GoogleCodeExporter]

Try indexing with -verbose and -logskip flags to see if the file is getting 
skipped.

The arbitrary limits are in the source so you can always hand edit and tweak 
them. I have a version at

http://github.com/junkblocker/codesearch

which I did to specifically add such options.

Original comment by [email protected] on 14 Mar 2013 at 4:07

[GoogleCodeExporter]

Thanks for the tip. Indeed I've removed those long lines and now everything 
works fine. I've seen that your copy of the code has that -maxlinelen that 
should be what I need. Now I have to understand how to build a go program ...

Original comment by [email protected] on 14 Mar 2013 at 5:57

[GoogleCodeExporter]

Alright, I figured it out, thanks.

repro$ awk '{print length($0)}' badfile | sort -n | tail
972
1001
1043
1071
1456
1529
1724
1792
2259
2328

and in index/write.go there's a 
    maxLineLen      = 2000

Original comment by [email protected] on 14 Mar 2013 at 6:36

[GoogleCodeExporter]

Feel free to close the issue whoever can.

Original comment by [email protected] on 14 Mar 2013 at 6:39

[GoogleCodeExporter]

I'm going to leave this open until I can get something like -logskip into
the mainline codesearch branch.

Original comment by [email protected] on 14 Mar 2013 at 2:08

[GoogleCodeExporter]

I don't know how far you guys should go with that, but having those 2 options 
to set the maxLineLen and maxFileSize on the command line would also help.

The default behavior could be to print a message like that (with a better 
phrasing probably / different options names) when a file got skipped.

=> /tmp/foo wasn't indexed (maxLine too long) / try to reindex with cindex 
-maxLineLen 3000

=> /tmp/foo wasn't indexed (file too big) / try to reindex with cindex 
-maxFileSize 1M

Original comment by [email protected] on 14 Mar 2013 at 4:26