Enhancement: raw search mode for performance boosting

[vim-ctrlspace]

Enhancement: raw search mode for performance boosting

[szw]

Yeah, the fuzzy search is not the fastest one here. Therefore, a huge boost you may notice with Ruby bindings: https://github.com/szw/vim-ctrlspace#gctrlspace_use_ruby_bindings. It's easy to get Vim with compiled-in Ruby bindings on Mac or Linux. I'm not sure how to proceed with Windows environment. Moreover, I'd love to have such description in the docs ;).

Also, perhaps you should try to puzzle with https://github.com/szw/vim-ctrlspace#gctrlspace_search_timing and provide values improving the smoothness of searching. These values are configurable because every setup is different and the overall perfomance depends on computer internals, OS, project size, etc.

Please, try to tweak it a bit, to see if you get any improvements. Let me know whether that helped you or not.

[stephencheng]

I am using osx 10.9, the vim is macvim with ruby 1.8.7 compiled-in. Guess the setting: gctrlspace_use_ruby_bindings is already used.

I tried tuning the gctrlspace_search_timing from [50,500] to [500,2000], it definitely helped a little. Two problems still:

1. The project base is big, there are over 90,000 files there. So there are hiccups between reissuing new search, eg add a few letters or remove letters. Too much or too less of the time delay both will impact user experience.
2. It seems the fuzzy logic produces lots of searched result items actually not quite relevant,eg, if I search, controllerjava => expecting: xxxxController.java (don't know how to do wildcard/regex in ctrlspace), it gives:

[stephencheng]

Ok, I found a way for raw search, but need a little hack on it, it's workable and worth the try.

workflow:

1. key short cut to open the file list file, eg: project/.hg/cs_files, put this to my tab1 as [file_list]
2. do some other work
3. if i need to search, then I go to tab1 [file_list], use vim to search / or ? with regex
4. use tricks from http://vim.wikia.com/wiki/Open_file_under_cursor to open files

[szw]

Are those irrelevant results the only ones? I think it should give you the desired xxxxController.java at the bottom. The files from the screenshot match the results too, but I guess on far higher positions. Hmm, perhaps the list could be trimmed, because it does not make sense to display them anyway. Just I'm not sure how does it impact the performance, because to display and navigate in a Vim window is rather cheap.

If you are on mac, then perhaps you should try the newest Ruby (I'm not sure how to provide it for MacVim) but the terminal Vim with Ruby can be automatically compiled with the Homebrew. The newest Ruby 2.1 is a lot faster than 1.8.7, and so all expensive regex operations will be perfomed much faster too.

[stephencheng]

I am happy with what current search has provided and will consider using it
for only small/middle project.
Is it possible to add this raw search feature? Basically it is just to
render a file and provide a key mapping, is it?

I was trying to explore if i can come of this plugin as a ctrlspace
companion but I found it will take me quite some time to learn vim script
and it might not worth the effort.

I am just a ruby beginner but without vim script guess pure ruby won't help
too much.

On Sat, May 17, 2014 at 5:24 PM, Szymon Wrozynski
[email protected]:

Are those irrelevant results the only ones? I think it should give you the
desired xxxxController.java at the bottom. The files from the screenshot
match the results too, but I guess on far higher positions. Hmm, perhaps
the list could be trimmed, because it does not make sense to display them
anyway. Just I'm not sure how does it impact the performance, because to
display and navigate in a Vim window is rather cheap.

If you are on mac, then perhaps you should try the newest Ruby (I'm not
sure how to provide it for MacVim) but the terminal Vim with Ruby can be
automatically compiled with the Homebrew. The newest Ruby 2.1 is a lot
faster than 1.8.7, and so all expensive regex operations will be perfomed
much faster too.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/29#issuecomment-43400119
.

[stephencheng]

All right, I learnt something during exploring solutions.

Wondered why there was no such a silly plugin, because you can simply do:
:r find . xxx => for whatever you want, then
use gf to jump to the file
or use: :g!/xxx/d to fileter out unwanted file

need a tmp file to be saved to play this

[szw]

It's a bit more complicated to provide the raw search feature, here in CtrlSpace. I must admit, project with 100k files is a beast. I've generated 2 sample projects to tests, with 10k and 100k files. I did some measurements and decided to redesign the search feature a bit, introducing some limits. I'll prepare a pull request soon.

[szw]

Ok, check out this branch: https://github.com/szw/vim-ctrlspace/tree/search_and_file_limits. I guess it should help a bit. Besides that, I still see a bit room for performance improvements, but first, please check how does it work with real projects.

[stephencheng]

I did some tests on the new branch.

It's positive on "searching", for example, my time delay settings are [500, 2000], instead of feeling deadly slowness and no response from the cursor prompt, now it's more predictable and quick for returning searching result. However I am not able to use the figure of how many milliseconds the performance has improved in searching.

A few negative items:

1. the fuzzy search now returns incorrect result, eg, my 90k base file list returns 170+ found files matching "*Controller.java", however it gave me only 15+ and less than 20 result
2. there is still need to improve the responsiveness when typing the search term, pulse, finish typing, reissue additional characters. Will it be easy to add a icon or hint message in the status bar to indicating that ctrlspace is in waiting for input, about to start searching, finish searching, done output etc?
3. one thing I could not understand, there are times the move up/down line of search result, it will become freezing for a sec or some millisecs.

[szw]

ad 1) That was the goal. And a lot of its responsiveness comes from that truncation or less relevant results. If you want to seem more tweak the g:ctrlspace_max_search_results value, or be more descriptive with your search letters.

ad 2) Yeah, no miracles here. 90k files needs to be processed after all. I could think about moving the whole process to Ruby. Perhaps that could speed up things. Perhaps some asynchronous processing could be introduced here as well. But right now the Ruby support is not mandatory. And to keep Ruby as an option only makes this whole redesign a lot harder.
The icons or sth telling about what the plugin is doing are interesting option.

3. Yeah, it's hard to tell what's going on. Perhaps Vim or underlying Ruby interpreter are freeing some resources or sth...

[szw]

Ok, I've moved a whole bunch of logic to ruby, to increase search speed. Looks promising IMHO. Please, check the latest updates on this branch: 84e715d

[stephencheng]

Yes, the performance boost is obvious.

however I noticed a issue that with default settings of
g:ctrlspace_max_files
g:ctrlspace_max_search_results
there is nearly no response if you want to do delete a char or back space, it will be just stuck for quite sometime, then if you hit a few backspace, all of sudden, the plugin window will be canceled.

But it behaves differently with settings of no limit of below:

let g:ctrlspace_max_files = 0
let g:ctrlspace_max_search_results = 0

Both typing search chars or back space is so smooth with these two settings. I don't understand why, but I love the no limit, the significant drawback for me currently would be only the loading time in this mode - 2 secs vs 500 millsecs.

[szw]

You're right. The adjusting didn't work properly since the number of displayed items was not a valid reference to compute the timing. I've improved it, so now the timings should behave nicely.