BUG: performance issue when expanded directories are being regenerated

[ghostbuster91]

BUG: performance issue when expanded directories are being regenerated

[ghostbuster91]

after turning on debug logging I can see that whenever I save a file LSP server recompiles some parts of the project which causes a wall of

e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:245: scan_dir_async - start 
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:245: scan_dir_async - start 
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:245: scan_dir_async - start 
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:245: scan_dir_async - start 
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:245: scan_dir_async - start 
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:245: scan_dir_async - start 
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish
e/lua/neo-tree/sources/filesystem/lib/fs_scan.lua:268: scan_dir_async - finish

followed by

...ckages/start/p_nvim-neotree/lua/neo-tree/git/ignored.lua:143: Failed to load ignored files for 
...ckages/start/p_nvim-neotree/lua/neo-tree/git/ignored.lua:143: Failed to load ignored files for 
...ckages/start/p_nvim-neotree/lua/neo-tree/git/ignored.lua:143: Failed to load ignored files for 
...ckages/start/p_nvim-neotree/lua/neo-tree/git/ignored.lua:143: Failed to load ignored files for 
...ckages/start/p_nvim-neotree/lua/neo-tree/git/ignored.lua:143: Failed to load ignored files for 
...ckages/start/p_nvim-neotree/lua/neo-tree/git/ignored.lua:143: Failed to load ignored files for 
...ckages/start/p_nvim-neotree/lua/neo-tree/git/ignored.lua:143: Failed to load ignored files for 
...ckages/start/p_nvim-neotree/lua/neo-tree/git/ignored.lua:143: Failed to load ignored files for 

A lot of directories gets rescanned even though they might have been already collapsed.

[pynappo]

i recently got #1343 merged to main which should handle the errors more gracefully without causing those mass stacktraces. can you switch branches and try it out?

edit: i don't know if this would fix performance in any way, though.

[pysan3]

@Hexdigest123 I don't think your issue is related to this issue.
Your issue is related to UI rerendering of neo-tree, and I believe is caused only when expanding a directory for the first time.
Please make a new issue or please correct me if I'm wrong tho.

OP says there's high CPU usage when a large number of files are updated in the disk, (for example by compiling a project with cmake, a lot of files in build will be regenerated), and neo-tree tries to catch up to those updates. (right? @ghostbuster91)

This is understandable, and we might have to say this is an expected behavior with the current codebase, as neo-tree has to recreate the tree and rerender the whole buffer for each file update (debounced every 50 ms iirc).
And the performance becomes much worse if the directory is a git repo.

This is a big issue, but it is very difficult to fix at the same time. I'm doing a big rewrite and will incorporate with nvim-nio which will hopefully make the situation better, but it will take time.