numirias/semshi

🌈 Semantic Highlighting for Python in Neovim

semshi's Issues

Async request caused an error: Invalid buffer id

Environment

• NVIM v0.4.0-1395-g3566267e7
• vim-fugitive: 645406f
• semshi: 527ebc5

Reproduce

1. nvim -u test_fugitive.vim
2. :UpdateRemotePlugins and restart by nvim -u test_fugitive.vim
3. :e ./semshi/setup.py
4. <Space>gi and <enter>
5. You will see the error information
   see video

Sorry, I tried it, but I didn't find a way to directly reproduce the problem. I need to use fugitive to reproduce this problem, but I think this is related to semshi, so I report the problem here.

test_fugitive.vim

set runtimepath+=./vim-fugitive
set runtimepath+=./semshi

nnoremap <silent> <Space>gi :Git add -p %<CR>

Possible glitch in class method with f-string

Could be a duplicate of #31 . However, in my case, I don't have any format specifiers.

#!/usr/bin/env python3
import logging


class SomeClass:
    def demo(self, name):
        security_group = []
        logging.warning(
            f"Security group {security_group} not working"
            f"foo bar {name}."
        )

c = SomeClass()
c.demo("world")

How to change the default red highlight

Hello, Thank you for your work on this, it's useful. I'm struggling to figure out how to change the red highting when you select or hover over a var. I think it's a bit obtrusive and want to make it more subtile. Thing is I don't know which of the params controls this:

function MyCustomHighlights()
hi semshiImported ctermfg=red
endfunction
autocmd FileType python call MyCustomHighlights()

Any ideas on how to tweak this?

Thanks!

Correctly highlight numeric literals containing an underscore

I just started using this, and absolutely love it! The only minor bug I have noticed is that numeric literals with underscores are not highlighted the same was as those without.

Not major at all, just letting you know. Thanks!

Doesn't work after "set background=dark"

Semshi stop working after I set background color, i have to restart neovim to get it work back.

cursor word highlight error

Vim: use text properties?

Is Vim 8 supported?
No. Semshi relies on Neovim's fast highlighting API to quickly update lots of highlights. Regular Vim unfortunately doesn't have an equivalent API. (If you think this can be implemented for Vim 8, let me know.)

You could use text properties in Vim probably.

Pseudo / Python code (from my WIP for jedi-vim using it to highlight usages):

if not vim_prop_type_added:
    vim.eval("prop_type_add('jediUsage', {'highlight': 'jediUsage'})")
    vim_prop_type_added = True

vim_prop_add(lnum, start_col+1, {
    'type': 'jediUsage',
    'bufnr': buf.number,
    'length': length,
})

JFI.
As for Neovim there are nice changes planned using tree-sitter for 0.5. Are you aware of them?

Keep highlighting when there are syntax errors

All the highlights are cleared when a syntax error detected. I don't know if there is a way to keep those highlights. I'm using a language which is basically Python but has some additional syntax. Semshi doesn't recognize those new syntax and stop highlighting. I only use Semshi for import and function arguments highlighting.

Thank you.

Cannot jump back after ":Semshi goto <anything>"

Vim has a jump stack which can be used to jump to the previous location, as shown here:
https://vim.fandom.com/wiki/Jumping_to_previously_visited_locations
This works by default when using tag files and jumping between tags, and also with some Vim movements (like going to the end of file).
When using :Semshi goto name or the other goto functions, the jump stack is not updated, so I cannot jump back to the original location with the default Ctrl+O mapping.

Make semantic highlighting available via LSP

Let's investigate if semantic highlighting can be provided via LSP any time soon.

• LSP implementations for neovim are being actively discussed (neovim/neovim#5522) and clients exist (e.g. LanguageClient-neovim).
• An extension of LSP for semantic highlighting is also being discussed (microsoft/language-server-protocol/issues/18).
• However, the latest protocol implementation proposal (microsoft/language-server-protocol/pull/124) seems a bit too limited. While we could work with the provided IDs and modifiers for coloring, we couldn't port features like highlighting related nodes at the cursor position. (Also, how would that affect performance?)
• It will probably take quite some time from the protocol specification to the implementation in a neovim LSP client.

Running inside Oni

As reported by @CrossR, running Semshi inside Oni raises an error when opening a Python file:

        if not self._options.active: # pylint: disable=protected-access
AttributeError: 'NoneType' object has no attribute 'active'

The cause seems to be that Semshi does some initialization on BufEnter, however Oni doesn't always trigger that event first, so Semshi may be uninitialized when a Python file is opened.

Highlighting disabled after writting

When I save changes in .py files (with :w), semshi highlighting is disabled until the next edit. Is it the expected behavior? If yes, is there a way to have it always enabled?

Thanks!
(PS: your plugin is great, thank you)

E117: Unknown function: SemshiBufWipeout

After I PluginInstall each time, key press q to quit, I will get a message from neovim:
Processing BufWipeout autocmd "*" occurs error:
E117: Unknown function: SemshiBufWipeout

Work on Darwin, Neovim:
NVIM v0.3.7
Build type: Release
LuaJIT 2.0.5

vim-plug:
Plug 'numirias/semshi', { 'do': ':UpdateRemotePlugins', 'for' :['python', 'vim-plug'] }

pynvim.api.common.NvimError

Every time I try to open a .py file I get the following error:
Error detected while processing function <SNR>59_filetype_changed[4]..remote#define#CommandBootstrap[5]..remote#define#request: line 2: error caught in request handler '/root/.config/nvim/plugged/semshi/rplugin/python3/semshi:command:Semshi [['enable']]': Traceback (most recent call last): File "/root/.config/nvim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 126, in cmd_semshi func(self, *args[1:]) File "/root/.config/nvim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 29, in wrapper self._init_with_vim() File "/root/.config/nvim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 61, in _init_with_vim self._options = Options(self._vim) File "/root/.config/nvim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 269, in __init__ val = vim.vars.get('semshi#' + key, val_default) File "/usr/local/lib/python3.6/dist-packages/pynvim/api/common.py", line 138, in get return self.__getitem__(key) File "/usr/local/lib/python3.6/dist-packages/pynvim/api/common.py", line 110, in __getitem__ raise transform_keyerror(exc) File "/usr/local/lib/python3.6/dist-packages/pynvim/api/common.py", line 108, in __getitem__ return self._get(key) File "/usr/local/lib/python3.6/dist-packages/pynvim/api/nvim.py", line 182, in request res = self._session.request(name, *args, **kwargs) File "/usr/local/lib/python3.6/dist-packages/pynvim/msgpack_rpc/session.py", line 102, in request raise self.error_wrapper(err) pynvim.api.common.NvimError: Key 'semshi#excluded_hl_groups' not found

Then after neovim started, syntax highlighting is not working.
I also reproduced this error with a minimal config of just semshi.

Different python version in neovim python3 host and virtualenv python

Consider a case where g:python3_host_prog is set to /usr/bin/python (3.5.x), and the current shell (e.g. $PATH) uses another python from virtualenv or anaconda, with different version (3.6.x).

In this case, Syntax from python 3.6+ is not supported. e.g.

   99 var = 1
E>100 print(f'{var}')

Output of :Semshi error is

Syntax error: invalid syntax (100, 14)

Is this a neovim-related issue? Can we configure python being used in code checking other than neovim's host python (or autodetect from $PATH)?

KeyError: 18446744073709551615

I'm running into an issue that's possibly related to neovim/pynvim#310

Error detected while processing function remote#define#notify:
line    6:
E475: Invalid argument: Channel doesn't exist

and the logs

2018-07-17 21:51:37,048 [INFO @ session.py:request:99] 36041 - 'Received error: [1, b"Key 'semshi#excluded_buffers' not found"]
2018-07-17 21:51:37,049 [INFO @ session.py:request:99] 36041 - 'Received error: [1, b"Key 'semshi#excluded_hl_groups' not found"]
2018-07-17 21:51:37,049 [INFO @ session.py:request:99] 36041 - 'Received error: [1, b"Key 'semshi#mark_selected_nodes' not found"]
2018-07-17 21:51:37,050 [INFO @ session.py:request:99] 36041 - 'Received error: [1, b"Key 'semshi#error_sign' not found"]
2018-07-17 21:51:37,050 [INFO @ session.py:request:99] 36041 - 'Received error: [1, b"Key 'semshi#error_sign_delay' not found"]
2018-07-17 21:51:37,050 [INFO @ session.py:request:99] 36041 - 'Received error: [1, b"Key 'semshi#always_update_all_highlights' not found"]
2018-07-17 21:51:37,051 [INFO @ session.py:request:99] 36041 - 'Received error: [1, b"Key 'semshi#tolerate_syntax_errors' not found"]
2018-07-17 21:51:37,051 [INFO @ session.py:request:99] 36041 - 'Received error: [1, b"Key 'semshi#update_delay_factor' not found"]
2018-07-17 21:51:37,051 [INFO @ session.py:request:99] 36041 - 'Received error: [1, b"Key 'semshi#self_to_attribute' not found"]
2018-07-17 21:51:37,119 [WARNING @ async_session.py:_on_message:83] 36041 - Traceback (most recent call last):
  File ".venv/lib/python3.6/site-packages/neovim/msgpack_rpc/async_session.py", line 80, in _on_message
    self._handlers.get(msg[0], self._on_invalid_message)(msg)
  File ".venv/lib/python3.6/site-packages/neovim/msgpack_rpc/async_session.py", line 101, in _on_response
    self._pending_requests.pop(msg[1])(msg[2], msg[3])
KeyError: 18446744073709551615

This happens basically all the time when moving around in a file.

How does it compare to vim-jedi

How does semshi compare to vim-jedi beside that semshi is just for syntax highlighting whereas jedi does completion, showing docs and many more?

Of cause that are huge differences, but I'm curious if there might be more than that. Maybe a small comparison in the FAQ would be great.

Thank you very much.

Doesn't support files without extension

semshi looks great, but unfortunately it doesn't work for many of my scripts/programs, which are stored with a file extension (i.e. myprog vs. myprog.py). When I try to enable these with semshi, I get:

Error detected while processing function remote#define#request:
line    2:
Semshi doesn't currently handle this file. (match pattern: "*.py")

However, it's perfectly valid on *nix to write a Python program with a shebang line and not use an extension (and in practice it seems many Pythong scripts do).

Could you add support for this please?

Add goto unresolved and goto unused

Thanks for the great plugin!

I suggest adding a goto unresolved and goto unused functions to Semshi.

My usecase is the following: I have some complicated code, which produces and endresult x. I move this code into a function f and replace it in the original context by x = f(). I then use Semshi to scan (visually) which varibles are unresolved in the complicated code, these I then provide as arguments to f. To make sure only the endresult x is needed I use semshi to scan for any unresolved variables in the original context, these are probably intermediate values from the original code. I also use semshi to identify unused variables, these were probably used in the original complicated code and should again be arguments for f.

Currently I'm using the syntax highlighting to visually identify unused and unresolved variables, but directly using semshi to goto the next unresolved/unused variable would be much quicker and more reliable.

To implement this I think the following changes would be necessary:

1. Keep track of unresolved/unused nodes (like errors)
2. Implement function that goes to the next unresolved/unused nodes

I'm not super sure where (codewise) 1. would fit in well. As far as I see the errors are recorded when the parser encounters a syntax error, but there the type of node is not yet available. I guess _parse would be the correct place?

2. should be very straightforward with implementations that basically clone _goto_error().

If you think these additions would be valuable to Semshi I can send a PR with the changes outlined above. I'm of course also happy if you implement it yourself. Just implementing it yourself and not dealing with my implementation might in the end even take less time for you.

Jumps on input

I get some jumpy behavior when I edit text and have a syntax error as it disappears and reappears. Do other folks see this? Should I be trying to disable other plugins to look for interactions? I'm using let g:deoplete#auto_complete_delay = 100.

Semshi stop working after source vimrc file.

When I am having vim running with Semshi and source the vimrc file with :so ~/.config/nvim/nvim.init, Semshi stop running.
No other commands, eg :Semshi can reenable Semshi. I have to restart vim for Semshi to work again.

This is a little annoying if I have long working session open, or task running in neovim's terminal.

Get a list of all the words in the unresolved group

Hello, I want to transition from writing the infamous from foo import * to from foo import bar, lol, etc. My idea is to type first the wildcard import, have semshi find those unresolved functions and classes, get a list of them and typing them into from foo import bar, lol, etc. Is this possible? Thanks.

Expressions with format specifiers in f-strings have incorrect offsets

def test(something):
    # "somethi" and "ng:b" are in different color.
    # it should be that "something" and ":b" are different color.
    return f'{something:b}'

Neovim daily fails to UpdateRemotePlugins for semshi

Neovim depreciates the neovim remote plugin in favor of pynvim. On the latest NVIM v0.3.2-dev `UpdateRemotePlugins' does not work for semshi.

Encountered ModuleNotFoundError loading plugin at /home/beavis/.vim/bundle/semshi/rplugin/python3/semshi:
No module named 'neovim'
Traceback (most recent call last):
  File "/home/beavis/.virtualenvs/neovim3/lib/python3.6/site-packages/pynvim/plugin/host.py", line 135, in
 _load
    module = imp.load_module(name, file, pathname, descr)
  File "/home/beavis/.virtualenvs/neovim3/lib/python3.6/imp.py", line 245, in load_module
    return load_package(name, filename)
  File "/home/beavis/.virtualenvs/neovim3/lib/python3.6/imp.py", line 217, in load_package
    return _load(spec)
  File "<frozen importlib._bootstrap>", line 684, in _load
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
ModuleNotFoundError: No module named 'neovim'

Here is my output for ':checkhealth'
https://gist.github.com/BonaBeavis/b4afcc37f71363b1a58608919450ff4c ( the // in the paths are from my removed username)

Delayed event handling (highlights, via timers?)

Currently the plugin triggers a lot of (RPC) events, e.g. on CursorMoved:

semshi/plugin/semshi.vim

Lines 85 to 88 in 801d0c9

	autocmd TextChanged <buffer> call SemshiTextChanged()
	autocmd TextChangedI <buffer> call SemshiTextChanged()
	autocmd CursorMoved <buffer> call SemshiCursorMoved(line('w0'), line('w$'))
	autocmd CursorMovedI <buffer> call SemshiCursorMoved(line('w0'), line('w$'))

I think it would be nice to specify some timeout for this, e.g. something short like 50ms, which would then avoid calling the Python plugin for when holding j etc.

This could be easily done using timers and checking if the cursor position was not changed when it fires again.

I'd be happy to create a PR, but wanted to ask for feedback first.

I think it makes sense for the CursorMoved* event, but also TextChanged* maybe.

Instead of CursorMoved* CursorHold* could be used also, but that is typically quite long (I have 750ms, the default is 4s or something like that).

A downside would be that it would still highlight for the old position until the next update. For this it could still explicitly clear highlights initially.

Support from foo import *

I know it's already stated in the README that currently Semshi does not support highlighting module from from foo import *.

But is there any progress on this aspect, is it possible to support that in Semshi as well?
My current solution is to add unresolved to the non-highlights group, but I would love to see it supported.

Thanks for the great plugin.

Issue with booleans in python3.8

Running neovim 0.4.3 and python 3.8, I am having an issue where semshi apparently stops working after trying to parse a True or False constant. This wasn't a problem with python 3.7. A simple example is:

class A:
    def __init__(self):
        self.a = 1
        self.b = True
        self.c = 2

The image below illustrates what happens when this is typed out, observe how 'self' is correctly italicized up until True, but is not italicized thereafter. And obviously the wonky highlighting on True itself.

The only highlighting which works thereafter is due to my python syntax config rather than semshi.

Unkown function: SemshiBufWipeout when closing windows in python3.8

When I close windows, for example, after updating plugins using PlugUpdate command provided by vim-plug, I saw the following errors:

Error detected while processing BufWipeout Autocommands for "*":
E117: Unknown function: SemshiBufWipeout
E117: Unknown function: SemshiBufWipeout
E117: Unknown function: SemshiBufWipeout

This is my vimrc

Plug 'numirias/semshi', { 'do': ':UpdateRemotePlugins', 'for' :['python', 'vim-plug'] }

I try to use :UpdateRemotePlugins manually ,but did not work

Cleanup and implications of async_call() usage

Thanks to #26, async API calls from threads are now implemented correctly. This may resolve some problems and allow some additional cleanups.

• Maybe proper unit test is required? (not sure)

  @wookayin I don't think so, the current tests should already catch that.

• Should we prevent creating wrapper function instance every time?

  @wookayin I don't think it has an impact on performance. However, is there a particular reason why you preferred the wrapper function over calling nvim.async_call() directly?

• Is it still necessary to use call_atomic() in small batches or does #26 remediate that issue?

• It seems now that that we schedule API calls correctly, there are some interactions with other plugins. E.g., I observed that using Semshi with deoplete now slows it down significantly. Is there something we can do about it?

Annotations are not highlighted before positional-only slash

In the code below, the int should be highlighted as a builtin, but it's not highlighted at all.

def foo(a:int, /):
	pass

Error detected while processing function remote#define#request:

Hi! I get this when I start neovim (0.2.2) and the latest semshi:

Error detected while processing function remote#define#request:                                                                                                                                
line    2:                                                                                                                                                                                     
no request handler registered for "/home/bz/rc.arch/bz/.vim/plugged/semshi/rplugin/python3/semshi:autocmd:BufEnter:*.py"

And then this:

error caught in async handler '/home/bz/rc.arch/bz/.vim/plugged/semshi/rplugin/python3/semshi:autocmd:CursorMoved:*.py []'                                                                     
Traceback (most recent call last):                                                                                                                                                             
  File "/home/bz/rc.arch/bz/.vim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 13, in wrapper                                                                                         
    if not self._options.active: # pylint: disable=protected-access                                                                                                                            
AttributeError: 'NoneType' object has no attribute 'active'

Unresponsive to syntax setting

Behavior: If semshi is turned on and the syntax setting for the current buffer changes either by syntax off or ownsyntax off (or similar), the highlighting doesn't disappear.

Wanted: Highlighting should disappear, as it is syntax highlighting.

f-strings interpolation is not highlighted as code

Hello, I noticed that the new format strings are not highlighted using semchi:

This is what I have using semchi:

bar is not highlighted differently from the string around it, it's not clear that there is an interpolation or not here.
When the interpolation has some special words (var not defined, self, ..), it is highlighted though, but it's minimal:

This is what I had using the plugin python-syntax only:

It's slightly better

I think the braces should be highlighted like normal parentheses, and the interpolation code like normal code if possible.

---

There are similar issues about incorrect highlighting offsets in f-strings, but I think this is different, it's more about the interpolation code that is highlighted like a string and not as code.

syntax error while UpdateRemotePlugin

Encountered SyntaxError loading plugin at /Users/yuhui/.local/share/nvim/plugged/semshi/rplugin/python3/semshi: invalid syntax (handler.py, line 234)

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/neovim/plugin/host.py", line 130, in _load
    module = imp.load_module(name, file, pathname, descr)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/imp.py", line 245, in load_module
    return load_package(name, filename)
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/imp.py", line 217, in load_package
    return _load(spec)
  File "<frozen importlib._bootstrap>", line 696, in _load
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "/Users/yuhui/.local/share/nvim/plugged/semshi/rplugin/python3/semshi/handler.py", line 234
    (id, line, name, self._buf_num), async=True)

Future AST changes may require different calculation of highlight offsets

Later than 3.7.0, col_offset for some async statements in the AST may be calculated differently, so maybe adjusting highlights becomes necessary.

https://bugs.python.org/issue31902

Not working on python 3.6 and earlier

I still don't have python 3.7 in my distro, and commit 7116e72dcb7baeb27165cd5cf8384e192596902 introduces async_ keyword which triggers an error on pre-3.7 pythons. Would there be a way to keep semshi compatible with all 3.xxx pythons?

error caught in async handler '/home/cbosdo/.config/nvim/plugged/semshi/rplugin/python3/semshi:autocmd:CursorMoved:*.py []'                                                                             
Traceback (most recent call last):
  File "/home/cbosdo/.config/nvim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 21, in wrapper
    func(self)
  File "/home/cbosdo/.config/nvim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 81, in event_cursor_moved
    self._update_viewport()
  File "/home/cbosdo/.config/nvim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 181, in _update_viewport
    self._cur_handler.viewport(start, stop)
  File "/home/cbosdo/.config/nvim/plugged/semshi/rplugin/python3/semshi/handler.py", line 55, in viewport
    self._add_visible_hls()
  File "/home/cbosdo/.config/nvim/plugged/semshi/rplugin/python3/semshi/util.py", line 19, in wrapper
    res = func(*args, **kwargs)
  File "/home/cbosdo/.config/nvim/plugged/semshi/rplugin/python3/semshi/handler.py", line 169, in _add_visible_hls
    self._add_hls(nodes_to_hl(visible))
  File "/home/cbosdo/.config/nvim/plugged/semshi/rplugin/python3/semshi/util.py", line 19, in wrapper
    res = func(*args, **kwargs)
  File "/home/cbosdo/.config/nvim/plugged/semshi/rplugin/python3/semshi/handler.py", line 254, in _add_hls
    [('nvim_buf_add_highlight', (buf, *n)) for n in node_or_nodes])
  File "/home/cbosdo/.config/nvim/plugged/semshi/rplugin/python3/semshi/handler.py", line 274, in _call_atomic_async
    self._vim.api.call_atomic(calls[i:i + batch_size], async_=True)
  File "/usr/lib/python3.6/site-packages/neovim/api/nvim.py", line 140, in request
    res = self._session.request(name, *args, **kwargs)
  File "/usr/lib/python3.6/site-packages/neovim/msgpack_rpc/session.py", line 86, in request
    .format(', '.join(kwargs.keys())))
ValueError: request got unsupported keyword argument(s): async_

Make use of buffer update events

The Neovim API now supports Buffer update events. This means we don't need to guess anymore where in the buffer code was changed to infer where we need to update highlights.

We should make use of these events as a replacement for TextChanged and simplify the parser accordingly.

Lag when scrolling

I've recently installed semshi to get some better highlighting for python, but i've noticed that it made neovim lag an insane ammount when scrolling. I've isolated the problem to semshi, as uninstalling it got everything back to normal. Any reasons that could be causing this?

Windows Install

Hi!

Wanted to give semshi a try out, but it looks like nvim has an issue with finding this plugin on Windows.

At a guess, I'd say its the symbolic link in the rplugins/python3/ folder....but when I removed that and stuck the files directly in there I got a bunch of errors about attempted relative import with no known parent package.

I guess to tick off the other bits -

• :CheckHealth reports no issues with python3.
• I've got some other plugins that require it too (ie 'mhartington/nvim-typescript').
• My linux box with the same setup has no issues recognising Semshi. (Same Python ver + Nvim ver).
• Manually running :UpdateRemotePlugins doesn't find Semshi either, nor does having it run that command after installing with plug-vim.

Anything else I can help with let me know!

Unknown function: SemshiBufWipeout

Hi, after a PlugUpdate I'm getting the following when opening python files. I had to remove the plugin to keep working.

How to disable/enable from vimrc/init.vim semshi highligting?

Hi.

I wrote (literally found and rewrite) power save function, which switches on some fancy features on power and switches off on a battery.

Is it possible to run Semshi enable or Semshi disable from .vimrc?

function! s:powersafe_mode(timer)
    let on_battery = s:check_battery()
    if on_battery != s:restricted
        let s:restricted=on_battery
        let g:ale_lint_on_text_changed=s:restricted ? 'never' : 'always'
	if s:restricted == 1
		setl updatetime=4000
		" call :Semshi("disable") ?
	else
		setl updatetime=400
		" call :Semshi("enable") ?
	endif
	call coc#config('git', {
		\ 'addGlameToVirtualText': s:restricted ? v:false: v:true,
		\ 'enableGutters': s:restricted ? v:false: v:true
		\})
    endif
endfunction

Disable reparsing in insert mode

How to disable reparsing in insert mode?

I've tried

    autocmd InsertEnter *.py Semshi disable
    autocmd InsertLeave *.py Semshi enable

but this disables syntax completely, all I want is to stop parsing till I get to normal mode.

Broken with neovim python-client 0.3.0: request from non-main thread

On startup After opening a python file, the following error occurs. Both on neovim 0.2.1 and 0.3.1.

request from non-main thread:
  File "$HOME/.dotfiles/vim/plugged/semshi/rplugin/python3/semshi/handler.py", line 242, in _update_hls
    self._add_hls(nodes_to_hl(add))
  File "$HOME/.dotfiles/vim/plugged/semshi/rplugin/python3/semshi/util.py", line 19, in wrapper
    res = func(*args, **kwargs)
  File "$HOME/.dotfiles/vim/plugged/semshi/rplugin/python3/semshi/handler.py", line 254, in _add_hls
    [('nvim_buf_add_highlight', (buf, *n)) for n in node_or_nodes])
  File "$HOME/.dotfiles/vim/plugged/semshi/rplugin/python3/semshi/handler.py", line 274, in _call_atomic_async
    self._vim.api.call_atomic(calls[i:i + batch_size], async_=True)

Installing neovim==0.2.6 is a workaround until it is fixed.
neovim API has breaking changes... See #25 (comment)

Issues with use of ellipsis (...) in FastAPI based service (Python 3.8)

Although I appreciate Semshi isn't yet tested for 3.8, I'd been using 3.8 for a few of the new features and figured I'd try it out. Sadly I ended up with a traceback.

From the FastAPI documentation - https://fastapi.tiangolo.com/tutorial/query-params-str-validations/#make-it-required :

from fastapi import FastAPI, Query

app = FastAPI()


@app.get("/items/")
async def read_items(q: str = Query(..., min_length=3)):
    results = {"items": [{"item_id": "Foo"}, {"item_id": "Bar"}]}
    if q:
        results.update({"q": q})
    return results

Trying to enable Semshi on this causes a traceback:

Error invoking '.../numirias/semshi/rplugin/python3/semshi:command:Semshi' on channel 9 (python3-rplugin-host):
error caught in request handler '.../numirias/semshi/rplugin/python3/semshi:command:Semshi [['enable']]':
Traceback (most recent call last):
  File ".../numirias/semshi/rplugin/python3/semshi/plugin.py", line 139, in cmd_semshi
    func(self, *args[1:])
  File ".../numirias/semshi/rplugin/python3/semshi/plugin.py", line 38, in wrapper
    func(self, *args, **kwargs)
  File ".../numirias/semshi/rplugin/python3/semshi/plugin.py", line 151, in enable
    self.highlight()
  File ".../numirias/semshi/rplugin/python3/semshi/plugin.py", line 38, in wrapper
    func(self, *args, **kwargs)
  File ".../numirias/semshi/rplugin/python3/semshi/plugin.py", line 171, in highlight
    self._cur_handler.update(force=True, sync=True)
  File ".../numirias/semshi/rplugin/python3/semshi/handler.py", line 67, in update
    self._update_step(force=force, sync=True)
  File ".../numirias/semshi/rplugin/python3/semshi/util.py", line 19, in wrapper
    res = func(*args, **kwargs)
  File ".../numirias/semshi/rplugin/python3/semshi/handler.py", line 159, in _update_step
    add, rem = self._parser.parse(code, force)
  File ".../numirias/semshi/rplugin/python3/semshi/util.py", line 19, in wrapper
    res = func(*args, **kwargs)
  File ".../numirias/semshi/rplugin/python3/semshi/parser.py", line 47, in parse
    return self._parse(*args, **kwargs)
  File ".../numirias/semshi/rplugin/python3/semshi/parser.py", line 68, in _parse
    new_nodes = self._make_nodes(code, new_lines, change_lineno)
  File ".../numirias/semshi/rplugin/python3/semshi/parser.py", line 108, in _make_nodes
    return visitor(lines, symtable_root, ast_root)
  File ".../numirias/semshi/rplugin/python3/semshi/util.py", line 19, in wrapper
    res = func(*args, **kwargs)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 46, in visitor
    visitor.visit(ast_root)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 122, in visit
    self._iter_node(node)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 397, in _iter_node
    self.visit(item)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 91, in visit
    self._visit_arg_defaults(node)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 150, in _visit_arg_defaults
    self.visit(arg_)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 127, in visit
    self._iter_node(node)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 397, in _iter_node
    self.visit(item)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 127, in visit
    self._iter_node(node)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 401, in _iter_node
    self.visit(value)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 127, in visit
    self._iter_node(node)
  File ".../numirias/semshi/rplugin/python3/semshi/visitor.py", line 388, in _iter_node
    for field in node._fields:
AttributeError: 'ellipsis' object has no attribute '_fields'

I'm running semshi at commit b50f1bc, neovim 0.5.0, and Python 3.8.0

This appears to work on 3.7.4.

goto error not working

Hey, love your Plugin!
It used to work for me but right now I got an error when using goto error (the universe has humor lol) and it does nothing. Here the error trace:
Error detected while processing function remote#define#request: line 2: error caught in request handler '/home/tbrodbeck/.config/nvim/plugged/semshi/rplugin/python3/semshi:command:Semshi [['goto', 'error']]': Traceback (most recent call last): File "/home/tbrodbeck/.config/nvim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 139, in cmd_semshi func(self, *args[1:]) File "/home/tbrodbeck/.config/nvim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 38, in wrapper func(self, *args, **kwargs) File "/home/tbrodbeck/.config/nvim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 183, in goto self._cur_handler.goto(*args, **kwargs) File "/home/tbrodbeck/.config/nvim/plugged/semshi/rplugin/python3/semshi/handler.py", line 332, in goto self._goto_error() File "/home/tbrodbeck/.config/nvim/plugged/semshi/rplugin/python3/semshi/handler.py", line 366, in _goto_error self._vim.current.window.cursor = self._error_pos(error) File "/home/tbrodbeck/.config/nvim/plugged/semshi/rplugin/python3/semshi/handler.py", line 372, in _error_pos len(self._parser.lines[error.lineno - 1]))) - 1 IndexError: list index out of range Press ENTER or type command to continue

Best, Till

More reliable plugin integration tests

The tests in test/test_plugin.py run against headless nvim instances. Since many features of Semshi operate asynchronously, it can be tricky to track when they have completed (esp. because sometimes we can't differentiate between a pending and a failed event). Consequently, some of the tests are based on timeouts which occasionally makes tests fail randomly on Travis.

This should be fixed soon because it also potentially confuses contributors submitting PRs.

(Also we should revisit the inadequate usage of fixtures that Pytest is nagging about.)

Unkown function: SemshiBufWipeout when closing windows

Hi, when I close windows, for example, after updating plugins using PlugUpdate command provided by vim-plug, I saw the following errors:

Error detected while processing BufWipeout Autocommands for "*":
E117: Unknown function: SemshiBufWipeout
E117: Unknown function: SemshiBufWipeout
E117: Unknown function: SemshiBufWipeout

This happens after I update to the latest version of semshi.

Syntax not same as yours.

Hi.
I'm was followed instructions exactly and installed all stuff. But my syntax highlighting looks ugly.

I use:
GNU bash, version 4.4.19(1)-release (x86_64-pc-linux-gnu) and Ubuntu 18.04.2 LTS
Can you give guidance what need to do that it look as yours?

Highlighting Module Level Variables

Is there a setting to allow highlighting of correctly capitalised module level variables?

An example would be MAX_NUMBER in the README example, where I'd like the option to see it highlighted in red both when it's defined and used.

Error is thrown if you run ":Semshi goto error" before opening a file

Seems it's not handled, the following error occurs:

Error detected while processing function remote#define#CommandBootstrap[5]..remote#define#request:
line    2:
error caught in request handler '/home/levi/.vim/plugged/semshi/rplugin/python3/semshi:command:Semshi [['goto', 'error']]':
Traceback (most recent call last):
  File "/home/levi/.vim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 119, in cmd_semshi
    func(self, *args[1:])
  File "/home/levi/.vim/plugged/semshi/rplugin/python3/semshi/plugin.py", line 163, in goto
    self._cur_handler.goto(*args, **kwargs)
AttributeError: 'NoneType' object has no attribute 'goto'