IPython kernel autocompletions (some) stop working after installing LSP

[jpcaram]

IPython kernel autocompletions (some) stop working after installing LSP

[krassowski]

Thanks for the detailed report. Supporting this was a high priority and it used to work before the rewrite for JupyterLab 4.0. This is similar to #1009 and earlier issues on this topic.

This is a hard topic

The problem is that the kernels do not return the the information necessary to resolve whether the completion should be inserted after or in place of the existing text. This means that we need to use some heuristics for reconciliation of completions from various sources which breaks the edge cases like the latex symbols (which are substitutions). If someone has time to take a look I will prioritise review, but otherwise this might need a foundational change in jupyter protocol.

[jpcaram]

Thanks for your response.

To clarify, I used the latex_symbols examples just for conciseness. This appears to affect all custom completions. I will have to double check if those that are just completion and not replacement still work.

I would like to help, and I could use some information:

• Is there something implemented that is specifically disabling or ignoring certain kinds of completions?
• I tried to debug the issue at the kernel level by print statements in my custom completers. I don't see anything on the terminal. It appears that the custom completer's callback is never called. Or are kernel prints to stdout suppressed?
• Why would it be a foundational change to the protocol? I seems that this would just require additional fields in a message to indicate replacement/completion, position, etc.

Basically, I would like to understand the problem better.

Also, what is the alternative these days? Should one write their own LSP, add features to an existing one, write a Jupyter completer plugin?

[krassowski]

This specific problem is introduced by preFilterMatches in setCompletionItems logic. If you disable pre-filtering:

you should be able to use latex_symbols the way. There is no change at the kernel level.

seems that this would just require additional fields in a message to indicate replacement/completion, position, etc.

Yes. More precisely, there are already fields for start and end position. The problem is that they are not always used and that there is no distinction between insert text and label which means that frontends have to guess what fragment should be inserted.

[jpcaram]

Disabling "Pre-filter matches" solves the problem in my case. This may be an edge case though (Ipython kernel, Jupyter lab, Jupyter LSP support).

I still don't fully understand why this is a big issue from the perspective of Jupyter (server, lab, lsp). Things seem to work just fine with the IPython kernel. If the issue is for other kernels, it's an issue with their implementation, not Jupyter's. In my opinion, the option on the lsp client should be disabled by default. Otherwise, it will hide the problem for ever, and kernel developers won't address their problem.

Please correct me if I'm wrong on my assumptions.