palantir/python-language-server

An implementation of the Language Server Protocol for Python

Python Language Server

A Python 2.7 and 3.5+ implementation of the Language Server Protocol.

Installation

The base language server requires Jedi to provide Completions, Definitions, Hover, References, Signature Help, and Symbols:

pip install python-language-server

If the respective dependencies are found, the following optional providers will be enabled:

• Rope for Completions and renaming
• Pyflakes linter to detect various errors
• McCabe linter for complexity checking
• pycodestyle linter for style checking
• pydocstyle linter for docstring style checking (disabled by default)
• autopep8 for code formatting
• YAPF for code formatting (preferred over autopep8)

Optional providers can be installed using the extras syntax. To install YAPF formatting for example:

pip install 'python-language-server[yapf]'

All optional providers can be installed using:

pip install 'python-language-server[all]'

If you get an error similar to 'install_requires' must be a string or list of strings then please upgrade setuptools before trying again.

pip install -U setuptools

3rd Party Plugins

Installing these plugins will add extra functionality to the language server:

• pyls-mypy Mypy type checking for Python 3
• pyls-isort Isort import sort code formatting
• pyls-black for code formatting using Black

Please see the above repositories for examples on how to write plugins for the Python Language Server. Please file an issue if you require assistance writing a plugin.

Configuration

Configuration is loaded from zero or more configuration sources. Currently implemented are:

• pycodestyle: discovered in ~/.config/pycodestyle, setup.cfg, tox.ini and pycodestyle.cfg.
• flake8: discovered in ~/.config/flake8, setup.cfg, tox.ini and flake8.cfg

The default configuration source is pycodestyle. Change the pyls.configurationSources setting to ['flake8'] in order to respect flake8 configuration instead.

Overall configuration is computed first from user configuration (in home directory), overridden by configuration passed in by the language client, and then overriden by configuration discovered in the workspace.

To enable pydocstyle for linting docstrings add the following setting in your LSP configuration: ` "pyls.plugins.pydocstyle.enabled": true`

See vscode-client/package.json for the full set of supported configuration options.

Language Server Features

Auto Completion:

Code Linting with pycodestyle and pyflakes:

Signature Help:

Go to definition:

Hover:

Find References:

Document Symbols:

Document Formatting:

Development

To run the test suite:

pip install .[test] && pytest

Develop against VS Code

The Python language server can be developed against a local instance of Visual Studio Code.

Install VSCode

# Setup a virtual env
virtualenv env
. env/bin/activate

# Install pyls
pip install .

# Install the vscode-client extension
cd vscode-client
yarn install

# Run VSCode which is configured to use pyls
# See the bottom of vscode-client/src/extension.ts for info
yarn run vscode -- $PWD/../

Then to debug, click View -> Output and in the dropdown will be pyls. To refresh VSCode, press Cmd + r

License

This project is made available under the MIT License.

python-language-server's Issues

TCP mode does not work

Simplest client possible:

import requests
import json


def main():
    url = "http://localhost:4000/"
    headers = {'content-type': 'application/json'}

    payload = {
        "method": "initialize",
        "jsonrpc": "2.0",
        "id": 0,
    }
    response = requests.post(
        url, data=json.dumps(payload), headers=headers).json()

if __name__ == "__main__":
    main()

Run pyls with:

pyls --tcp --host 0.0.0.0 --port 4000

Result:

----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 52078)
Traceback (most recent call last):
  File "/usr/lib64/python3.4/socketserver.py", line 617, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib64/python3.4/socketserver.py", line 344, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/home/meh/devel/sandbox/python-language-server/pyls/python_ls.py", line 33, in __init__
    super(PythonLanguageServer, self).__init__(*args, **kwargs)
TypeError: __init__() takes 3 positional arguments but 4 were given
----------------------------------------

Goto Definition throws exception for builtin types (str, list, etc.)

Add code actions to add/remove/sort imports

A sort imports action could always be available.
An add/remove import action could be registered for locations where pyflakes complained about missing/unused imports.

See: https://github.com/alecthomas/importmagic for a library supporting this functionality.

Handle urlencoded parts of document URI

On OS-X, when sending a hover request to some locations, an exception within Jedi is caused:

textDocument/hover {'position': {'character': 31, 'line': 300}, 'textDocument': {'uri': 'file:///Users/tomv/Library/Application%20Support/Sublime%20Text%203/Packages/LSP/main.py'}} 

Useful portion of the stack here, note the urlencoded spaces:

File "/Users/tomv/.virtualenvs/python3/lib/python3.5/site-packages/jedi/evaluate/imports.py", line 530, in get_modules_containing_name
      for file_name in os.listdir(d):
   FileNotFoundError: [Errno 2] No such file or directory: '/Users/tomv/Library/Application%20Support/Sublime%20Text%203/Packages/LSP'

Adding unquote to every occurrence of urlparse(...).path should fix this issue:

>>> from urllib.parse import urlparse, unquote
>>> urlparse("/Users/tomv/Library/Application%20Support/Sublime%20Text%203/Packages/LSP").path
'/Users/tomv/Library/Application%20Support/Sublime%20Text%203/Packages/LSP'
>>> unquote(urlparse("/Users/tomv/Library/Application%20Support/Sublime%20Text%203/Packages/LSP").path)
'/Users/tomv/Library/Application Support/Sublime Text 3/Packages/LSP'

I'll supply a PR once I can verify a fix.

[Doc] Add installation manual

Currently I use PLS with neovim. But I would like to try it out with VSCode like that in README. (Fairly, I cannot tell the behavior I see is due to client or server, so I need another client to compare.)

However I cannot find extension in VSCode market. Would you like provide a installation manual?

Or do I need create a wrapper extension for PLS? Thanks!

Signature Help support has gone missing

Likely missed during the pluggy refactoring?

Adding the signature help back in to CORE_PLUGINS restores this functionality.

PR coming up shortly.

Shell out to linters instead of running in the same interpreter

This lets us avoid having to find and parse configs, as well as remain a little more IO bound which will help with threading/multiprocessing/whatever we do in the future

[Suggestion] Use File based logger

Hi, thanks for bringing us Python Language Server!

I'm using it with LanguageClient_neovim. And I'm debugging some issue with PLS.
Usually, the server is opened by language client. Users cannot control where language server can log.

PLS now logs to stderr by default. It would be helpful if PLS can log to a file. So editor users can inspect log independent of language client's behavior. For example, logging to /tmp/LanguageServer.log by default and file path can be configured by command line argument.

Thank you!

Sometimes a TypeError is thrown when using get_uri_like

For example textDocument/definition fails with TypeError.

README: Consider migrating .rst -> .md

• Markdown has less overhead for people who want to contribute directly from GitHub (no build step / rich diffs)
• Most Palantir open source projects use markdown for their READMEs (exception: typedjsonrpc)

I'm happy to do the conversion if you think this makes sense @gatesn

[Feature Request] More performant server

Hi. Python language server is well written, thank you for your contribution.

However, after using I find some improvements can be done.

1. logger
   Tracked by #15. Logging to stderr will block stdout writing.

2. threaded reading
   Now PLS in stdio mode is single threaded.

   python-language-server/pyls/language_server.py

   Line 29 in 6c781d4

   server.handle()

   
   So if client sends too many requests than stdin can buffer, PLS will also block because it cannot write to stdout. (It seems stdin/stdout/stderr share one common buffer)

3. cached compeletion
   PLS generally is slower than deoplete-jedi, another jedi based completion plugin. The main reason is deoplete-jedi uses a cache. https://github.com/zchee/deoplete-jedi/blob/master/rplugin/python3/deoplete/sources/deoplete_jedi/cache.py If PLS can also support cache it should be faster.

Thanks!

Ability to control dependencies

My plan is to utilize this language server in my Python extension for VS Code.

However, there are a few gaps (obviously) in your implementation and my extension. For instance a combination of multiple linters, and also provide the ability for users to customize each linter with command line args (configurable via the settings.json or via linter specific config files - this isn't an issue).

Similarly I support two formatters in my extenion.

So, the question comes down to what's the best way forward for me, in utilizing this language server.

Implement continuous integration with CircleCi

How to launch on console for hosting?

Refactor to use json-rpc library

Refactor the language server to make use of a proper JSONRPC library, e.g. https://github.com/pavlov99/json-rpc or https://github.com/mbr/tinyrpc

textDocument/signature help calls parameters params

According to protocol.md it is named parameters , but pyls returns params. Example with subprocess.call:

{
  "jsonrpc": "2.0",
  "result": {
    "signatures": [
      {
        "activeParameter": 0,
        "documentation": "Run command with arguments.  Wait for command to complete, then\nreturn the returncode attribute.\n\nThe arguments are the same as for the Popen constructor.  Example:\n\nretcode = call([\"ls\", \"-l\"])",
        "params": [
          {
            "documentation": "",
            "label": "popenargs"
          },
          {
            "documentation": "",
            "label": "kwargs"
          }
        ],
        "label": "call(*popenargs, **kwargs)"
      }
    ],
    "activeSignature": 0
  },
  "id": 9
}

HTH and thanks for your work on pyls!

Update to LSP 3.0

• Use rootUri in favour of rootPath

Support partial result streaming

microsoft/vscode#20010

Process messages asynchronously with multiprocessing pool

Given a single stdio language server is responsible for a single workspace we may end up running into performance issues, particularly around cheap requests being blocked being more expensive requests.

Unfortunately, Jedi is not thread safe because it relies heavily on global state for caching. This is unfortunate as most of our non-jedi providers are io-bound (spawn subprocesses) and would work quite well with python threading, even with the GIL.

Instead it might be worth having a multiprocessing pool of Jedi workers each with their own cache and a results queue that is consumed and written back to the client.

To make things easy, if we already have a multiprocessing results queue we may as well use multiprocessing for everything.

Support python packages outside the repository root

Content-Length should be byte length rather string length

Spec:
https://github.com/Microsoft/language-server-protocol/blob/master/protocol.md#header-part

Content-Length number The length of the content part in bytes. This header is required.

But implementation is string length.

Read part:

https://github.com/palantir/python-language-server/blob/master/pyls/jsonrpc.py#L136

Write part:
https://github.com/palantir/python-language-server/blob/master/pyls/jsonrpc.py#L147-L148

Write part is more convoluted. json.dumps by default return all ascii character so it is implicitly equal to byte length,

Empty root URI results in failed config in_parents call

2017-08-02 01:55:18,887 UTC - ERROR - pyls.language_server - CAUGHT Traceback (most recent call last): File "/home/phantomape/Tools/anaconda2/lib/python2.7/site-packages/pyls/language_server.py", line 67, in wrapped return func(*args, **kwargs) File "/home/phantomape/Tools/anaconda2/lib/python2.7/site-packages/pyls/python_ls.py", line 108, in m_text_document__did_open self.workspace.put_document(textDocument['uri'], textDocument['text'], version=textDocument.get('version')) File "/home/phantomape/Tools/anaconda2/lib/python2.7/site-packages/pyls/workspace.py", line 41, in put_document doc_uri, content, sys_path=self.syspath_for_path(path), version=version File "/home/phantomape/Tools/anaconda2/lib/python2.7/site-packages/pyls/workspace.py", line 69, in syspath_for_path files = config.find_parents(self.root, path, ['setup.py']) or [] File "/home/phantomape/Tools/anaconda2/lib/python2.7/site-packages/pyls/config.py", line 74, in find_parents raise ValueError("Path %s not in %s" % (path, root)) ValueError: Path /1 not in

Cache jedi scripts between hooks

Start the language-server from the command line

Hi, I would like to integrate your server in an editor I am working on (https://github.com/ftomassetti/kanvas).

My first naive attempt was to run:

python pyls/language_server.py

in a virtualenv.

That does not work:

Traceback (most recent call last):
  File "pyls/language_server.py", line 4, in <module>
    from .server import JSONRPCServer
ValueError: Attempted relative import in non-package

I am probably doing something blatantly wrong. What is the best way to just start the server?

Support incremental TextDocumentSyncKind

Right now we get full text files each time they change. This is obviously bad.

I wonder if we could then propagate these changes to only re-lint the changed parts of the file?

Python 3 support

Hello, is there a road-map to implement Python 3 support?

textDocument/definition returns wrong path on windows

Notice the wrong path where d drive occurs twice. {"uri":"file:///D:\\D:\\tmp\\python-helloworld\\helloworld.py".... textDocument/documentSymbol seems to work.

The log is from vim-lsp.

8/5/2017 3:59:30 PM:["lsp#register_server","server registered","pyls"]
8/5/2017 3:59:30 PM:["lsp#register_server","server registered","rls"]
8/5/2017 3:59:30 PM:["s:on_text_document_did_open()",1]
8/5/2017 3:59:30 PM:[{"response":{"data":{"__data__":"vim-lsp","lps_id":1,"server_name":"pyls"},"message":"started lsp server successfully"}}]
8/5/2017 3:59:30 PM:["--->",1,"pyls",{"method":"initialize","params":{"rootUri":"file:///D:/tmp/python-helloworld","capabilities":{},"rootPath":"file:///D:/tmp/python-helloworld"}}]
8/5/2017 3:59:34 PM:["<---",1,"pyls",{"response":{"id":1,"jsonrpc":"2.0","result":{"capabilities":{"codeLensProvider":{"resolveProvider":false},"executeCommandProvider":{"commands":[]},"hoverProvider":true,"documentFormattingProvider":true,"signatureHelpProvider":{"triggerCharacters":["(",","]},"documentSymbolProvider":true,"definitionProvider":true,"completionProvider":{"resolveProvider":false,"triggerCharacters":["."]},"codeActionProvider":true,"referencesProvider":true,"documentRangeFormattingProvider":true,"textDocumentSync":2}}},"request":{"method":"initialize","jsonrpc":"2.0","id":1,"params":{"rootUri":"file:///D:/tmp/python-helloworld","capabilities":{},"rootPath":"file:///D:/tmp/python-helloworld"}}}]
8/5/2017 3:59:34 PM:["--->",1,"pyls",{"method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///D:/tmp/python-helloworld/helloworld.py","version":1,"languageId":"python","text":"#!/usr/bin/env python\nclass MyClass:\n    variable = \"blah\"\n\n    def function(self):\n        print(\"This is a message inside the class.\")\n\nmyobjectx = MyClass()\n\nmyobjectx.variable"}}}]
8/5/2017 3:59:34 PM:[{"response":{"data":{"path":"file:///D:/tmp/python-helloworld/helloworld.py","__data__":"vim-lsp","filetype":"python","server_name":"pyls"},"message":"textDocument/open sent"}}]
8/5/2017 3:59:34 PM:[{"response":{"data":{"path":"file:///D:/tmp/python-helloworld/helloworld.py","__data__":"vim-lsp","server_name":"pyls"},"message":"not dirty"}}]
8/5/2017 3:59:34 PM:["<---(stderr)",1,"pyls",["2017-08-05 15:59:34,223 UTC - ERROR - pyls.language_server - CAUGHT\r","Traceback (most recent call last):\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\language_server.py\", line 65, in wrapped\r","    return func(*args, **kwargs)\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\python_ls.py\", line 111, in m_text_document__did_open\r","    self.lint(textDocument['uri'])\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\python_ls.py\", line 93, in lint\r","    self._hooks.pyls_lint, doc_uri\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\python_ls.py\", line 58, in _hook\r","    return hook(config=self.config, workspace=self.workspace, document=doc, **kwargs)\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 745, in __call__\r","    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 339, in _hookexec\r","    return self._inner_hookexec(hook, methods, kwargs)\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 302, in __call__\r","    return outcome.get_result()\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 280, in get_result\r","    _reraise(*ex)  # noqa\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 265, in __init__\r","    self.result = func()\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 300, in <lambda>\r","    outcome = _CallOutcome(lambda: self.oldcall(hook, hook_impls, kwargs))\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 334, in <lambda>\r","    _MultiCall(methods, kwargs, hook.spec_opts).execute()\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 614, in execute\r","    res = hook_impl.function(*args)\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\plugins\\pycodestyle_lint.py\", line 15, in pyls_lint\r","    config_files = config.find_parents(document.path, CONFIG_FILES)\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\config.py\", line 37, in find_parents\r","    return find_parents(self.root_path, path, names)\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\config.py\", line 70, in find_parents\r","    raise ValueError(\"Path %s not in %s\" % (path, root))\r","ValueError: Path /D:/tmp/python-helloworld/helloworld.py not in file:///D:/tmp/python-helloworld\r","2017-08-05 15:59:34,226 UTC - ERROR - jsonrpc.manager - API Exception: {'message': 'Path /D:/tmp/python-helloworld/helloworld.py not in file:///D:/tmp/python-helloworld', 'args': (u'Path /D:/tmp/python-helloworld/helloworld.py not in file:///D:/tmp/python-helloworld',), 'type': 'ValueError'}\r","Traceback (most recent call last):\r","  File \"c:\\python27\\lib\\site-packages\\jsonrpc\\manager.py\", line 108, in _get_responses\r","    result = method(*request.args, **request.kwargs)\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\language_server.py\", line 65, in wrapped\r","    return func(*args, **kwargs)\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\python_ls.py\", line 111, in m_text_document__did_open\r","    self.lint(textDocument['uri'])\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\python_ls.py\", line 93, in lint\r","    self._hooks.pyls_lint, doc_uri\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\python_ls.py\", line 58, in _hook\r","    return hook(config=self.config, workspace=self.workspace, document=doc, **kwargs)\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 745, in __call__\r","    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 339, in _hookexec\r","    return self._inner_hookexec(hook, methods, kwargs)\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 302, in __call__\r","    return outcome.get_result()\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 280, in get_result\r","    _reraise(*ex)  # noqa\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 265, in __init__\r","    self.result = func()\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 300, in <lambda>\r","    outcome = _CallOutcome(lambda: self.oldcall(hook, hook_impls, kwargs))\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 334, in <lambda>\r","    _MultiCall(methods, kwargs, hook.spec_opts).execute()\r","  File \"c:\\python27\\lib\\site-packages\\pluggy.py\", line 614, in execute\r","    res = hook_impl.function(*args)\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\plugins\\pycodestyle_lint.py\", line 15, in pyls_lint\r","    config_files = config.find_parents(document.path, CONFIG_FILES)\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\config.py\", line 37, in find_parents\r","    return find_parents(self.root_path, path, names)\r","  File \"c:\\python27\\lib\\site-packages\\pyls\\config.py\", line 70, in find_parents\r","    raise ValueError(\"Path %s not in %s\" % (path, root))\r","ValueError: Path /D:/tmp/python-helloworld/helloworld.py not in file:///D:/tmp/python-helloworld\r",""]]
8/5/2017 3:59:37 PM:[{"response":{"data":{"__data__":"vim-lsp","server_name":"pyls"},"message":"server already started"}}]
8/5/2017 3:59:37 PM:[{"response":{"data":{"__data__":"vim-lsp","init_result":{"id":1,"jsonrpc":"2.0","result":{"capabilities":{"codeLensProvider":{"resolveProvider":false},"executeCommandProvider":{"commands":[]},"hoverProvider":true,"documentFormattingProvider":true,"signatureHelpProvider":{"triggerCharacters":["(",","]},"documentSymbolProvider":true,"definitionProvider":true,"completionProvider":{"resolveProvider":false,"triggerCharacters":["."]},"codeActionProvider":true,"referencesProvider":true,"documentRangeFormattingProvider":true,"textDocumentSync":2}}},"server_name":"pyls"},"message":"lsp server already initialized"}}]
8/5/2017 3:59:37 PM:[{"response":{"data":{"path":"file:///D:/tmp/python-helloworld/helloworld.py","__data__":"vim-lsp","server_name":"pyls"},"message":"already opened"}}]
8/5/2017 3:59:37 PM:[{"response":{"data":{"path":"file:///D:/tmp/python-helloworld/helloworld.py","__data__":"vim-lsp","server_name":"pyls"},"message":"not dirty"}}]
8/5/2017 3:59:37 PM:["--->",1,"pyls",{"method":"textDocument/documentSymbol","on_notification":"---funcref---","params":{"textDocument":{"uri":"file:///D:/tmp/python-helloworld/helloworld.py"}}}]
8/5/2017 3:59:37 PM:["<---",1,"pyls",{"response":{"id":2,"jsonrpc":"2.0","result":[{"location":{"uri":"file:///D:/tmp/python-helloworld/helloworld.py","range":{"end":{"character":13,"line":1},"start":{"character":6,"line":1}}},"name":"MyClass","kind":5},{"location":{"uri":"file:///D:/tmp/python-helloworld/helloworld.py","range":{"end":{"character":12,"line":2},"start":{"character":4,"line":2}}},"name":"variable","kind":13},{"location":{"uri":"file:///D:/tmp/python-helloworld/helloworld.py","range":{"end":{"character":9,"line":7},"start":{"character":0,"line":7}}},"name":"myobjectx","kind":13}]},"request":{"method":"textDocument/documentSymbol","jsonrpc":"2.0","id":2,"params":{"textDocument":{"uri":"file:///D:/tmp/python-helloworld/helloworld.py"}}}}]
8/5/2017 3:59:37 PM:["s:on_text_document_did_open()",2]
8/5/2017 3:59:43 PM:["s:on_text_document_did_close()",2]
8/5/2017 3:59:46 PM:[{"response":{"data":{"__data__":"vim-lsp","server_name":"pyls"},"message":"server already started"}}]
8/5/2017 3:59:46 PM:[{"response":{"data":{"__data__":"vim-lsp","init_result":{"id":1,"jsonrpc":"2.0","result":{"capabilities":{"codeLensProvider":{"resolveProvider":false},"executeCommandProvider":{"commands":[]},"hoverProvider":true,"documentFormattingProvider":true,"signatureHelpProvider":{"triggerCharacters":["(",","]},"documentSymbolProvider":true,"definitionProvider":true,"completionProvider":{"resolveProvider":false,"triggerCharacters":["."]},"codeActionProvider":true,"referencesProvider":true,"documentRangeFormattingProvider":true,"textDocumentSync":2}}},"server_name":"pyls"},"message":"lsp server already initialized"}}]
8/5/2017 3:59:46 PM:[{"response":{"data":{"path":"file:///D:/tmp/python-helloworld/helloworld.py","__data__":"vim-lsp","server_name":"pyls"},"message":"already opened"}}]
8/5/2017 3:59:46 PM:[{"response":{"data":{"path":"file:///D:/tmp/python-helloworld/helloworld.py","__data__":"vim-lsp","server_name":"pyls"},"message":"not dirty"}}]
8/5/2017 3:59:46 PM:["--->",1,"pyls",{"method":"textDocument/definition","on_notification":"---funcref---","params":{"textDocument":{"uri":"file:///D:/tmp/python-helloworld/helloworld.py"},"position":{"character":15,"line":7}}}]
8/5/2017 3:59:46 PM:["<---",1,"pyls",{"response":{"id":3,"jsonrpc":"2.0","result":[{"uri":"file:///D:\\D:\\tmp\\python-helloworld\\helloworld.py","range":{"end":{"character":13,"line":1},"start":{"character":6,"line":1}}}]},"request":{"method":"textDocument/definition","jsonrpc":"2.0","id":3,"params":{"textDocument":{"uri":"file:///D:/tmp/python-helloworld/helloworld.py"},"position":{"character":15,"line":7}}}}]
8/5/2017 3:59:46 PM:["s:on_text_document_did_open()",4]

Fix PyPi publishing

Support configuration for enabling/disabling/configuring plugins

As well as enabling/disabling everything, we want to:

• Pass pycodestyle / pyflakes configs
• Choose the style for Yapf (pep8, google, config file)

Python 3 issues - byte vs str

Thanks a lot for the server, it's amazing job.

I tried to use it with python3.5 and noticed some issues with mixing byte and str usage. While it works in python2 (I think mainly because most code and paths are ascii encoded) it raises an exception in 3rd python.

So far I noticed 2 bugs affected by it.

Starting a server in --tcp mode.

pyls.server.JSONRPCServer#_content_length fails at line.startswith call, because line is raw bytes and header is an unicode string. I had to modify it to be line.startswith(b"Content-Length: "). And the same applies to next line _, value = line.split(b"Content-Length: ").

Also sending back response (pyls.server.JSONRPCServer#_write_message) fails on write, but it works if encoded - self.wfile.write(response.encode()).

But this changes will break the stdin/stdout communication. So perhaps pyls.language_server.LanguageServer implementation needs some extra init parameter or attribute to flag if communications are in raw bytes or in unicode string.

Allow log level configuration with -v -vv -vvv etc

This is much easier than falling back to a config file as we need to do now

Support configuration from config file as well as via initialization opts

Parse workspace config and store it in the config object

Workspace config should be acquired and parsed from setup.cfg (and some other places) and stored in a global configuration dictionary. This is then available to plugins so that can piggy-back on the language server's config parsing.

'Invalid params' for textDocument/definition request

I'm trying to use 'textDocument/definition' and most of the time it works ok but for attached snippet server returns Invalid parms response.

Code:

import math

math.sqrt( x )

Request:

{
  "jsonrpc": "2.0",
  "id": "4",
  "method": "textDocument/definition",
  "params": {
    "textDocument": {
      "uri": "file:///home/wywrzal/development/ZEND/runtime-IDE/Test/test.py"
    },
    "uri": "file:///home/wywrzal/development/ZEND/runtime-IDE/Test/test.py",
    "position": {
      "line": 2,
      "character": 7
    }
  }
}

Response:

{
  "jsonrpc": "2.0",
  "id": "4",
  "error": {
    "code": -32602,
    "message": "Invalid params",
    "data": {
      "message": "unsupported operand type(s) for -: \u0027NoneType\u0027 and \u0027int\u0027",
      "args": [
        "unsupported operand type(s) for -: \u0027NoneType\u0027 and \u0027int\u0027"
      ],
      "type": "TypeError"
    }
  }
}

Language server breaks if rootPath/rootUri not set

Reported by @IBestuzhev

Open file in VS Code, not directory.

In this case root is not send in initialize call from client, and it breaks the Workspace. self.root becomes b''. And initialize calls succeeds. But all later calls are failing on pyls.workspace.Workspace#_check_in_workspace, because it compares str(doc_path) with byte(self.root).

The issue here is in urlparse. It works both with byte and str, and output depends on input. But for None it defaults to bytes

I think there should be some extra support for empty document root. Because in protocol spec it can be null.

>>> urlparse('/my/file.txt').path
'/my/file.txt'
>>> urlparse(b'/my/file.txt').path
b'/my/file.txt'
>>> urlparse(None).path
b''

Compute capabilities based on available hookspecs

If there are no hooks implemented for codeAction for example, then we shouldn't announce the capability. It just makes the client/server do more work.

Newline issues on Windows

Hi there,

Thanks for the excellent work you've done on the language server! Overall, it's working great on OSX and Linux - I'm working on incorporating it into Oni: https://github.com/extr0py/oni

There is one blocking issue on Windows, however:

Issue: The LSP protocol expects there to be \r\n\r\n following the Content-Header. This works as expected on OSX and Linux. However, on Windows, we actually get \r\r\n\r\r\n, which some LSP clients will not handle this. vscode-jsonrpc is strict on matching this and therefore never realizes the stream is complete, and never completes initialization. I've tested this using the stdio strategy - I haven't validated with the tcp strategy.

Defect: It turns out Python has some 'magic' behavior with newlines - see https://stackoverflow.com/questions/2536545/how-to-write-unix-end-of-line-characters-in-windows-using-python. It looks like Python is converting \n on Windows to os.linesep - which is \r\n.

This is the impacted code (server.py):

    def _write_message(self, msg):
        body = json.dumps(msg, separators=(",", ":"))
        content_length = len(body)
        response = (
            "Content-Length: {}\r\n"
            "Content-Type: application/vscode-jsonrpc; charset=utf8\r\n\r\n"
            "{}".format(content_length, body)
        )
        self.wfile.write(response)
        self.wfile.flush()

In this case, the \n in the \r\n is getting expanded - so we actually end up with \r\n.

Proposed Fix: The os.linesep should be checked. If it is \n, we should use \r\n as the line ending to conform to the protocol. If it is \r\n, that means \n will be expanded to \r\n, so we should use \n as the terminator above.

I'm not an expert in Python, so there may be a cleaner way to fix this. It looks like when reading from a file, there are options in terms of handling newlines, so maybe there is an option to set when writing to the output. I'm not sure as well if this would cause problems with the tcp server.

Let me know if this isn't clear - happy to give more info.

Test against Python 3.4 not 3.6

Support workspace symbols

README: badges

Potentially:

• Python versions
• MIT License
• CircleCi build status (once implemented)

Error with `textDocument/documentSymbol` protocol

Hi there,

I'm hitting an issue with the textDocument/documentSymbol request - documented here

I'm getting this error message from the language server on both Windows and OSX:

bundle.js:54145 [LANGUAGE CLIENT - ERROR]: 2017-07-06 16:39:00,608 UTC - ERROR - pyls.language_server - CAUGHT
Traceback (most recent call last):
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pluggy.py", line 606, in execute
    args = [all_kwargs[argname] for argname in hook_impl.argnames]
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pluggy.py", line 606, in <listcomp>
    args = [all_kwargs[argname] for argname in hook_impl.argnames]
KeyError: 'position'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pyls\language_server.py", line 64, in wrapped
    return func(*args, **kwargs)
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pyls\python_ls.py", line 138, in m_text_document__document_symbol
    return self.document_symbols(textDocument['uri'])
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pyls\python_ls.py", line 76, in document_symbols
    return flatten(self._hook(self._pm.hook.pyls_definitions, doc_uri))
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pyls\python_ls.py", line 57, in _hook
    return hook(workspace=self.workspace, document=doc, **kwargs)
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pluggy.py", line 745, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pluggy.py", line 339, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pluggy.py", line 302, in __call__
    return outcome.get_result()
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pluggy.py", line 279, in get_result
    raise ex[1].with_traceback(ex[2])
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pluggy.py", line 265, in __init__
    self.result = func()
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pluggy.py", line 300, in <lambda>
    outcome = _CallOutcome(lambda: self.oldcall(hook, hook_impls, kwargs))
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pluggy.py", line 334, in <lambda>
    _MultiCall(methods, kwargs, hook.spec_opts).execute()
  File "c:\users\bryan\appdata\local\programs\python\python36\lib\site-packages\pluggy.py", line 611, in execute
    "hook call must provide argument %r" % (argname,))
pluggy.HookCallError: hook call must provide argument 'position'

And these are the two requests I made to the server (after initialization):

{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///C:\\test\\python\\test.py","languageId":"python","version":4,"text":"import sys\r\n\r\ndata = sys.stdin.readlines()\r\n\r\ndata = sys.stdin.readlines()\r\n\r\noutput = \"Counted\", len(data), \"lines\"\r\nprint(output)\r\n"}}}

{"jsonrpc":"2.0","id":1,"method":"textDocument/documentSymbol","params":{"textDocument":{"uri":"file:///C:\\test\\python\\test.py"}}}

It looks like somehow it's expecting a position argument for textDocument/documentSymbol, even though it isn't required as part of the protocol

Let me know if you need any more info!

Should we enforce a style guide?

Possibly PEP 8

Add support for pylint in favour of pyflakes as default linter

Upgrade to vanilla jedi 0.10.0 now that it is released

Support for mypy checking as well (i.e typing)

Other lint-runners (like ALE and neomake) supports having mypy as a linter as well, which uses the typing library in stdlib and type annotations to verify optional static typing. It would be nice to have that support in python-language-server as well. :)

Log rotation and management

Right now the Python language server only writes one log file. If the service crashes, and restarts, you lose the old log events.

Launch LS in stdio mode

Hello, i want to start your Language Service in stdio mode as separate process and communicate with it through web IDE (not VScode). I don’t see any documentation about how it is done, could you please point me where I can find it?
Thanks

Publish to PyPi with tagged Circle builds

Installing via pip

Is it possible to install pyls via pip?

A call to pip install --user pyls

fails as

Collecting pyls
  Downloading pyls-0.1.3.zip
    Complete output from command python setup.py egg_info:
    Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.14.tar.gz
    Extracting in /tmp/tmpTC5h_r
    Now working in /tmp/tmpTC5h_r/distribute-0.6.14
    Building a Distribute egg in /tmp/pip-build-NGPQpl/pyls
    Traceback (most recent call last):
      File "setup.py", line 37, in <module>
        exec(open(init_path).read(), d)
      File "<string>", line 8, in <module>
      File "/tmp/tmpTC5h_r/distribute-0.6.14/setuptools/__init__.py", line 2, in <module>
        from setuptools.extension import Extension, Library
      File "/tmp/tmpTC5h_r/distribute-0.6.14/setuptools/extension.py", line 2, in <module>
        from setuptools.dist import _get_unpatched
      File "/tmp/tmpTC5h_r/distribute-0.6.14/setuptools/dist.py", line 6, in <module>
        from setuptools.command.install import install
      File "/tmp/tmpTC5h_r/distribute-0.6.14/setuptools/command/__init__.py", line 8, in <module>
        from setuptools.command import install_scripts
      File "/tmp/tmpTC5h_r/distribute-0.6.14/setuptools/command/install_scripts.py", line 3, in <module>
        from pkg_resources import Distribution, PathMetadata, ensure_directory
      File "/tmp/tmpTC5h_r/distribute-0.6.14/pkg_resources.py", line 2691, in <module>
        add_activation_listener(lambda dist: dist.activate())
      File "/tmp/tmpTC5h_r/distribute-0.6.14/pkg_resources.py", line 668, in subscribe
        callback(dist)
      File "/tmp/tmpTC5h_r/distribute-0.6.14/pkg_resources.py", line 2691, in <lambda>
        add_activation_listener(lambda dist: dist.activate())
      File "/tmp/tmpTC5h_r/distribute-0.6.14/pkg_resources.py", line 2192, in activate
        self.insert_on(path)
      File "/tmp/tmpTC5h_r/distribute-0.6.14/pkg_resources.py", line 2293, in insert_on
        "with distribute. Found one at %s" % str(self.location))
    ValueError: A 0.7-series setuptools cannot be installed with distribute. Found one at /usr/lib/python2.7/dist-packages
    /tmp/pip-build-NGPQpl/pyls/distribute-0.6.14-py2.7.egg
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-NGPQpl/pyls/setup.py", line 31, in <module>
        main()
      File "/tmp/pip-build-NGPQpl/pyls/setup.py", line 11, in main
        distribute_setup.use_setuptools()
      File "setup_utils/distribute_setup.py", line 145, in use_setuptools
        return _do_download(version, download_base, to_dir, download_delay)
      File "setup_utils/distribute_setup.py", line 125, in _do_download
        _build_egg(egg, tarball, to_dir)
      File "setup_utils/distribute_setup.py", line 116, in _build_egg
        raise IOError('Could not build the egg.')
    IOError: Could not build the egg.
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-NGPQpl/pyls/

and to pip3 install --user pyls

fails as

Collecting pyls
  Using cached pyls-0.1.3.zip
    Complete output from command python setup.py egg_info:
    Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.14.tar.gz
    Extracting in /tmp/tmp5zufnl0r
    Traceback (most recent call last):
      File "/tmp/pip-build-ldrp9vpf/pyls/setup_utils/distribute_setup.py", line 143, in use_setuptools
        raise ImportError
    ImportError
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-ldrp9vpf/pyls/setup.py", line 31, in <module>
        main()
      File "/tmp/pip-build-ldrp9vpf/pyls/setup.py", line 11, in main
        distribute_setup.use_setuptools()
      File "/tmp/pip-build-ldrp9vpf/pyls/setup_utils/distribute_setup.py", line 145, in use_setuptools
        return _do_download(version, download_base, to_dir, download_delay)
      File "/tmp/pip-build-ldrp9vpf/pyls/setup_utils/distribute_setup.py", line 125, in _do_download
        _build_egg(egg, tarball, to_dir)
      File "/tmp/pip-build-ldrp9vpf/pyls/setup_utils/distribute_setup.py", line 99, in _build_egg
        _extractall(tar)
      File "/tmp/pip-build-ldrp9vpf/pyls/setup_utils/distribute_setup.py", line 467, in _extractall
        self.chown(tarinfo, dirpath)
    TypeError: chown() missing 1 required positional argument: 'numeric_owner'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-ldrp9vpf/pyls/

Allow selection of different providers via initializationOptions