python-websockets/websockets

Library for building WebSocket servers and clients in Python

What is websockets?

websockets is a library for building WebSocket servers and clients in Python with a focus on correctness, simplicity, robustness, and performance.

Built on top of asyncio, Python's standard asynchronous I/O framework, the default implementation provides an elegant coroutine-based API.

An implementation on top of threading and a Sans-I/O implementation are also available.

Documentation is available on Read the Docs.

Here's an echo server with the asyncio API:

#!/usr/bin/env python

import asyncio
from websockets.server import serve

async def echo(websocket):
    async for message in websocket:
        await websocket.send(message)

async def main():
    async with serve(echo, "localhost", 8765):
        await asyncio.get_running_loop().create_future()  # run forever

asyncio.run(main())

Here's how a client sends and receives messages with the threading API:

#!/usr/bin/env python

from websockets.sync.client import connect

def hello():
    with connect("ws://localhost:8765") as websocket:
        websocket.send("Hello world!")
        message = websocket.recv()
        print(f"Received: {message}")

hello()

Does that look good?

Get started with the tutorial!

---

websockets for enterprise

Available as part of the Tidelift Subscription

The maintainers of websockets and thousands of other packages are working with Tidelift to deliver commercial support and maintenance for the open source dependencies you use to build your applications. Save time, reduce risk, and improve code health, while paying the maintainers of the exact dependencies you use. Learn more.

---

(If you contribute to websockets and would like to become an official support provider, let me know.)

Why should I use websockets?

The development of websockets is shaped by four principles:

1. Correctness: websockets is heavily tested for compliance with 6455. Continuous integration fails under 100% branch coverage.
2. Simplicity: all you need to understand is msg = await ws.recv() and await ws.send(msg). websockets takes care of managing connections so you can focus on your application.
3. Robustness: websockets is built for production. For example, it was the only library to handle backpressure correctly before the issue became widely known in the Python community.
4. Performance: memory usage is optimized and configurable. A C extension accelerates expensive operations. It's pre-compiled for Linux, macOS and Windows and packaged in the wheel format for each system and Python version.

Documentation is a first class concern in the project. Head over to Read the Docs and see for yourself.

Why shouldn't I use websockets?

• If you prefer callbacks over coroutines: websockets was created to provide the best coroutine-based API to manage WebSocket connections in Python. Pick another library for a callback-based API.

• If you're looking for a mixed HTTP / WebSocket library: websockets aims at being an excellent implementation of 6455: The WebSocket Protocol and 7692: Compression Extensions for WebSocket. Its support for HTTP is minimal — just enough for an HTTP health check.

  If you want to do both in the same server, look at HTTP frameworks that build on top of websockets to support WebSocket connections, like Sanic.

What else?

Bug reports, patches and suggestions are welcome!

To report a security vulnerability, please use the Tidelift security contact. Tidelift will coordinate the fix and disclosure.

For anything else, please open an issue or send a pull request.

Participants must uphold the Contributor Covenant code of conduct.

websockets is released under the BSD license.

websockets's Issues

`pip install websockets` installs asyncio 0.4 with python 3.4.0.

Expectation is that with python 3.4.0 that the built in asyncio would be used, not an external module.

pip install fails

The following command:

pip install git+https://github.com/aaugustin/websockets.git

with the following logged by pip:

------------------------------------------------------------
/home/amirouche/.virtualenvs/suprm3/bin/pip run on Thu Oct 24 01:24:34 2013
Downloading/unpacking git+https://github.com/aaugustin/websockets.git

  Cloning https://github.com/aaugustin/websockets.git to /tmp/pip-tgmzlg-build

  Found command 'git' at '/usr/bin/git'
  Running command /usr/bin/git clone -q https://github.com/aaugustin/websockets.git /tmp/pip-tgmzlg-build
  Running setup.py egg_info for package from git+https://github.com/aaugustin/websockets.git

    Traceback (most recent call last):

      File "<string>", line 16, in <module>

      File "/tmp/pip-tgmzlg-build/setup.py", line 11, in <module>

        with open(os.path.join(root, 'README')) as f:

    FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-tgmzlg-build/README'

    Complete output from command python setup.py egg_info:

    Traceback (most recent call last):

  File "<string>", line 16, in <module>

  File "/tmp/pip-tgmzlg-build/setup.py", line 11, in <module>

    with open(os.path.join(root, 'README')) as f:

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-tgmzlg-build/README'

----------------------------------------

Command python setup.py egg_info failed with error code 1 in /tmp/pip-tgmzlg-build

Exception information:
Traceback (most recent call last):
  File "/home/amirouche/.virtualenvs/suprm3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/home/amirouche/.virtualenvs/suprm3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg/pip/commands/install.py", line 266, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/home/amirouche/.virtualenvs/suprm3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg/pip/req.py", line 1057, in prepare_files
    req_to_install.run_egg_info()
  File "/home/amirouche/.virtualenvs/suprm3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg/pip/req.py", line 236, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/home/amirouche/.virtualenvs/suprm3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg/pip/util.py", line 662, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command python setup.py egg_info failed with error code 1 in /tmp/pip-tgmzlg-build

When using websocket_client.close() the worker never terminates

This is what I have in my test. (stop_client being called in tearDown)

    def start_client(self, path='', **kwds):
        client = connect('ws://localhost:8765/' + path, **kwds)
        self.client = self.loop.run_until_complete(client)

    def stop_client(self):
        self.client.close()
        print("$ Wait for client to terminate")
        self.loop.run_until_complete(self.client.worker)
        print("$ Client stopped")

Best practices for integration with other asyncio libraries?

Hello there,

I'm certain this is not the correct place for this question, but it does involve the websockets library rather directly and may be beneficial for other users by way of documentation. So apologies for bringing this up if it's just noise.

I want to know what the best way for me to "block on" two separate asyncio-enabled libraries simultaneously is. By "block on" I mean I want to react immediately to messages from two different libraries' "yield from" calls.

In my app, the two libraries are websockets and asyncio_redis, but I presume it's the same regardless of which two async libraries you'd want to do this with. Here's an example of my websocket server and my current (working) solution:

#!/usr/bin/env python3
import asyncio
import asyncio_redis
import websockets


@asyncio.coroutine
def handler(websocket, path):
    redis = yield from asyncio_redis.Connection.create(host='localhost', port=6379)
    subscriber_conn = yield from asyncio_redis.Connection.create(host='localhost', port=6379)
    subscriber = yield from subscriber_conn.start_subscribe()
    yield from subscriber.subscribe(['the-channel'])

    # continually check messages
    while True:
        client_msg = websocket.recv()
        redis_msg = subscriber.next_published()
        done_tasks, pending_tasks = yield from asyncio.wait((client_msg, redis_msg), return_when=asyncio.FIRST_COMPLETED)
        for task in done_tasks:
            if task._coro == client_msg:
                yield from handle_client_message(task.result())
            if task._coro == redis_msg:
                yield from handle_redis_message(task.result())


if __name__ == '__main__':
    start_server = websockets.serve(handler, 'localhost', 9000)
    asyncio.get_event_loop().run_until_complete(start_server)
    asyncio.get_event_loop().run_forever()

handle_client_message and handle_redis_message are just stand-ins for the real app code, but I didn't want to clutter up the example any further.

You can see from this code that the key mechanism I'm using is:

done_tasks, pending_tasks = yield from asyncio.wait([some_tasks], asyncio.FIRST_COMPLETED)

This does what I want, but I can't help but think it's The Wrong Way to do this (not least because I then have to use a private variable in task._coro to decide which task fired the result).

What's the expected way to tie websockets in with other libraries such that you don't have to have a request/reply style of app (as the echo example is)?

Move beyond a one-page, API-based documentation

The current documentation is minimal.

It could include a tutorial, background on async programming, uses cases, etc.

Allow Sub-Protocol negotiation during the Websocket handshake.

The RFC6455 Section 1.9 leaves us with a header field to negotiate a sub-protocol during the Websocket handshake. It would be nice if you could allow your library users to interfere at this time so we can see which protocols are advertised by the client, choose one of them or enforce a completely different value and send it back to the client. Of course the ws_handler needs to know which protocol has been chosen.

UnicodeDecodeError installing inside a docker container

Step 19 : RUN pip-3.3 install websockets
---> Running in 0621d2d66601
Downloading/unpacking websockets
Downloading websockets-1.0.tar.gz
Running setup.py egg_info for package websockets
Traceback (most recent call last):
File "", line 16, in
File "/tmp/pip_build_root/websockets/setup.py", line 18, in
long_description = '\n\n'.join(f.read().split('\n\n')[1:])
File "/usr/lib/python3.3/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 467: ordinal not in range(128)
Complete output from command python setup.py egg_info:
Traceback (most recent call last):

File "", line 16, in

File "/tmp/pip_build_root/websockets/setup.py", line 18, in

long_description = '\n\n'.join(f.read().split('\n\n')[1:])

File "/usr/lib/python3.3/encodings/ascii.py", line 26, in decode

return codecs.ascii_decode(input, self.errors)[0]

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 467: ordinal not in range(128

Wheel packages

Steal this: Polyconseil/aioamqp@c46f9a0

Handler for closed connections not allowed to finish

During shutdown I wait for clients to close connections. When this is done, the handler is not allowed to finish.

To trigger the error, run the following script, interrupt it (Ctrl+C) with a connection open, then close the connection.

#!/usr/bin/env python

import asyncio
import logging
import signal

import websockets

logging.getLogger().setLevel(logging.DEBUG)

connectionClosedEvents = []

@asyncio.coroutine
def echo(websocket, path):
    print("Connection opened.")
    connectionClosedEvent = asyncio.Event()
    connectionClosedEvents.append(connectionClosedEvent)
    s = yield from websocket.recv()
    if s is not None:
        yield from websocket.send(s)
    print("Connection closed.")
    connectionClosedEvent.set()

@asyncio.coroutine
def shutdown():
    print("Shutdown initialized.")
    print("Waiting for open connections to close ...")
    for e in connectionClosedEvents:
        yield from e.wait()
    print("All connections closed.")
    echoServer.close()
    print("Waiting for server to close ...")
    yield from echoServer.wait_closed()
    print("Server closed.")
    print("Stopping event loop ...")
    eventLoop.stop()
    print("Event loop stopped.")
    print("Shutdown complete.")

eventLoop = asyncio.get_event_loop()
eventLoop.add_signal_handler(signal.SIGINT, asyncio.async, shutdown())
echoServer =\
    eventLoop.run_until_complete(websockets.serve(echo, "0.0.0.0", 8765))
eventLoop.run_forever()

Typical output:

Connection opened.
^CShutdown initialized.
Waiting for open connections to close ...
Connection closed.
All connections closed.
Waiting for server to close ...
Server closed.
Stopping event loop ...
Event loop stopped.
Shutdown complete.
Task was destroyed but it is pending!
task: <Task pending coro=<handler() running at /tmp/tmp.OhmZJ4uKUJ/venv/lib/python3.4/site-packages/websockets/server.py:77> wait_for=<Future finished result=None>>

The reason for the error is that websockets.server.connection_lost removes the connection from self.websockets. Thus in websockets.server.wait_closed the connection is not waited for:

            yield from asyncio.wait(
                [ws.handler_task for ws in self.websockets], loop=self.loop)

I will submit a proposed fix for this shortly.

Enforce maximum message size limit

Cheers, thanks for writing this

One feature I am missing is the ability to enforce a maximum message size for received messages, i.e. set an arbitrary size limit for the socket and disconnect with error (1009) if a message is too large.

Add support for custom HTTP headers in handshake request

See #46 for a use case.

add on_open() and on_close() methods, similar to tornado web socket implementation?

Would it be possible to add high-level on_open() and on_close() methods, similar to the web sockets implementation in Tornado, so that initialisation and cleanup can be performed?

Consider using a proper HTTP implementation

The HTTP parsing and generation code is very simple at this time.

It could be worth relying on an external library for these parts.

socket.send() raised exception.

I have two tasks in the asyncio thread, one being the websocket connection handler and the other processing items from a pipe and sending data to websocket clients.

The code can be seen here: https://raw.githubusercontent.com/marksamman/pylinkshortener/master/app/websocket/__init__.py

Under heavy load (connecting a websocket client, running apachebench to flood the pipe, disconnect while it's being flooded), I noticed "socket.send() raised exception." being printed and that the application eventually crashes. Output:

[socket.send() raised exception repeated over 1000 times ...]
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
Future/Task exception was never retrieved
future: Task(<handler>)<exception=AttributeError("'NoneType' object has no attribute 'shutdown'",)>
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/websockets/server.py", line 60, in handler
    yield from self.close()
  File "/usr/local/lib/python3.4/site-packages/websockets/protocol.py", line 119, in close
    yield from asyncio.wait_for(self.worker, timeout=self.timeout)
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py", line 413, in wait_for
    return fut.result()
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/futures.py", line 243, in result
    raise self._exception
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py", line 279, in _step
    result = coro.throw(exc)
  File "/usr/local/lib/python3.4/site-packages/websockets/protocol.py", line 210, in run
    msg = yield from self.read_message()
  File "/usr/local/lib/python3.4/site-packages/websockets/protocol.py", line 228, in read_message
    frame = yield from self.read_data_frame()
  File "/usr/local/lib/python3.4/site-packages/websockets/protocol.py", line 266, in read_data_frame
    frame = yield from self.read_frame()
  File "/usr/local/lib/python3.4/site-packages/websockets/protocol.py", line 295, in read_frame
    frame = yield from read_frame(self.reader.readexactly, is_masked)
  File "/usr/local/lib/python3.4/site-packages/websockets/framing.py", line 64, in read_frame
    data = yield from read_bytes(reader, 2)
  File "/usr/local/lib/python3.4/site-packages/websockets/framing.py", line 96, in read_bytes
    return (yield from reader(n))
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/streams.py", line 461, in readexactly
    block = yield from self.read(n)
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/streams.py", line 432, in read
    yield from self._waiter
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/futures.py", line 348, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py", line 332, in _wakeup
    value = future.result()
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/futures.py", line 243, in result
    raise self._exception
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/selector_events.py", line 487, in write
    n = self._sock.send(data)
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py", line 281, in _step
    result = coro.send(value)
  File "/usr/local/lib/python3.4/site-packages/websockets/server.py", line 63, in handler
    self.writer.write_eof()
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/streams.py", line 257, in write_eof
    return self._transport.write_eof()
  File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/selector_events.py", line 531, in write_eof
    self._sock.shutdown(socket.SHUT_WR)
AttributeError: 'NoneType' object has no attribute 'shutdown'

I've tried adding "if client.open:" before sending and also tried handling exceptions from my application but neither helps. It seems like this issue needs to be addressed at a lower level, possibly in the websockets module to handle the exceptions properly if a client has disconnected.

Provide a better API to access HTTP headers

Currently a raw representation as a list of 2-uples is available in WebSocketClient/ServerProtocol.raw_request/response_headers.

Since these are read-only attributes, the problems that the API should solve include:

• case-insensitive access — this is easy
• access to headers with multiple values — this is a mess, maybe we can just ignore the problem and tell users who care to use the raw API
• encoding issues — maybe email.parser.BytesHeaderParser() handles everything for us, I'm not sure

Document how you serve both websockets and a aiohttp server on the same port.

I'd like to have some HTTP pages alongside my websocket handler.

How should I integrate?

Would this works?

#!/usr/bin/env python

import asyncio
from aiohttp import web
import websockets

@asyncio.coroutine
def hello(websocket, path):
    name = yield from websocket.recv()
    print("< {}".format(name))
    greeting = "Hello {}!".format(name)
    yield from websocket.send(greeting)
    print("> {}".format(greeting))

@asyncio.coroutine
def handle(request):
    name = request.match_info.get('name', "Anonymous")
    text = "Hello, " + name
    return web.Response(body=text.encode('utf-8'))

@asyncio.coroutine
def init_http_server(loop):
    app = web.Application(loop=loop)
    app.router.add_route('GET', '/{name}', handle)

    srv = yield from loop.create_server(app.make_handler(),
                                        '127.0.0.1', 8080)
    print("Server started at http://127.0.0.1:8080")
    return srv



loop = asyncio.get_event_loop()

websocket_server = websockets.serve(hello, 'localhost', 8765)
http_server = init_http_server(loop)

loop.run_until_complete(websocket_server)
loop.run_until_complete(http_server)

loop.run_forever()

Closing SSL connection in server fails assertion

When my test SSL client closes the connection to my test SSL server, the closing of the connection fails an assertion on line 338 in protocol.py. However, when I change the assertion to an if-statement, it works just fine. My server and client are given below. The server and client use the same "cert.pem" file create with the following command:
openssl req -new -x509 -days 3650 -nodes -out cert.pem -keyout cert.pem

Server

import asyncio
from ssl import PROTOCOL_TLSv1, SSLContext
import websockets

@asyncio.coroutine
def hello(websocket, path):
    name = yield from websocket.recv()
    print("< {}".format(name))
    greeting = "Hello {}!".format(name)
    print("> {}".format(greeting))
    yield from websocket.send(greeting)

ssl_context = SSLContext(PROTOCOL_TLSv1)
ssl_context.load_cert_chain('cert.pem')
kwds = {'ssl': ssl_context}
start_server = websockets.serve(hello, 'localhost', 8765, **kwds)

asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

Client

import asyncio
from ssl import PROTOCOL_TLSv1, SSLContext
import websockets

@asyncio.coroutine
def hello():
    ssl_context = SSLContext(PROTOCOL_TLSv1)
    ssl_context.load_cert_chain('cert.pem')
    kwds = {'ssl': ssl_context}
    websocket = yield from websockets.connect('wss://localhost:8765/', **kwds)
    name = input("What's your name? ")
    yield from websocket.send(name)
    print("> {}".format(name))
    greeting = yield from websocket.recv()
    print("< {}".format(greeting))

asyncio.get_event_loop().run_until_complete(hello())

Pass event loop explicitly to all asyncio objects

Python: 3.4
OS: Windows 7 (Firewall turned off)
Virtual environment was used (pyvenv).

Server code:

#!/usr/bin/env python

import asyncio
import threading
import websockets

@asyncio.coroutine
def hello(websocket, path):
    name = yield from websocket.recv()
    print("< {}".format(name))
    greeting = "Hello {}!".format(name)
    yield from websocket.send(greeting)
    print("> {}".format(greeting))

class ServerThread(threading.Thread):

  def run(self):
    self.loop = asyncio.new_event_loop()
    coro = websockets.serve(hello, 'localhost', 8765,
                            loop=self.loop)
    self.loop.run_until_complete(coro)
    self.loop.run_forever()

ServerThread().start()

Exception traceback in client.py (documentation example code):

Traceback (most recent call last):
  File "C:\tajf\lib\site-packages\websockets\client.py", line 61, in handshake
    status_code, headers = yield from read_response(self.reader)
  File "C:\tajf\lib\site-packages\websockets\http.py", line 63, in read_response

    status_line, headers = yield from read_message(stream)
  File "C:\tajf\lib\site-packages\websockets\http.py", line 80, in read_message
    start_line = yield from read_line(stream)
  File "C:\tajf\lib\site-packages\websockets\http.py", line 99, in read_line
    line = yield from stream.readline()
  File "c:\Python34_32bit\lib\asyncio\streams.py", line 425, in readline
    yield from self._wait_for_data('readline')
  File "c:\Python34_32bit\lib\asyncio\streams.py", line 393, in _wait_for_data
    yield from self._waiter
  File "c:\Python34_32bit\lib\asyncio\futures.py", line 386, in __iter__
    yield self  # This tells Task to wait for completion.
  File "c:\Python34_32bit\lib\asyncio\tasks.py", line 287, in _wakeup
    value = future.result()
  File "c:\Python34_32bit\lib\asyncio\futures.py", line 275, in result
    raise self._exception
  File "c:\Python34_32bit\lib\asyncio\selector_events.py", line 662, in _read_ready
    data = self._sock.recv(self.max_size)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "client.py", line 15, in <module>
    asyncio.get_event_loop().run_until_complete(hello())
  File "c:\Python34_32bit\lib\asyncio\base_events.py", line 316, in run_until_complete
    return future.result()
  File "c:\Python34_32bit\lib\asyncio\futures.py", line 275, in result
    raise self._exception
  File "c:\Python34_32bit\lib\asyncio\tasks.py", line 234, in _step
    result = coro.throw(exc)
  File "client.py", line 8, in hello
    websocket = yield from websockets.connect('ws://localhost:8765/')
  File "C:\tajf\lib\site-packages\websockets\client.py", line 124, in connect
    wsuri, origin=origin, subprotocols=subprotocols)
  File "C:\tajf\lib\site-packages\websockets\client.py", line 63, in handshake
    raise InvalidHandshake("Malformed HTTP message") from exc
websockets.exceptions.InvalidHandshake: Malformed HTTP message
Task was destroyed but it is pending!
task: <Task pending coro=<run() running at C:\tajf\lib\site-packages\websockets\protocol.py:232> wait_for=<Future pending cb=[Task._wakeup()]>>

Some other time it raises ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host with the same traceback.

Websocket won't close with 3.4.2

I'm following the loop example here: http://aaugustin.github.io/websockets/#example

I had code that would read from a queue (producer) and then publish to the websocket.
With Python 3.4.1, this code worked fine.
When I upgraded to Python 3.4.2, this code no longer detected when the other end of the socket was closed.

In fact, all I get in my console is "socket.send() raised exception", which sounds a lot like #23

I noticed the change logs for 3.4.3 and 3.4.4 mention some fixes and patches to the asyncio module. I'm using pyenv and it looks like it doesn't have those available at the moment so I can't quickly verify.

Expected? client: "Malformed HTTP message" -> destruction of pending asyncio tasks

I am using websockets to make hundreds of client connections.

As a test client, as load increases, and the number of connections approaches several hundred websockets connections per asyncio loop, I am seeing new connections fail with increasing frequency raising "Malformed HTTP message" exceptions.

I don't know if the Malformed HTTP is caused by my test server or not. But as shown below I see many asyncio tasks being destroyed while in a pending state. I am curious if this is expected behavior given the above.

Task was destroyed but it is pending! task: <Task pending coro=<run() running at /usr/local/lib/python3.4/site-packages/websockets/protocol.py:232> wait_for=<Future pending cb=[Task._wakeup()]>>

Obtain the client's ip.

When a client connects, I don't see a way to obtain their ip.
Storing the IP important is for my project in order to detect/prevent spam and attacks of all kinds. If there is a way to get the IP, pardon me, but I couldn't find it in the docs.

Warning when running the example code on Python 3.4

What's your name? mYk

mYk
< Hello mYk!
Task was destroyed but it is pending!
task: <Task pending coro=<run() running at /Users/myk/Documents/dev/websockets/websockets/protocol.py:253> wait_for=>

Add functions to allow proper subclassing of WebSocket*Protocol

As seen with the echo server example, https://docs.python.org/3/library/asyncio-protocol.html#echo-server - I would like to be able to use a subclass of WebSocket*Protocol (in my case the server) and override methods similar (perhaps not the same) as those to implement desired functionality. I could make a thin wrapper class that emulates this if it would be considered useful.

Thor performance benchmark

Hello,

I'm trying to run a performance benchmark using thor (node.js package) but websockets keeps closing the connections.
I've tracked down the problem to line 73 of framing.py, where apparently the mask check fails. I'm using websockets 2.2 with python 3.4.
While I understand that websockets aims for strict correctness of implementation it is also true that it's the only websocket library that doesn't seem to like the messages that thor generates (I've tested many in many different languages).

Do you think it could work in some more flexible mode where it could accept these input? Or could there be a bug in websockets protocol implementation?

Let me know if you need more info.

QueueEmpty exception in pip 1.0

Hello,

I've noticed a bug in pip's version 1.0 of websockets - protocol.py:141 fails because the right exception to catch is asyncio.queues.QueueEmpty.
However it appears to be fixed in current github version. Can you please update the pip's version?

Review connection close handling

The RFC says that:

• any side of the connection can initiate the websocket closing handshake
• the server should close the TCP connection; the client should wait for the server to do so

When I first read it, I misunderstood it as stating that the server should initiate the websocket closing handshake.

I must review the implementation to clear this misunderstanding.

Allow to pass event loop into websockets.serve()

websockets does not currently allows us to explicitly pass an event loop instance into websockets.serve(). Subsequent call to create_server() relies on existence of the default event loop, which is, however, not guaranteed, even in the main thread, since the calling code may deliberately suppress invoking of default event loop. According to PEP 3156:

It is okay to set the current event loop to None , in which case subsequent calls to get_event_loop() will raise an exception. This is useful for testing code that should not depend on the existence of a default event loop.

Also see https://www.python.org/dev/peps/pep-3156/#passing-an-event-loop-around-explicitly

Most of asyncio calls allow us to explicitly pass an event loop instance via the keyword-only loop argument. Please consider implementing such argument on websockets.serve() to benefit the code that not relies on the existence of the default event loop.

Correct form of closing the connection from the server

Hi. I'm running a websockets server. I was wondering what is the correct form of closing the connection from the server.
I include here the relevant part of my code:

@asyncio.coroutine
def ws_serve(self,websocket,path):
    """
    Maintains a websocket connection.
    """
    logging.info('Handling a new websocket connection')
    # Generate a new Authenticator class:
    at = Authenticator(websocket,path)

    while True:
        # Read a wrapped message from the websocket:
        wmsg = yield from websocket.recv()
        if wmsg is None:
            # Remote connection was closed.
            break

        # Handle the message at the Authenticator Class:
        yield from at.recv_msg(wmsg)


if __name__ == '__main__':
    logging.info('Starting websocket server')
    start_server = websockets.serve(ws_serve,'localhost',8765)
    eloop = asyncio.get_event_loop()
    eloop.run_until_complete(start_server)
    eloop.run_forever()

I thought about a few options to close the connection from the server:

1. Having at.recv_msg return a value, and check that value inside the ws_serve coroutine. The cons of this idea is that the point where I decide to close the server might be somewhere pretty deep inside Authenticator.
2. Raising some special exception inside the Authenticator class. This exception will propagate all the way from somewhere inside Authenticator to ws_serve, and the infinite loop at ws_serve will terminate.

I'm pretty new to asyncio, so there might be some design idea that I'm missing. Do you have any other ideas to solve this?

p.s. This won't be complete by saying how much I appreciate the work put into building this library. It really makes my life easier. Thank you :)

Ending pending tasks properly

I am writing a small script and would like to know how to properly deal with the scenario that the server exits while client is still connected. The idea is to catch SIGINT, close server, loop, and wait for pending tasks to complete before exiting. Right now, I run into this error:

Task was destroyed but it is pending!
task: <Task pending coro=<run() done, defined at /home/vagrant/.pyenv/versions/3.4.2/lib/python3.4/site-packages/websokets/protocol.py:222> wait_for=<Future pending cb=[Task._wakeup()]>>

Here is my current script. Thank you very much for your work, this library has been nice!

@asyncio.coroutine
def handler(websocket, path):
    while True:
        if not websocket.open:
            break

        yield from websocket.send("Hello!")
        yield from asyncio.sleep(1.0)

def boot_server(host, port):
    print('Atlas server is listening on http://', host, ':', port, sep='')

    return websockets.serve(handler, host, port)

def main():
    server = boot_server('0.0.0.0', 8765)
    event_loop = asyncio.get_event_loop()

    try:
        event_loop.run_until_complete(server)
        event_loop.run_forever()
    except KeyboardInterrupt:
        print('\nCtrl-C (SIGINT) caught. Exiting...')
    finally:
        server.close()
        event_loop.close()

if __name__ == "__main__":
    main()

collections.abc should get imported explicitly

Currently some of the failures of the unittests are involving that abc is not part of the collections module since version 3.3.

https://docs.python.org/3/library/collections.abc.html#module-collections.abc

“New in version 3.3: Formerly, this module was part of the collections module.”

======================================================================
ERROR: test_protocol_custom_request_headers_list (websockets.test_client_server.SSLClientServerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/adam/github/notmy/websockets/websockets/test_client_server.py", line 97, in test_protocol_custom_request_headers_list
    self.start_client('raw_headers', extra_headers=[('X-Spam', 'Eggs')])
  File "/home/adam/github/notmy/websockets/websockets/test_client_server.py", line 279, in start_client
    self.client = self.loop.run_until_complete(client)
  File "/usr/lib/python3.4/asyncio/base_events.py", line 316, in run_until_complete
    return future.result()
  File "/usr/lib/python3.4/asyncio/futures.py", line 275, in result
    raise self._exception
  File "/usr/lib/python3.4/asyncio/tasks.py", line 238, in _step
    result = next(coro)
  File "/home/adam/github/notmy/websockets/websockets/client.py", line 143, in connect
    extra_headers=extra_headers)
  File "/home/adam/github/notmy/websockets/websockets/client.py", line 53, in handshake
    if isinstance(extra_headers, collections.abc.Mapping):
AttributeError: 'module' object has no attribute 'abc'

collections.abc should get explicitly imported in client.py and server.py.

…
import collections
import collections.abc
…

Review cancellation management

I wrote this library before asyncio sorted out cancellation and shielding.

I'm not sure it handles cancellations correctly. See #50 for an example.

Unable to StreamWriter.drain()

I'm constantly getting the following exception:

Future/Task exception was never retrieved:
Traceback (most recent call last):
  File "/home/lolgall/venv/lib/python3.3/site-packages/asyncio/tasks.py", line 283, in _step
    result = next(coro)
  File "/home/lolgall/venv/lib/python3.3/site-packages/websockets/protocol.py", line 170, in send
    yield from self.write_frame(opcode, data)
  File "/home/lolgall/venv/lib/python3.3/site-packages/websockets/protocol.py", line 313, in write_frame
    yield from self.writer.drain()
  File "/home/lolgall/venv/lib/python3.3/site-packages/asyncio/streams.py", line 287, in drain
    return self._protocol._make_drain_waiter()
  File "/home/lolgall/venv/lib/python3.3/site-packages/asyncio/streams.py", line 184, in _make_drain_waiter
    assert waiter is None or waiter.cancelled()
AssertionError

What would be the reason?

Account for the removal of tulip.task in 27985f4

the wrapper task has been removed after revision 8c72a16d58a4

any revision after this one break websockets : https://code.google.com/p/tulip/source/browse/?r=8c72a16d58a4

Receive messages and periodically push data to client.

Hey, does anyone have a clue about how to push data to client, lets say every 0.5s, and still be able to receive messages?

By now I can only do one, or another and as I'm new to Python making it has proven quite a quest for me. Any ideas?

connect to a socket source that also includes other parameters

A bit of a novice here, so excuse me if this is obvious.
(I am trying to translate some node.js into python as well)

I am trying to create a connection to a web service that requires two keys to be passed in the connection string.

The node.js uses an npm module and looks like this:
var meshblu = require('meshblu');
var request = require('request');

      var conn = meshblu.createConnection({
      "uuid": "0d3a53a0-2a0b-11e3-b09c-ff4de847b2cc",
      "token": "qirqglm6yb1vpldixflopnux4phtcsor"
    });

    conn.on('ready', function(data){
    console.log('Ready');

They also have a REST interface that requires this in the header like this using PowerShell:
$authHeader = @{
meshblu_auth_uuid = $obj.uuid
meshblu_auth_token = $obj.token
}
$skynetUsers = Invoke-RestMethod -URI http://skynet.im/devices?type=user -Headers $authHeader -Method Get
$skynetUsers.devices

Any idea how to handle this with the websockets?

Cancelling a ping triggers an exception when the pong is received

Adding websocket.ping().cancel() to example/client.py results in the following stack trace:

Traceback (most recent call last):
  File "/Users/aaugustin/.virtualenvs/websockets/src/asyncio/asyncio/tasks.py", line 259, in _step
    result = coro.send(value)
  File "/Users/aaugustin/Documents/dev/websockets/websockets/protocol.py", line 207, in run
    msg = yield from self.read_message()
  File "/Users/aaugustin/Documents/dev/websockets/websockets/protocol.py", line 225, in read_message
    frame = yield from self.read_data_frame()
  File "/Users/aaugustin/Documents/dev/websockets/websockets/protocol.py", line 283, in read_data_frame
    waiter.set_result(None)
  File "/Users/aaugustin/.virtualenvs/websockets/src/asyncio/asyncio/futures.py", line 275, in set_result
    raise InvalidStateError('{}: {!r}'.format(self._state, self))
asyncio.futures.InvalidStateError: CANCELLED: Future<CANCELLED>

URI passed to handler function only includes path

Protocol, hostname and port are missing. Attempting to parse the URI with websockets.uri.parse_uri results in an InvalidURI exception.

pong responses are TCP fragmented; is this a bug?

I'm having trouble staying connected to a websocket using websockets 2.5 in python 3.4 on an Ubuntu 14.04 system. Bottom line, it looks like the automatic pongs sent by the Python client get TCP fragmented, sent in two packets. I'm guessing the server isn't handling that and dropping my client because it thinks it's dead. Does this sound plausible? If so, is it something the client library could change?

The websocket in question can be found on this page. Unfortunately the websocket URL requires a JWT that expires, so I can't link the socket endpoint directly, but FWIW the host is livestats.proxy.lolesports.com. If I connect to the websocket with the Python library it works fine for 90 seconds until my connection is dropped by the server. But a Javascript page in Chrome stays connected fine for an hour+ at a time.

I looked with Wireshark and discovered the server is sending a ping to the client every 45 seconds. Chrome sends a single packet back as a masked pong response and the server seems happy. The python library also sends a masked pong response back whose contents look correct. But I noticed in Wireshark that these pongs always are fragmented into two TCP packets; one for the mask data and one for the pong response itself (the string hello). I'm guessing the server doesn't understand that and decides to hang up on the client.

Does that analysis sound plausible? Am I right that the way websockets is coded, it sends pong packets as two fragments? Even if it's a bug in the server that it doesn't understand the fragmented pong, it might still be nice to have the python code avoid sending fragments both for efficiency and in case other servers have similar problems.

PS: thanks for your library, it's great!

release/websockets-2.0.tar.gz can't installed by pip, but from pypi is ok.

download websockets-2.0.tar.gz from release tab to d:/download. then install it use "pip install d:/download/websockets-2.0.tar.gz", met a error.
but websockets-2.0.tar.gz from pypi websit is ok.

Get client addr (host and port) on server?

I can't found this in documentation and source code. client.host and client.port returns host and port of server. I'm inattentive or it is still not implemented?

If this is not implemented, then simplest solution — self.transport = transport in connection_made. Then i can get host and port by client.transport.get_extra_info('peername').

Send to two clients in one server?How can I do?

I want send a message to two clients, how can i do? I can not find the client connect handle or client addr!

import asyncio
import websockets

@asyncio.coroutine
def hello(websocket, uri):
    name = yield from websocket.recv()
    print(uri)
    print("< {}".format(name))
    greeting = "Hello {}!".format(name)
    print("> {}".format(greeting))
    yield from websocket.send(greeting)

start_server = websockets.serve(hello, 'localhost', 8765)

asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()

Document how to wait on multiple futures

As requested here: edf3a32#commitcomment-10455621

test_client_server.py doesn't run on Debian

Maybe, I'm stupid?

~/repos/instaballot/vendor/websockets/websockets (master) > cat /etc/release
PRETTY_NAME="Debian GNU/Linux 7 (wheezy)"
NAME="Debian GNU/Linux"
VERSION_ID="7"
VERSION="7 (wheezy)"
ID=debian
ANSI_COLOR="1;31"
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support/"
BUG_REPORT_URL="http://bugs.debian.org/"

~/repos/instaballot/vendor/websockets/websockets (master) > python test_client_server.py
Traceback (most recent call last):
File "test_client_server.py", line 9, in
from .client import *
SystemError: Parent module '' not loaded, cannot perform relative import

~/repos/instaballot/vendor/websockets/websockets (master) > python --version
Python 3.4.2

test_protocol unittest failures on Windows

After fixing #62 there is still receive three failures. Note that these failures are identical for this repo and for my fork. Since these involving the protocol which I am not know enough yet to fix, I am dumping them to here.

Currently I am on my Linux environment:
4.0.7-2-ARCH
Python 3.4.3
64bit

If we got different failures here, I am here to help.

[adam@sza8215 websockets]$ python -m unittest
...........Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: /usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=10, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37256, 0, 0)>
  self._loop = events.get_event_loop()
/usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=9, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37256, 0, 0)>
  self._loop = events.get_event_loop()
/usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=12, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37257, 0, 0)>
  self._loop = events.get_event_loop()
/usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=11, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37257, 0, 0)>
  self._loop = events.get_event_loop()
/usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=14, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37258, 0, 0)>
  self._loop = events.get_event_loop()
/usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=13, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37258, 0, 0)>
  self._loop = events.get_event_loop()
....Exception ignored in: /usr/lib/python3.4/unittest/mock.py:1836: ResourceWarning: unclosed <socket.socket fd=15, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37259, 0, 0), raddr=('::1', 8642, 0, 0)>
  self.name = name
...............Exception ignored in: Exception ignored in: /usr/lib/python3.4/_collections_abc.py:584: ResourceWarning: unclosed <ssl.SSLSocket fd=9, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37276, 0, 0), raddr=('::1', 8642, 0, 0)>
  self, *args = args
/usr/lib/python3.4/_collections_abc.py:584: ResourceWarning: unclosed <ssl.SSLSocket fd=11, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37277, 0, 0), raddr=('::1', 8642, 0, 0)>
  self, *args = args
....Exception ignored in: Exception ignored in: /usr/lib/python3.4/asyncio/base_events.py:437: ResourceWarning: unclosed <ssl.SSLSocket fd=12, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37278, 0, 0), raddr=('::1', 8642, 0, 0)>
  handle = events.Handle(callback, args, self)
/usr/lib/python3.4/asyncio/base_events.py:437: ResourceWarning: unclosed <ssl.SSLSocket fd=9, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37279, 0, 0), raddr=('::1', 8642, 0, 0)>
  handle = events.Handle(callback, args, self)
...Exception ignored in: Exception ignored in: /usr/lib/python3.4/threading.py:213: ResourceWarning: unclosed <socket.socket fd=16, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37259, 0, 0)>
  self._lock = lock
/usr/lib/python3.4/threading.py:213: ResourceWarning: unclosed <socket.socket fd=10, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37266, 0, 0)>
  self._lock = lock
....Exception ignored in: /usr/lib/python3.4/selectors.py:396: ResourceWarning: unclosed <ssl.SSLSocket fd=10, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37286, 0, 0)>
  key = super().register(fileobj, events, data)
..........................................F..........................................F.F...............................
======================================================================
FAIL: test_close_timeout_before_eof_received (websockets.test_protocol.ClientTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/adam/github/notmy/websockets/websockets/test_protocol.py", line 480, in test_close_timeout_before_eof_received
    self.assertFalse(self.before.cancelled())
AssertionError: True is not false

======================================================================
FAIL: test_close_handshake_timeout (websockets.test_protocol.ServerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/adam/github/notmy/websockets/websockets/test_protocol.py", line 360, in test_close_handshake_timeout
    self.assertFalse(self.before.cancelled())
AssertionError: True is not false

======================================================================
FAIL: test_close_timeout_before_connection_lost (websockets.test_protocol.ServerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/adam/github/notmy/websockets/websockets/test_protocol.py", line 376, in test_close_timeout_before_connection_lost
    self.assertFalse(self.before.cancelled())
AssertionError: True is not false

----------------------------------------------------------------------
Ran 160 tests in 4.876s

FAILED (failures=3)

Add drain support for send messages

The send method simply writes to the transport, which internally buffers the message until flushed to kernel level IO.

When writing lots of messages the write-queue can fill up.

Maybe switching to StreamWriter class would be a choice ? Otherwise flow-control must be done in user-land which is ugly.

PyPI version fails

The basic example client.py and server.py is not working right now on a fresh install. But the repository version is working like a charm.

Please upload a fix to PyPI.

Websocket Server still running after close and wait_close coroutine calls

Hello,

Here is the situation:

• I have a server running a websocket instance (works)
• I serve clients with a SSL websocket (works)
• I want to perform, for any reason, a restart of the websocket server. In this case, Everything seem to work. The connection are closed etc. but when I try to run again the server, asyncio complains the event loop is sill running.

My environment!:
Both Ubuntu 14.04 LTS and Debian 8.0 jessie
Python 3.4
websocket from pypi (last version in date)

The corresponding reduced code:

class MyClass:
    def serve(self, interface, port):
        self.loop = asyncio.get_event_loop()
        if self.loop.is_running():
            self.loop.stop()

        self.wsserver = websockets.serve(WebSocketHandler,
                                         interface,
                                         port,
                                         loop=self.loop)
        self.server = self.loop.run_until_complete(self.wsserver)
        self.loop.run_forever()

    def close(self):
        self.server.close()
        self.loop.run_until_complete(self.server.wait_closed())
        self.loop.stop()


if __name__ == '__main__':
    instance  = MyClass()
    instance.serve('0.0.0.0', 4444)

    //In my program, the following calls are made from a signal handler
    instance.close()
    instance.serve('0.0.0.0', 5555)

The last call gives the error
"Event loop is running"

I tried to stop() and close(), the loop, wait for loop to stop running. But nothing seems to work.
Any idea why I cannot restart the server?
Thank you

Consider using service_identity for certification verification

As far as I know, the stdlib support for SSL isn't best-of-breed, and security cannot be up to end-users.

https://pypi.python.org/pypi/service_identity could be a worthy upgrade for the current certificate verification code.

Silent exception in server.py

If an exception is raised in example/server:hello [1], it's silenced. I'm not sure what solution is best, but for sure silencing the exception is not a good.

[1] https://github.com/aaugustin/websockets/blob/master/example/server.py#L7

Can't stop server correctly: "Task was destroyed but it is pending!"

If server has connected client (and waits conn.recv()), then Ctrl+C dont'close it, even if i call all closing functions:

server.close()
loop.run_until_complete(server.wait_closed())
loop.close()

I get this error:

Task was destroyed but it is pending!
task: <Task pending coro=<run() running at websockets/protocol.py:235> 
  wait_for=<Future pending cb=[Task._wakeup()]>
  cb=[_wait.<locals>._on_completion() at asyncio/tasks.py:399]>
Task was destroyed but it is pending!
task: <Task pending coro=<get() running at asyncio/queues.py:198> 
  wait_for=<Future pending cb=[Task._wakeup()]>
  cb=[_wait.<locals>._on_completion() at asyncio/tasks.py:399]>
Task was destroyed but it is pending!
task: <Task pending coro=<handler() running at websockets/server.py:64> 
  wait_for=<Future pending cb=[Task._wakeup()]>>

You can see full server and client example with this problem by jfs or simple server example in my original question on ru.SO.

I'm doing something wrong or there really a bug?