python-websockets / websockets Goto Github PK
View Code? Open in Web Editor NEWLibrary for building WebSocket servers and clients in Python
Home Page: https://websockets.readthedocs.io/
License: BSD 3-Clause "New" or "Revised" License
Library for building WebSocket servers and clients in Python
Home Page: https://websockets.readthedocs.io/
License: BSD 3-Clause "New" or "Revised" License
Currently some of the failures of the unittests are involving that abc
is not part of the collections
module since version 3.3.
https://docs.python.org/3/library/collections.abc.html#module-collections.abc
βNew in version 3.3: Formerly, this module was part of the collections module.β
======================================================================
ERROR: test_protocol_custom_request_headers_list (websockets.test_client_server.SSLClientServerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/adam/github/notmy/websockets/websockets/test_client_server.py", line 97, in test_protocol_custom_request_headers_list
self.start_client('raw_headers', extra_headers=[('X-Spam', 'Eggs')])
File "/home/adam/github/notmy/websockets/websockets/test_client_server.py", line 279, in start_client
self.client = self.loop.run_until_complete(client)
File "/usr/lib/python3.4/asyncio/base_events.py", line 316, in run_until_complete
return future.result()
File "/usr/lib/python3.4/asyncio/futures.py", line 275, in result
raise self._exception
File "/usr/lib/python3.4/asyncio/tasks.py", line 238, in _step
result = next(coro)
File "/home/adam/github/notmy/websockets/websockets/client.py", line 143, in connect
extra_headers=extra_headers)
File "/home/adam/github/notmy/websockets/websockets/client.py", line 53, in handshake
if isinstance(extra_headers, collections.abc.Mapping):
AttributeError: 'module' object has no attribute 'abc'
collections.abc
should get explicitly imported in client.py
and server.py
.
β¦
import collections
import collections.abc
β¦
download websockets-2.0.tar.gz from release tab to d:/download. then install it use "pip install d:/download/websockets-2.0.tar.gz", met a error.
but websockets-2.0.tar.gz from pypi websit is ok.
Protocol, hostname and port are missing. Attempting to parse the URI with websockets.uri.parse_uri results in an InvalidURI exception.
Hello,
I'm trying to run a performance benchmark using thor (node.js package) but websockets keeps closing the connections.
I've tracked down the problem to line 73 of framing.py, where apparently the mask check fails. I'm using websockets 2.2 with python 3.4.
While I understand that websockets aims for strict correctness of implementation it is also true that it's the only websocket library that doesn't seem to like the messages that thor generates (I've tested many in many different languages).
Do you think it could work in some more flexible mode where it could accept these input? Or could there be a bug in websockets protocol implementation?
Let me know if you need more info.
Cheers, thanks for writing this
One feature I am missing is the ability to enforce a maximum message size for received messages, i.e. set an arbitrary size limit for the socket and disconnect with error (1009) if a message is too large.
A bit of a novice here, so excuse me if this is obvious.
(I am trying to translate some node.js into python as well)
I am trying to create a connection to a web service that requires two keys to be passed in the connection string.
The node.js uses an npm module and looks like this:
var meshblu = require('meshblu');
var request = require('request');
var conn = meshblu.createConnection({
"uuid": "0d3a53a0-2a0b-11e3-b09c-ff4de847b2cc",
"token": "qirqglm6yb1vpldixflopnux4phtcsor"
});
conn.on('ready', function(data){
console.log('Ready');
They also have a REST interface that requires this in the header like this using PowerShell:
$authHeader = @{
meshblu_auth_uuid = $obj.uuid
meshblu_auth_token = $obj.token
}
$skynetUsers = Invoke-RestMethod -URI http://skynet.im/devices?type=user -Headers $authHeader -Method Get
$skynetUsers.devices
Any idea how to handle this with the websockets?
The HTTP parsing and generation code is very simple at this time.
It could be worth relying on an external library for these parts.
During shutdown I wait for clients to close connections. When this is done, the handler is not allowed to finish.
To trigger the error, run the following script, interrupt it (Ctrl+C) with a connection open, then close the connection.
#!/usr/bin/env python
import asyncio
import logging
import signal
import websockets
logging.getLogger().setLevel(logging.DEBUG)
connectionClosedEvents = []
@asyncio.coroutine
def echo(websocket, path):
print("Connection opened.")
connectionClosedEvent = asyncio.Event()
connectionClosedEvents.append(connectionClosedEvent)
s = yield from websocket.recv()
if s is not None:
yield from websocket.send(s)
print("Connection closed.")
connectionClosedEvent.set()
@asyncio.coroutine
def shutdown():
print("Shutdown initialized.")
print("Waiting for open connections to close ...")
for e in connectionClosedEvents:
yield from e.wait()
print("All connections closed.")
echoServer.close()
print("Waiting for server to close ...")
yield from echoServer.wait_closed()
print("Server closed.")
print("Stopping event loop ...")
eventLoop.stop()
print("Event loop stopped.")
print("Shutdown complete.")
eventLoop = asyncio.get_event_loop()
eventLoop.add_signal_handler(signal.SIGINT, asyncio.async, shutdown())
echoServer =\
eventLoop.run_until_complete(websockets.serve(echo, "0.0.0.0", 8765))
eventLoop.run_forever()
Typical output:
Connection opened.
^CShutdown initialized.
Waiting for open connections to close ...
Connection closed.
All connections closed.
Waiting for server to close ...
Server closed.
Stopping event loop ...
Event loop stopped.
Shutdown complete.
Task was destroyed but it is pending!
task: <Task pending coro=<handler() running at /tmp/tmp.OhmZJ4uKUJ/venv/lib/python3.4/site-packages/websockets/server.py:77> wait_for=<Future finished result=None>>
The reason for the error is that websockets.server.connection_lost
removes the connection from self.websockets
. Thus in websockets.server.wait_closed
the connection is not waited for:
yield from asyncio.wait(
[ws.handler_task for ws in self.websockets], loop=self.loop)
I will submit a proposed fix for this shortly.
What's your name? mYk
mYk
< Hello mYk!
Task was destroyed but it is pending!
task: <Task pending coro=<run() running at /Users/myk/Documents/dev/websockets/websockets/protocol.py:253> wait_for=>
Hello,
I've noticed a bug in pip's version 1.0 of websockets - protocol.py:141 fails because the right exception to catch is asyncio.queues.QueueEmpty.
However it appears to be fixed in current github version. Can you please update the pip's version?
I am writing a small script and would like to know how to properly deal with the scenario that the server exits while client is still connected. The idea is to catch SIGINT, close server, loop, and wait for pending tasks to complete before exiting. Right now, I run into this error:
Task was destroyed but it is pending!
task: <Task pending coro=<run() done, defined at /home/vagrant/.pyenv/versions/3.4.2/lib/python3.4/site-packages/websokets/protocol.py:222> wait_for=<Future pending cb=[Task._wakeup()]>>
Here is my current script. Thank you very much for your work, this library has been nice!
@asyncio.coroutine
def handler(websocket, path):
while True:
if not websocket.open:
break
yield from websocket.send("Hello!")
yield from asyncio.sleep(1.0)
def boot_server(host, port):
print('Atlas server is listening on http://', host, ':', port, sep='')
return websockets.serve(handler, host, port)
def main():
server = boot_server('0.0.0.0', 8765)
event_loop = asyncio.get_event_loop()
try:
event_loop.run_until_complete(server)
event_loop.run_forever()
except KeyboardInterrupt:
print('\nCtrl-C (SIGINT) caught. Exiting...')
finally:
server.close()
event_loop.close()
if __name__ == "__main__":
main()
I am using websockets to make hundreds of client connections.
As a test client, as load increases, and the number of connections approaches several hundred websockets connections per asyncio loop, I am seeing new connections fail with increasing frequency raising "Malformed HTTP message" exceptions.
I don't know if the Malformed HTTP is caused by my test server or not. But as shown below I see many asyncio tasks being destroyed while in a pending state. I am curious if this is expected behavior given the above.
Task was destroyed but it is pending! task: <Task pending coro=<run() running at /usr/local/lib/python3.4/site-packages/websockets/protocol.py:232> wait_for=<Future pending cb=[Task._wakeup()]>>
Would it be possible to add high-level on_open() and on_close() methods, similar to the web sockets implementation in Tornado, so that initialisation and cleanup can be performed?
As requested here: edf3a32#commitcomment-10455621
the wrapper task has been removed after revision 8c72a16d58a4
any revision after this one break websockets : https://code.google.com/p/tulip/source/browse/?r=8c72a16d58a4
I wrote this library before asyncio sorted out cancellation and shielding.
I'm not sure it handles cancellations correctly. See #50 for an example.
Hi. I'm running a websockets server. I was wondering what is the correct form of closing the connection from the server.
I include here the relevant part of my code:
@asyncio.coroutine
def ws_serve(self,websocket,path):
"""
Maintains a websocket connection.
"""
logging.info('Handling a new websocket connection')
# Generate a new Authenticator class:
at = Authenticator(websocket,path)
while True:
# Read a wrapped message from the websocket:
wmsg = yield from websocket.recv()
if wmsg is None:
# Remote connection was closed.
break
# Handle the message at the Authenticator Class:
yield from at.recv_msg(wmsg)
if __name__ == '__main__':
logging.info('Starting websocket server')
start_server = websockets.serve(ws_serve,'localhost',8765)
eloop = asyncio.get_event_loop()
eloop.run_until_complete(start_server)
eloop.run_forever()
I thought about a few options to close the connection from the server:
I'm pretty new to asyncio, so there might be some design idea that I'm missing. Do you have any other ideas to solve this?
p.s. This won't be complete by saying how much I appreciate the work put into building this library. It really makes my life easier. Thank you :)
Hello there,
I'm certain this is not the correct place for this question, but it does involve the websockets library rather directly and may be beneficial for other users by way of documentation. So apologies for bringing this up if it's just noise.
I want to know what the best way for me to "block on" two separate asyncio-enabled libraries simultaneously is. By "block on" I mean I want to react immediately to messages from two different libraries' "yield from" calls.
In my app, the two libraries are websockets
and asyncio_redis
, but I presume it's the same regardless of which two async libraries you'd want to do this with. Here's an example of my websocket server and my current (working) solution:
#!/usr/bin/env python3
import asyncio
import asyncio_redis
import websockets
@asyncio.coroutine
def handler(websocket, path):
redis = yield from asyncio_redis.Connection.create(host='localhost', port=6379)
subscriber_conn = yield from asyncio_redis.Connection.create(host='localhost', port=6379)
subscriber = yield from subscriber_conn.start_subscribe()
yield from subscriber.subscribe(['the-channel'])
# continually check messages
while True:
client_msg = websocket.recv()
redis_msg = subscriber.next_published()
done_tasks, pending_tasks = yield from asyncio.wait((client_msg, redis_msg), return_when=asyncio.FIRST_COMPLETED)
for task in done_tasks:
if task._coro == client_msg:
yield from handle_client_message(task.result())
if task._coro == redis_msg:
yield from handle_redis_message(task.result())
if __name__ == '__main__':
start_server = websockets.serve(handler, 'localhost', 9000)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
handle_client_message
and handle_redis_message
are just stand-ins for the real app code, but I didn't want to clutter up the example any further.
You can see from this code that the key mechanism I'm using is:
done_tasks, pending_tasks = yield from asyncio.wait([some_tasks], asyncio.FIRST_COMPLETED)
This does what I want, but I can't help but think it's The Wrong Way to do this (not least because I then have to use a private variable in task._coro
to decide which task fired the result).
What's the expected way to tie websockets in with other libraries such that you don't have to have a request/reply style of app (as the echo example is)?
The current documentation is minimal.
It could include a tutorial, background on async programming, uses cases, etc.
The RFC6455 Section 1.9 leaves us with a header field to negotiate a sub-protocol during the Websocket handshake. It would be nice if you could allow your library users to interfere at this time so we can see which protocols are advertised by the client, choose one of them or enforce a completely different value and send it back to the client. Of course the ws_handler
needs to know which protocol has been chosen.
When a client connects, I don't see a way to obtain their ip.
Storing the IP important is for my project in order to detect/prevent spam and attacks of all kinds. If there is a way to get the IP, pardon me, but I couldn't find it in the docs.
Steal this: Polyconseil/aioamqp@c46f9a0
When my test SSL client closes the connection to my test SSL server, the closing of the connection fails an assertion on line 338 in protocol.py. However, when I change the assertion to an if-statement, it works just fine. My server and client are given below. The server and client use the same "cert.pem" file create with the following command:
openssl req -new -x509 -days 3650 -nodes -out cert.pem -keyout cert.pem
Server
import asyncio
from ssl import PROTOCOL_TLSv1, SSLContext
import websockets
@asyncio.coroutine
def hello(websocket, path):
name = yield from websocket.recv()
print("< {}".format(name))
greeting = "Hello {}!".format(name)
print("> {}".format(greeting))
yield from websocket.send(greeting)
ssl_context = SSLContext(PROTOCOL_TLSv1)
ssl_context.load_cert_chain('cert.pem')
kwds = {'ssl': ssl_context}
start_server = websockets.serve(hello, 'localhost', 8765, **kwds)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
Client
import asyncio
from ssl import PROTOCOL_TLSv1, SSLContext
import websockets
@asyncio.coroutine
def hello():
ssl_context = SSLContext(PROTOCOL_TLSv1)
ssl_context.load_cert_chain('cert.pem')
kwds = {'ssl': ssl_context}
websocket = yield from websockets.connect('wss://localhost:8765/', **kwds)
name = input("What's your name? ")
yield from websocket.send(name)
print("> {}".format(name))
greeting = yield from websocket.recv()
print("< {}".format(greeting))
asyncio.get_event_loop().run_until_complete(hello())
The RFC says that:
When I first read it, I misunderstood it as stating that the server should initiate the websocket closing handshake.
I must review the implementation to clear this misunderstanding.
Hello,
Here is the situation:
My environment!:
Both Ubuntu 14.04 LTS and Debian 8.0 jessie
Python 3.4
websocket from pypi (last version in date)
The corresponding reduced code:
class MyClass:
def serve(self, interface, port):
self.loop = asyncio.get_event_loop()
if self.loop.is_running():
self.loop.stop()
self.wsserver = websockets.serve(WebSocketHandler,
interface,
port,
loop=self.loop)
self.server = self.loop.run_until_complete(self.wsserver)
self.loop.run_forever()
def close(self):
self.server.close()
self.loop.run_until_complete(self.server.wait_closed())
self.loop.stop()
if __name__ == '__main__':
instance = MyClass()
instance.serve('0.0.0.0', 4444)
//In my program, the following calls are made from a signal handler
instance.close()
instance.serve('0.0.0.0', 5555)
The last call gives the error
"Event loop is running"
I tried to stop() and close(), the loop, wait for loop to stop running. But nothing seems to work.
Any idea why I cannot restart the server?
Thank you
As far as I know, the stdlib support for SSL isn't best-of-breed, and security cannot be up to end-users.
https://pypi.python.org/pypi/service_identity could be a worthy upgrade for the current certificate verification code.
I'm constantly getting the following exception:
Future/Task exception was never retrieved:
Traceback (most recent call last):
File "/home/lolgall/venv/lib/python3.3/site-packages/asyncio/tasks.py", line 283, in _step
result = next(coro)
File "/home/lolgall/venv/lib/python3.3/site-packages/websockets/protocol.py", line 170, in send
yield from self.write_frame(opcode, data)
File "/home/lolgall/venv/lib/python3.3/site-packages/websockets/protocol.py", line 313, in write_frame
yield from self.writer.drain()
File "/home/lolgall/venv/lib/python3.3/site-packages/asyncio/streams.py", line 287, in drain
return self._protocol._make_drain_waiter()
File "/home/lolgall/venv/lib/python3.3/site-packages/asyncio/streams.py", line 184, in _make_drain_waiter
assert waiter is None or waiter.cancelled()
AssertionError
What would be the reason?
Expectation is that with python 3.4.0 that the built in asyncio would be used, not an external module.
Currently a raw representation as a list of 2-uples is available in WebSocketClient/ServerProtocol.raw_request/response_headers
.
Since these are read-only attributes, the problems that the API should solve include:
email.parser.BytesHeaderParser()
handles everything for us, I'm not sureThe basic example client.py and server.py is not working right now on a fresh install. But the repository version is working like a charm.
Please upload a fix to PyPI.
Hey, does anyone have a clue about how to push data to client, lets say every 0.5s, and still be able to receive messages?
By now I can only do one, or another and as I'm new to Python making it has proven quite a quest for me. Any ideas?
Python: 3.4
OS: Windows 7 (Firewall turned off)
Virtual environment was used (pyvenv).
Server code:
#!/usr/bin/env python
import asyncio
import threading
import websockets
@asyncio.coroutine
def hello(websocket, path):
name = yield from websocket.recv()
print("< {}".format(name))
greeting = "Hello {}!".format(name)
yield from websocket.send(greeting)
print("> {}".format(greeting))
class ServerThread(threading.Thread):
def run(self):
self.loop = asyncio.new_event_loop()
coro = websockets.serve(hello, 'localhost', 8765,
loop=self.loop)
self.loop.run_until_complete(coro)
self.loop.run_forever()
ServerThread().start()
Exception traceback in client.py (documentation example code):
Traceback (most recent call last):
File "C:\tajf\lib\site-packages\websockets\client.py", line 61, in handshake
status_code, headers = yield from read_response(self.reader)
File "C:\tajf\lib\site-packages\websockets\http.py", line 63, in read_response
status_line, headers = yield from read_message(stream)
File "C:\tajf\lib\site-packages\websockets\http.py", line 80, in read_message
start_line = yield from read_line(stream)
File "C:\tajf\lib\site-packages\websockets\http.py", line 99, in read_line
line = yield from stream.readline()
File "c:\Python34_32bit\lib\asyncio\streams.py", line 425, in readline
yield from self._wait_for_data('readline')
File "c:\Python34_32bit\lib\asyncio\streams.py", line 393, in _wait_for_data
yield from self._waiter
File "c:\Python34_32bit\lib\asyncio\futures.py", line 386, in __iter__
yield self # This tells Task to wait for completion.
File "c:\Python34_32bit\lib\asyncio\tasks.py", line 287, in _wakeup
value = future.result()
File "c:\Python34_32bit\lib\asyncio\futures.py", line 275, in result
raise self._exception
File "c:\Python34_32bit\lib\asyncio\selector_events.py", line 662, in _read_ready
data = self._sock.recv(self.max_size)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "client.py", line 15, in <module>
asyncio.get_event_loop().run_until_complete(hello())
File "c:\Python34_32bit\lib\asyncio\base_events.py", line 316, in run_until_complete
return future.result()
File "c:\Python34_32bit\lib\asyncio\futures.py", line 275, in result
raise self._exception
File "c:\Python34_32bit\lib\asyncio\tasks.py", line 234, in _step
result = coro.throw(exc)
File "client.py", line 8, in hello
websocket = yield from websockets.connect('ws://localhost:8765/')
File "C:\tajf\lib\site-packages\websockets\client.py", line 124, in connect
wsuri, origin=origin, subprotocols=subprotocols)
File "C:\tajf\lib\site-packages\websockets\client.py", line 63, in handshake
raise InvalidHandshake("Malformed HTTP message") from exc
websockets.exceptions.InvalidHandshake: Malformed HTTP message
Task was destroyed but it is pending!
task: <Task pending coro=<run() running at C:\tajf\lib\site-packages\websockets\protocol.py:232> wait_for=<Future pending cb=[Task._wakeup()]>>
Some other time it raises ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
with the same traceback.
This is what I have in my test. (stop_client being called in tearDown)
def start_client(self, path='', **kwds):
client = connect('ws://localhost:8765/' + path, **kwds)
self.client = self.loop.run_until_complete(client)
def stop_client(self):
self.client.close()
print("$ Wait for client to terminate")
self.loop.run_until_complete(self.client.worker)
print("$ Client stopped")
websockets
does not currently allows us to explicitly pass an event loop instance into websockets.serve()
. Subsequent call to create_server()
relies on existence of the default event loop, which is, however, not guaranteed, even in the main thread, since the calling code may deliberately suppress invoking of default event loop. According to PEP 3156:
It is okay to set the current event loop to None , in which case subsequent calls to get_event_loop() will raise an exception. This is useful for testing code that should not depend on the existence of a default event loop.
Also see https://www.python.org/dev/peps/pep-3156/#passing-an-event-loop-around-explicitly
Most of asyncio
calls allow us to explicitly pass an event loop instance via the keyword-only loop
argument. Please consider implementing such argument on websockets.serve()
to benefit the code that not relies on the existence of the default event loop.
As seen with the echo server example, https://docs.python.org/3/library/asyncio-protocol.html#echo-server - I would like to be able to use a subclass of WebSocket*Protocol (in my case the server) and override methods similar (perhaps not the same) as those to implement desired functionality. I could make a thin wrapper class that emulates this if it would be considered useful.
If server has connected client (and waits conn.recv()
), then Ctrl+C dont'close it, even if i call all closing functions:
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()
I get this error:
Task was destroyed but it is pending!
task: <Task pending coro=<run() running at websockets/protocol.py:235>
wait_for=<Future pending cb=[Task._wakeup()]>
cb=[_wait.<locals>._on_completion() at asyncio/tasks.py:399]>
Task was destroyed but it is pending!
task: <Task pending coro=<get() running at asyncio/queues.py:198>
wait_for=<Future pending cb=[Task._wakeup()]>
cb=[_wait.<locals>._on_completion() at asyncio/tasks.py:399]>
Task was destroyed but it is pending!
task: <Task pending coro=<handler() running at websockets/server.py:64>
wait_for=<Future pending cb=[Task._wakeup()]>>
You can see full server and client example with this problem by jfs or simple server example in my original question on ru.SO.
I'm doing something wrong or there really a bug?
I can't found this in documentation and source code. client.host
and client.port
returns host and port of server. I'm inattentive or it is still not implemented?
If this is not implemented, then simplest solution β self.transport = transport
in connection_made. Then i can get host and port by client.transport.get_extra_info('peername')
.
I'd like to have some HTTP pages alongside my websocket handler.
How should I integrate?
Would this works?
#!/usr/bin/env python
import asyncio
from aiohttp import web
import websockets
@asyncio.coroutine
def hello(websocket, path):
name = yield from websocket.recv()
print("< {}".format(name))
greeting = "Hello {}!".format(name)
yield from websocket.send(greeting)
print("> {}".format(greeting))
@asyncio.coroutine
def handle(request):
name = request.match_info.get('name', "Anonymous")
text = "Hello, " + name
return web.Response(body=text.encode('utf-8'))
@asyncio.coroutine
def init_http_server(loop):
app = web.Application(loop=loop)
app.router.add_route('GET', '/{name}', handle)
srv = yield from loop.create_server(app.make_handler(),
'127.0.0.1', 8080)
print("Server started at http://127.0.0.1:8080")
return srv
loop = asyncio.get_event_loop()
websocket_server = websockets.serve(hello, 'localhost', 8765)
http_server = init_http_server(loop)
loop.run_until_complete(websocket_server)
loop.run_until_complete(http_server)
loop.run_forever()
If an exception is raised in example/server:hello [1], it's silenced. I'm not sure what solution is best, but for sure silencing the exception is not a good.
[1] https://github.com/aaugustin/websockets/blob/master/example/server.py#L7
The send method simply writes to the transport, which internally buffers the message until flushed to kernel level IO.
When writing lots of messages the write-queue can fill up.
Maybe switching to StreamWriter class would be a choice ? Otherwise flow-control must be done in user-land which is ugly.
I want send a message to two clients, how can i do? I can not find the client connect handle or client addr!
import asyncio
import websockets
@asyncio.coroutine
def hello(websocket, uri):
name = yield from websocket.recv()
print(uri)
print("< {}".format(name))
greeting = "Hello {}!".format(name)
print("> {}".format(greeting))
yield from websocket.send(greeting)
start_server = websockets.serve(hello, 'localhost', 8765)
asyncio.get_event_loop().run_until_complete(start_server)
asyncio.get_event_loop().run_forever()
I'm following the loop example here: http://aaugustin.github.io/websockets/#example
I had code that would read from a queue (producer) and then publish to the websocket.
With Python 3.4.1, this code worked fine.
When I upgraded to Python 3.4.2, this code no longer detected when the other end of the socket was closed.
In fact, all I get in my console is "socket.send() raised exception", which sounds a lot like #23
I noticed the change logs for 3.4.3 and 3.4.4 mention some fixes and patches to the asyncio module. I'm using pyenv and it looks like it doesn't have those available at the moment so I can't quickly verify.
The following command:
pip install git+https://github.com/aaugustin/websockets.git
with the following logged by pip:
------------------------------------------------------------
/home/amirouche/.virtualenvs/suprm3/bin/pip run on Thu Oct 24 01:24:34 2013
Downloading/unpacking git+https://github.com/aaugustin/websockets.git
Cloning https://github.com/aaugustin/websockets.git to /tmp/pip-tgmzlg-build
Found command 'git' at '/usr/bin/git'
Running command /usr/bin/git clone -q https://github.com/aaugustin/websockets.git /tmp/pip-tgmzlg-build
Running setup.py egg_info for package from git+https://github.com/aaugustin/websockets.git
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/tmp/pip-tgmzlg-build/setup.py", line 11, in <module>
with open(os.path.join(root, 'README')) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-tgmzlg-build/README'
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/tmp/pip-tgmzlg-build/setup.py", line 11, in <module>
with open(os.path.join(root, 'README')) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-tgmzlg-build/README'
----------------------------------------
Command python setup.py egg_info failed with error code 1 in /tmp/pip-tgmzlg-build
Exception information:
Traceback (most recent call last):
File "/home/amirouche/.virtualenvs/suprm3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg/pip/basecommand.py", line 139, in main
status = self.run(options, args)
File "/home/amirouche/.virtualenvs/suprm3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg/pip/commands/install.py", line 266, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/home/amirouche/.virtualenvs/suprm3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg/pip/req.py", line 1057, in prepare_files
req_to_install.run_egg_info()
File "/home/amirouche/.virtualenvs/suprm3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg/pip/req.py", line 236, in run_egg_info
command_desc='python setup.py egg_info')
File "/home/amirouche/.virtualenvs/suprm3/lib/python3.3/site-packages/pip-1.3.1-py3.3.egg/pip/util.py", line 662, in call_subprocess
% (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command python setup.py egg_info failed with error code 1 in /tmp/pip-tgmzlg-build
After fixing #62 there is still receive three failures. Note that these failures are identical for this repo and for my fork. Since these involving the protocol which I am not know enough yet to fix, I am dumping them to here.
Currently I am on my Linux environment:
4.0.7-2-ARCH
Python 3.4.3
64bit
If we got different failures here, I am here to help.
[adam@sza8215 websockets]$ python -m unittest
...........Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: Exception ignored in: /usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=10, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37256, 0, 0)>
self._loop = events.get_event_loop()
/usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=9, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37256, 0, 0)>
self._loop = events.get_event_loop()
/usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=12, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37257, 0, 0)>
self._loop = events.get_event_loop()
/usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=11, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37257, 0, 0)>
self._loop = events.get_event_loop()
/usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=14, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37258, 0, 0)>
self._loop = events.get_event_loop()
/usr/lib/python3.4/asyncio/futures.py:151: ResourceWarning: unclosed <socket.socket fd=13, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37258, 0, 0)>
self._loop = events.get_event_loop()
....Exception ignored in: /usr/lib/python3.4/unittest/mock.py:1836: ResourceWarning: unclosed <socket.socket fd=15, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37259, 0, 0), raddr=('::1', 8642, 0, 0)>
self.name = name
...............Exception ignored in: Exception ignored in: /usr/lib/python3.4/_collections_abc.py:584: ResourceWarning: unclosed <ssl.SSLSocket fd=9, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37276, 0, 0), raddr=('::1', 8642, 0, 0)>
self, *args = args
/usr/lib/python3.4/_collections_abc.py:584: ResourceWarning: unclosed <ssl.SSLSocket fd=11, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37277, 0, 0), raddr=('::1', 8642, 0, 0)>
self, *args = args
....Exception ignored in: Exception ignored in: /usr/lib/python3.4/asyncio/base_events.py:437: ResourceWarning: unclosed <ssl.SSLSocket fd=12, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37278, 0, 0), raddr=('::1', 8642, 0, 0)>
handle = events.Handle(callback, args, self)
/usr/lib/python3.4/asyncio/base_events.py:437: ResourceWarning: unclosed <ssl.SSLSocket fd=9, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 37279, 0, 0), raddr=('::1', 8642, 0, 0)>
handle = events.Handle(callback, args, self)
...Exception ignored in: Exception ignored in: /usr/lib/python3.4/threading.py:213: ResourceWarning: unclosed <socket.socket fd=16, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37259, 0, 0)>
self._lock = lock
/usr/lib/python3.4/threading.py:213: ResourceWarning: unclosed <socket.socket fd=10, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37266, 0, 0)>
self._lock = lock
....Exception ignored in: /usr/lib/python3.4/selectors.py:396: ResourceWarning: unclosed <ssl.SSLSocket fd=10, family=AddressFamily.AF_INET6, type=2049, proto=6, laddr=('::1', 8642, 0, 0), raddr=('::1', 37286, 0, 0)>
key = super().register(fileobj, events, data)
..........................................F..........................................F.F...............................
======================================================================
FAIL: test_close_timeout_before_eof_received (websockets.test_protocol.ClientTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/adam/github/notmy/websockets/websockets/test_protocol.py", line 480, in test_close_timeout_before_eof_received
self.assertFalse(self.before.cancelled())
AssertionError: True is not false
======================================================================
FAIL: test_close_handshake_timeout (websockets.test_protocol.ServerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/adam/github/notmy/websockets/websockets/test_protocol.py", line 360, in test_close_handshake_timeout
self.assertFalse(self.before.cancelled())
AssertionError: True is not false
======================================================================
FAIL: test_close_timeout_before_connection_lost (websockets.test_protocol.ServerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/adam/github/notmy/websockets/websockets/test_protocol.py", line 376, in test_close_timeout_before_connection_lost
self.assertFalse(self.before.cancelled())
AssertionError: True is not false
----------------------------------------------------------------------
Ran 160 tests in 4.876s
FAILED (failures=3)
Adding websocket.ping().cancel()
to example/client.py
results in the following stack trace:
Traceback (most recent call last):
File "/Users/aaugustin/.virtualenvs/websockets/src/asyncio/asyncio/tasks.py", line 259, in _step
result = coro.send(value)
File "/Users/aaugustin/Documents/dev/websockets/websockets/protocol.py", line 207, in run
msg = yield from self.read_message()
File "/Users/aaugustin/Documents/dev/websockets/websockets/protocol.py", line 225, in read_message
frame = yield from self.read_data_frame()
File "/Users/aaugustin/Documents/dev/websockets/websockets/protocol.py", line 283, in read_data_frame
waiter.set_result(None)
File "/Users/aaugustin/.virtualenvs/websockets/src/asyncio/asyncio/futures.py", line 275, in set_result
raise InvalidStateError('{}: {!r}'.format(self._state, self))
asyncio.futures.InvalidStateError: CANCELLED: Future<CANCELLED>
I'm having trouble staying connected to a websocket using websockets 2.5 in python 3.4 on an Ubuntu 14.04 system. Bottom line, it looks like the automatic pongs sent by the Python client get TCP fragmented, sent in two packets. I'm guessing the server isn't handling that and dropping my client because it thinks it's dead. Does this sound plausible? If so, is it something the client library could change?
The websocket in question can be found on this page. Unfortunately the websocket URL requires a JWT that expires, so I can't link the socket endpoint directly, but FWIW the host is livestats.proxy.lolesports.com. If I connect to the websocket with the Python library it works fine for 90 seconds until my connection is dropped by the server. But a Javascript page in Chrome stays connected fine for an hour+ at a time.
I looked with Wireshark and discovered the server is sending a ping to the client every 45 seconds. Chrome sends a single packet back as a masked pong response and the server seems happy. The python library also sends a masked pong response back whose contents look correct. But I noticed in Wireshark that these pongs always are fragmented into two TCP packets; one for the mask data and one for the pong response itself (the string hello
). I'm guessing the server doesn't understand that and decides to hang up on the client.
Does that analysis sound plausible? Am I right that the way websockets is coded, it sends pong packets as two fragments? Even if it's a bug in the server that it doesn't understand the fragmented pong, it might still be nice to have the python code avoid sending fragments both for efficiency and in case other servers have similar problems.
PS: thanks for your library, it's great!
Maybe, I'm stupid?
~/repos/instaballot/vendor/websockets/websockets (master) > cat /etc/release
PRETTY_NAME="Debian GNU/Linux 7 (wheezy)"
NAME="Debian GNU/Linux"
VERSION_ID="7"
VERSION="7 (wheezy)"
ID=debian
ANSI_COLOR="1;31"
HOME_URL="http://www.debian.org/"
SUPPORT_URL="http://www.debian.org/support/"
BUG_REPORT_URL="http://bugs.debian.org/"
~/repos/instaballot/vendor/websockets/websockets (master) > python test_client_server.py
Traceback (most recent call last):
File "test_client_server.py", line 9, in
from .client import *
SystemError: Parent module '' not loaded, cannot perform relative import
~/repos/instaballot/vendor/websockets/websockets (master) > python --version
Python 3.4.2
Step 19 : RUN pip-3.3 install websockets
---> Running in 0621d2d66601
Downloading/unpacking websockets
Downloading websockets-1.0.tar.gz
Running setup.py egg_info for package websockets
Traceback (most recent call last):
File "", line 16, in
File "/tmp/pip_build_root/websockets/setup.py", line 18, in
long_description = '\n\n'.join(f.read().split('\n\n')[1:])
File "/usr/lib/python3.3/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 467: ordinal not in range(128)
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 16, in
File "/tmp/pip_build_root/websockets/setup.py", line 18, in
long_description = '\n\n'.join(f.read().split('\n\n')[1:])
File "/usr/lib/python3.3/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 467: ordinal not in range(128
I have two tasks in the asyncio thread, one being the websocket connection handler and the other processing items from a pipe and sending data to websocket clients.
The code can be seen here: https://raw.githubusercontent.com/marksamman/pylinkshortener/master/app/websocket/__init__.py
Under heavy load (connecting a websocket client, running apachebench to flood the pipe, disconnect while it's being flooded), I noticed "socket.send() raised exception." being printed and that the application eventually crashes. Output:
[socket.send() raised exception repeated over 1000 times ...]
socket.send() raised exception.
socket.send() raised exception.
socket.send() raised exception.
Future/Task exception was never retrieved
future: Task(<handler>)<exception=AttributeError("'NoneType' object has no attribute 'shutdown'",)>
Traceback (most recent call last):
File "/usr/local/lib/python3.4/site-packages/websockets/server.py", line 60, in handler
yield from self.close()
File "/usr/local/lib/python3.4/site-packages/websockets/protocol.py", line 119, in close
yield from asyncio.wait_for(self.worker, timeout=self.timeout)
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py", line 413, in wait_for
return fut.result()
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/futures.py", line 243, in result
raise self._exception
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py", line 279, in _step
result = coro.throw(exc)
File "/usr/local/lib/python3.4/site-packages/websockets/protocol.py", line 210, in run
msg = yield from self.read_message()
File "/usr/local/lib/python3.4/site-packages/websockets/protocol.py", line 228, in read_message
frame = yield from self.read_data_frame()
File "/usr/local/lib/python3.4/site-packages/websockets/protocol.py", line 266, in read_data_frame
frame = yield from self.read_frame()
File "/usr/local/lib/python3.4/site-packages/websockets/protocol.py", line 295, in read_frame
frame = yield from read_frame(self.reader.readexactly, is_masked)
File "/usr/local/lib/python3.4/site-packages/websockets/framing.py", line 64, in read_frame
data = yield from read_bytes(reader, 2)
File "/usr/local/lib/python3.4/site-packages/websockets/framing.py", line 96, in read_bytes
return (yield from reader(n))
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/streams.py", line 461, in readexactly
block = yield from self.read(n)
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/streams.py", line 432, in read
yield from self._waiter
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/futures.py", line 348, in __iter__
yield self # This tells Task to wait for completion.
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py", line 332, in _wakeup
value = future.result()
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/futures.py", line 243, in result
raise self._exception
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/selector_events.py", line 487, in write
n = self._sock.send(data)
BrokenPipeError: [Errno 32] Broken pipe
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/tasks.py", line 281, in _step
result = coro.send(value)
File "/usr/local/lib/python3.4/site-packages/websockets/server.py", line 63, in handler
self.writer.write_eof()
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/streams.py", line 257, in write_eof
return self._transport.write_eof()
File "/usr/local/Cellar/python3/3.4.0_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/asyncio/selector_events.py", line 531, in write_eof
self._sock.shutdown(socket.SHUT_WR)
AttributeError: 'NoneType' object has no attribute 'shutdown'
I've tried adding "if client.open:" before sending and also tried handling exceptions from my application but neither helps. It seems like this issue needs to be addressed at a lower level, possibly in the websockets module to handle the exceptions properly if a client has disconnected.
See #46 for a use case.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.