fanout/django-grip

Django GRIP library

django-grip's Issues

Security vulnerabilities [High & Medium Severity] in [email protected] introduced by [email protected] dependency

There are two vulnerabilities reported by Snyk in [email protected] which is an indirect dependency in my case because [email protected] requires it.

I've checked the Werkzeug releases, and it is the last release in the 2.*.* before the 3.*.* started. The fix is in the version 3.0.1.

Apparently, the django-grip setup.py file explicitly specifies that the required version of Werkzeug should be < version 3.* ('Werkzeug>=1.0,<3')

Pin [email protected] to [email protected] to fix
  ✗ Inefficient Algorithmic Complexity (new) [Medium Severity][https://security.snyk.io/vuln/SNYK-PYTHON-WERKZEUG-6035177] in [email protected]
    introduced by [email protected] > [email protected] > [email protected]
  ✗ Denial of Service (DoS) (new) [High Severity][https://security.snyk.io/vuln/SNYK-PYTHON-WERKZEUG-6041510] in [email protected]
    introduced by [email protected] > [email protected] > [email protected]

Is django-grip ready to update the Werkzeug dependency to 3.0.1?

Duplicate 'POST /publish' request are sent when using websocket over HTTP

Duplicate 'POST /publish' request are sent when using websocket over HTTP

Environment:

Django(2.0.7)

python manage.py runserver 9000

Pushpin

pushpin --route="* localhost:9000,over_http"

In Django APP's URL.py:

    url(r'^socket/test_userb$', views.test_userb, name='test_userb'),
    url(r'^socket/sendto_b$', views.sendto_b, name='sendto_b'),

In Django APP's views.py:

@websocket_only
def test_userb(request):
    ws = request.wscontext

    # if this is a new connection, accept it and subscribe it to a channel
    if ws.is_opening():
        ws.accept()
        ws.subscribe('user_b')
    ......


def sendto_b(request):
    sendContent = WebSocketMessageFormat('sendto_b\n')
    print(sendContent.content)
    publish('user_b', sendContent)
    return HttpResponse('Ok\n')

In client code:

function sendToB() {
      var xhr = new XMLHttpRequest();
      xhr.open('GET', "http://localhost:7999/users/socket/sendto_b");
      xhr.send();
      .....
}

HTTP Packets:
client to Pushpin:

GET /users/socket/sendto_b HTTP/1.1
Host: localhost:7999
Connection: keep-alive
Origin: null
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Accept: */*
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7

Pushpin to Django:

GET /users/socket/sendto_b HTTP/1.1
Host: localhost:7999
Accept-Encoding: deflate, gzip
Accept: */*
Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7
Origin: null
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Grip-Sig: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1MzIwMTYyNjcsImlzcyI6InB1c2hwaW4ifQ.NK6DkhmE6XY2B1P5Dt-hLNyBytY6Y_6CIPNpM8Gspfc
Grip-Feature: status, session, link:next, filter:skip-self, filter:skip-users, filter:require-sub, filter:build-id

(Two Accept(200 OK) packets omitted)

First publish:
TCP port 62008 -> port 5561

POST /publish/ HTTP/1.1
{"items": [{"ws-message": {"content": "sendto_b\n"}, "channel": "user_b"}]}

Second publish:
TCP port 62010 -> port 5561

POST /publish/ HTTP/1.1
{"items": [{"ws-message": {"content": "sendto_b\n"}, "channel": "user_b"}]}

Log Output from Django server:

Performing system checks...

System check identified no issues (0 silenced).
July 19, 2018 - 15:30:21
Django version 2.0.7, using settings 'mysite.settings'
Starting development server at http://127.0.0.1:9000/
Quit the server with CONTROL-C.
sendto_b

[19/Jul/2018 15:30:38] "GET /users/socket/sendto_b HTTP/1.1" 200 3

And client received two pushes.

[BUG] Breaking changes from urllib3 version 2.0.0 above.

it seems that newer versions of urllib3 will break the library as it uses method_whitelist which is removed in version 2.0.0 and above of urllib3

got an unexpected keyword argument 'method_whitelist'

Workaround was to use lower version 1.27 below of urllib3.

client disconnected unexpectedly python 3, Django

my views.py
`def echo(request):
ws = request.wscontext

if ws.is_opening():
    ws.accept()
    ws.subscribe('room')

while ws.can_recv():
    message = ws.recv()
    if message is None:
        ws.close()
        break

    ws.send(message)`

When I send a string to the channel "абв"
Getting an error:
Internal Server Error: /pushpin/echo/ Traceback (most recent call last): File "/Users/folt/Documents/env/api_server/lib/python3.5/site-packages/django/core/handlers/exception.py", line 42, in inner response = get_response(request) File "/Users/folt/Documents/env/api_server/lib/python3.5/site-packages/django/core/handlers/base.py", line 187, in _get_response response = self.process_exception_by_middleware(e, request) File "/Users/folt/Documents/env/api_server/lib/python3.5/site-packages/django/core/handlers/base.py", line 185, in _get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) File "/Users/folt/Documents/env/api_server/lib/python3.5/site-packages/django_grip.py", line 190, in wrapped_view response = view_func(*args, **kwargs) File "/Users/folt/Documents/Working/api_server/api_server/apps/api_v1/app_pushpin/views.py", line 69, in echo message = ws.recv() File "/Users/folt/Documents/env/api_server/lib/python3.5/site-packages/django_grip.py", line 142, in recv raise IOError('client disconnected unexpectedly') OSError: client disconnected unexpectedly [21/Mar/2017 13:33:32] "POST /pushpin/echo/ HTTP/1.1" 500 69632

How to Deploy with Websockets?

I think I misunderstand something. I've been studying the WebSocket echo service example in the README. What I see is a loop that waits to receive a message and then echoes that back to the client. Normally I deploy using nginx and uwsgi but this seems like it would tie up one of my workers. If the websocket server implementation is busy waiting on the next message, how do other workers service requests?

How does this work with pushpin? If that's a necessary part of the story, it isn't clear from the README.

Maybe I'm misunderstanding the GRIP protocol but I thought it would breakdown the busy-waiting server implementation into a series of http request/replies.

Setting up wscontext.meta OR wscontext.orig_meta causes an error.

Seems to be 2 errors on lines

• django-grip/django_grip.py

  Line 321 in 58298ba

  for nk, nv in wscontext.meta:

  
  

  django-grip/django_grip.py

  Line 333 in 58298ba

  for ok, ov in wscontext.orig_meta:

I suppose that wscontext.meta and wscontext.orig_meta are dictionaries.
Iterating over it should be done using six.iteritems(wscontext.meta) or wscontext.meta.items()

Otherwise now I'm getting the error Error: too many values to unpack

unable to open websocket in django

Hi,

I am trying to open a websocket using pushpin in django. I am getting the following error.

route file is
* 127.0.0.1:8000,over_http

I have started the pushpin proxy server, installed django grip, configured the settings file.

i have copied the example websocket view code in readme which is:

@websocket_only
def echo(request):
print(request)
# since we used the decorator, this will always be a non-None value
ws = request.wscontext

# if this is a new connection, accept it and subscribe it to a channel
if ws.is_opening():
    ws.accept()
    ws.subscribe('test')

# here we loop over any messages
while ws.can_recv():
    message = ws.recv()

    # if return value is None, then the connection is closed
    if message is None:
        ws.close()
        break

    # echo the message
    ws.send(message)

my url config to the view is

url(r'^socket/$', echo, name="socket"),

This urlconf is included in the root urlconf for the regex starting with 'users/'

In the client side, i use an example websocket connection opening code

<script type="text/javascript">
    function WebSocketTest() {
        if ("WebSocket" in window) {
            alert("WebSocket is supported by your Browser!");

            // Let us open a web socket
            var ws = new WebSocket("ws://127.0.0.1:7999/users/socket");

            ws.onopen = function () {
                // Web Socket is connected, send data using send()
                alert("Message is sent...");
            };

            ws.onmessage = function (evt) {
                var received_msg = evt.data;
                alert("Message is received...");
            };

            ws.onclose = function () {
                // websocket is closed.
                alert("Connection is closed...");
            };
        }

        else {
            // The browser doesn't support WebSocket
            alert("WebSocket NOT supported by your Browser!");
        }
    }
</script>

<div id="sse">
    <a href="javascript:WebSocketTest()">Run WebSocket</a>
</div>

clicking on the run websocket on the browser shows the first alert that browser supports websocket and then it shows the alert that the websocket connection is closed.

The local django dev server shows the following in the terminal:

[26/Dec/2015 16:56:17] "POST /users/socket HTTP/1.1" 500 81431

Could you help me figure out where I am going wrong ?

Thanks and regards,
Andy

Werkzeug incompatibility

I'm trying to use django-grip (as a dependency of django-eventstream) in the same project as tensorflow (which indirectly depends on Werkzeug>=1.0.1. I've taken a look at the code and the newer versions of Werkzeug should be compatible up to Werkzeug 2.10, at which point the "The multiple parameter of parse_options_header is deprecated.".

https://werkzeug.palletsprojects.com/en/2.1.x/changes/#version-2-1-0

Would it be possible to change setup.py to specify Werkzeug<2.1 instead of the current Werkzeug<1? I can submit a PR if you prefer.

Thanks!

Document how to enable keep-alive for websockets

It is unclear whether set_hold_stream is applicable to WebSockets.
I am trying to enable keep-alives for a WebSocket by doing this:

django_grip.set_hold_stream(request, [], keep_alive_timeout=30, keep_alive_data="somestring")

In Pushpin log (running at --log-level 10), I see no relevant mentions of keep-alives (there are some apparently unrelated keep-alives with ttl of 60).

I use wscat CLI client for testing, and no keep-alive messages (I would expect "somestring") arrive there, and also the WebSocket itself gets terminated in 50s (which is the timeout configured in HAProxy, behind which Pushpin runs).

I plan to run packet capture of traffic between django app and pushpin tomorrow.

openning websocket fails in Django(2.0.7)

openning websocket fails in Django(2.0.7)

Settings:

• Django + django-grip

• Start Django backend server

python manage.py runserver 9000

• Start pushpin

pushpin --route="* localhost:9000"

1. Success case using curl:

curl -i -H 'Content-Type: application/websocket-events' -d OPEN$'\r'$'\n' http://127.0.0.1:7999/users/socket/

HTTP/1.1 200 OK
Date: Tue, 17 Jul 2018 20:31:20 GMT
Server: WSGIServer/0.2 CPython/3.6.3
Content-Type: application/websocket-events
Sec-WebSocket-Extensions: grip
X-Frame-Options: SAMEORIGIN
Content-Length: 59

OPEN
TEXT 2a
c:{"channel": "test", "type": "subscribe"}

2. Fail case using javascript source code to open a websocket:

<script type = "text/javascript">
    function WebSocketTest() {
        console.log("before open");
        var ws = new WebSocket("ws://127.0.0.1:7999/users/socket/");
        /////// Code fails right here in the above line
    
    }
</script>

By capturing HTTP packets, I can see client requests:
TCP port 62775 -> TCP port 7999

GET ws://127.0.0.1:7999/users/socket/ HTTP/1.1
Host: 127.0.0.1:7999
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: file://
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,zh-CN;q=0.8,zh;q=0.7
Sec-WebSocket-Key: A7FIs1KWY/tVejJrSGFyDg==
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits

pushpin forward the request packet as:
TCP port 62777 -> 9000

GET ws://127.0.0.1:7999/users/socket/ HTTP/1.1
Host: 127.0.0.1:7999
Connection: Upgrade
Pragma: no-cache
Cache-Control: no-cache
Upgrade: websocket
Origin: file://
Sec-WebSocket-Version: 13
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
Grip-Sig: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1MzE4NjM5OTAsImlzcyI6InB1c2hwaW4ifQ.J-D-qfx93Isn9tQejtik4xWLcnLxDQZVIHlX9yjhh1c
Grip-Feature: status, session, link:next, filter:skip-self, filter:skip-users, filter:require-sub, filter:build-id
Sec-WebSocket-Extensions: grip
Sec-WebSocket-Key: LEbAmwodtjDsXPaUR6Orjw==

the Django server responds with:

HTTP/1.1 400 Bad Request
Date: Tue, 17 Jul 2018 20:27:51 GMT
Server: WSGIServer/0.2 CPython/3.6.3
Content-Type: text/html; charset=utf-8
X-Frame-Options: SAMEORIGIN
Content-Length: 39
Request must contain WebSocket events.

Documentation and sample usage is confusing

Sample usage is confusing

I'm trying to use pushpin in my app. I use websocket(instead of HTTP streaming) as transports.

But the README and the documentation is quite confusing(allow me to use such harsh words).

I have several questions:

1. In your README, what is the two code snippets in 'Example view:' supposed to run against? Are they supposed to run in Django? If so, what directory should they be placed into?
   I managed to figure out the set up.

2. I noticed that your documentation(https://pushpin.org/docs/usage/#transports) mentioned that "WebSocket-over-HTTP" should be preferred.
   My question is: If websocket is mocked by HTTP, then why bother using websocket at all(I mean, why not using pure HTTP instead)? I do not get the rational of it.

3. Do you have a front end plus backend demo for websocket + pushpin?

4. I noticed that in you code, only HTTP request method 'POST' is processed(e.g. https://github.com/fanout/headline/blob/master/headlineapp/views.py). From my understanding, it seems that you discourage HTTP request method 'GET'. If so, how should the backend respond to such a client request below?

var chatSocket = new WebSocket(
      'ws://' + window.location.host +
      '/ws/chat/' + roomName + '/');

From my understanding, the above code sends an HTTP request 'GET'(see below), but it seems that 'GET' is discouraged, which is quite confusing to me.(I combed through your documentation but it is not addressed in your documentation. And honestly, your documentation is too brief to be friendly to newcomers of your product(sorry to be so blunt)).

GET /ws/chat/aaaa/ HTTP/1.1\r\n
        [Expert Info (Chat/Sequence): GET /ws/chat/aaaa/ HTTP/1.1\r\n]
            [GET /ws/chat/aaaa/ HTTP/1.1\r\n]
            [Severity level: Chat]
            [Group: Sequence]
        Request Method: GET
        Request URI: /ws/chat/aaaa/
        Request Version: HTTP/1.1
    Host: localhost:8000\r\n
    Connection: Upgrade\r\n
    Pragma: no-cache\r\n
    Cache-Control: no-cache\r\n
    Upgrade: websocket\r\n
    Origin: http://localhost:8000\r\n
    Sec-WebSocket-Version: 13\r\n