tailhook/libwebsite

An HTTP and websocket protocol implementation for libev event loop (UNMAINTAINED, use rust and tk-http crate)

libwebsite's Issues

Build Instructions

As in zerogw, the waf build instructions would be helpful.

Websockets don't support non-masked messages

I'm not sure why masked messages exist (doesn't seem to provide any real security, if you care that much use SSL), but regardless libwebsite doesn't work with unmasked messages.

The solution is pretty easy; you can pretend like the mask is 0, or simply skip the unmask step. I implemented a combination of the two which worked well enough (setting mask = NULL if !has_mask, but then if you have a partial message set websocket_partial_mask = 0).

HTTP Streaming

HTTP streaming would be dope. If you don't have a lot of time, I'm fine with looking into it and seeing if I can implement this (forked the repo).

One of the tests sometimes fails

======================================================================
FAIL: testBadClose (__main__.WebSocket)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/yaourt-tmp-vitaly/aur-libwebsite/src/libwebsite-0.2.22/test/httptest.py", line 310, in testBadClose
    self.assertTrue(val < 7*100000)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 21 tests in 7.634s

Content-Length problems

In HTTP/1.1 it says:

The field value MAY be preceded by any amount of LWS, though a single SP is preferred.

libwebsite always puts more spaces in front of the value like:

Content-Length:    9550

Which causes problems to clients that don't adhere to HTTP/1.1 specification and only accept exactly one SP. One such client is Tsung testing framework. It can only handle one SP.

This is really Tsungs issue.

Build Fails on OS X, Kubuntu Linux

Fails on OS X as follows:

$ python3 ./waf build
Waf: Entering directory `/Users/awainzin/libwebsite/build'
[1/3] c: src/core.c -> build/src/core.c.0.o
[2/3] c: src/search.c -> build/src/search.c.0.o
../src/search.c:1:20: error: malloc.h: No such file or directory
In file included from ../src/search.c:7:
../src/search.h:4:21: error: obstack.h: No such file or directory
In file included from ../src/search.h:6,
                 from ../src/search.c:7:
/Users/awainzin/libwebsite/include/website.h:122: error: field ‘pieces’ has incomplete type
In file included from ../src/search.c:7:
../src/search.h:15: error: field ‘pieces’ has incomplete type
../src/search.h:37: error: field ‘pieces’ has incomplete type
../src/search.c: In function ‘ws_match_new’:
../src/search.c:16: warning: implicit declaration of function ‘malloc’
../src/search.c:16: warning: incompatible implicit declaration of built-in function ‘malloc’
../src/search.c:20: warning: implicit declaration of function ‘obstack_init’
../src/search.c: In function ‘ws_match_free’:
../src/search.c:29: warning: implicit declaration of function ‘obstack_free’
../src/search.c:30: warning: implicit declaration of function ‘free’
../src/search.c: In function ‘ws_match_add’:
../src/search.c:43: warning: implicit declaration of function ‘obstack_alloc’
../src/search.c:44: warning: cast to pointer from integer of different size
../src/search.c: In function ‘ws_match_iadd’:
../src/search.c:69: warning: cast to pointer from integer of different size
../src/search.c: In function ‘ws_match_compile’:
../src/search.c:118: warning: implicit declaration of function ‘obstack_blank’
../src/search.c:121: warning: implicit declaration of function ‘obstack_base’
../src/search.c:121: warning: initialization makes pointer from integer without a cast
../src/search.c:140: warning: implicit declaration of function ‘obstack_ptr_grow’
../src/search.c:142: warning: implicit declaration of function ‘obstack_finish’
../src/search.c:142: warning: assignment makes pointer from integer without a cast
../src/search.c: In function ‘ws_fuzzy_new’:
../src/search.c:172: warning: incompatible implicit declaration of built-in function ‘malloc’
../src/search.c: In function ‘ws_fuzzy_add’:
../src/search.c:217: warning: cast to pointer from integer of different size
../src/search.c: In function ‘ws_rfuzzy_compile’:
../src/search.c:248: warning: initialization makes pointer from integer without a cast
../src/search.c:274: warning: assignment makes pointer from integer without a cast
../src/search.c:275: warning: assignment makes pointer from integer without a cast
../src/search.c: In function ‘ws_fuzzy_compile’:
../src/search.c:293: warning: initialization makes pointer from integer without a cast
../src/search.c:319: warning: assignment makes pointer from integer without a cast
../src/search.c:320: warning: assignment makes pointer from integer without a cast
../src/core.c:16:20: error: endian.h: No such file or directory
In file included from ../src/core.h:5,
                 from ../src/core.c:20:
/Users/awainzin/libwebsite/include/website.h:5:21: error: obstack.h: No such file or directory
In file included from ../src/core.h:5,
                 from ../src/core.c:20:
/Users/awainzin/libwebsite/include/website.h:122: error: field ‘pieces’ has incomplete type
../src/core.c: In function ‘ws_request_init’:
../src/core.c:67: warning: implicit declaration of function ‘obstack_init’
../src/core.c: In function ‘ws_request_free’:
../src/core.c:100: warning: implicit declaration of function ‘obstack_free’
../src/core.c: In function ‘ws_message_send’:
../src/core.c:319: error: ‘EXFULL’ undeclared (first use in this function)
../src/core.c:319: error: (Each undeclared identifier is reported only once
../src/core.c:319: error: for each function it appears in.)
../src/core.c: In function ‘read_websocket’:
../src/core.c:505: warning: implicit declaration of function ‘htobe16’
../src/core.c:511: warning: implicit declaration of function ‘htobe64’
../src/core.c: In function ‘ws_enable_websocket’:
../src/core.c:753: warning: ‘SHA1’ is deprecated (declared at /usr/include/openssl/sha.h:124)
../src/core.c: In function ‘ws_head_slice’:
../src/core.c:840: warning: implicit declaration of function ‘obstack_alloc’
../src/core.c:841: warning: assignment makes pointer from integer without a cast
../src/core.c:843: warning: implicit declaration of function ‘obstack_blank’
../src/core.c:862: warning: implicit declaration of function ‘obstack_ptr_grow’
../src/core.c:867: warning: implicit declaration of function ‘obstack_finish’
../src/core.c:867: warning: assignment makes pointer from integer without a cast
../src/core.c:944: warning: assignment makes pointer from integer without a cast
../src/core.c: In function ‘ws_accept_callback’:
../src/core.c:1035: warning: implicit declaration of function ‘accept4’
../src/core.c:1036: error: ‘SOCK_NONBLOCK’ undeclared (first use in this function)
../src/core.c:1036: error: ‘SOCK_CLOEXEC’ undeclared (first use in this function)
../src/core.c:1040: error: ‘ENONET’ undeclared (first use in this function)
../src/core.c:1053: error: ‘TCP_CORK’ undeclared (first use in this function)
../src/core.c: In function ‘ws_add_tcp’:
../src/core.c:1144: warning: passing argument 2 of ‘bind’ from incompatible pointer type
../src/core.c: In function ‘ws_add_unix’:
../src/core.c:1159: warning: passing argument 2 of ‘bind’ from incompatible pointer type
../src/core.c: In function ‘ws_send_reply’:
../src/core.c:1213: warning: implicit declaration of function ‘writev’
../src/core.c: In function ‘ws_statusline_len’:
../src/core.c:1294: warning: implicit declaration of function ‘obstack_grow’
../src/core.c:1300: warning: implicit declaration of function ‘obstack_object_size’
../src/core.c: In function ‘ws_finish_headers’:
../src/core.c:1348: warning: assignment makes pointer from integer without a cast
../src/core.c: In function ‘ws_reply_data’:
../src/core.c:1370: warning: format ‘%12d’ expects type ‘int’, but argument 5 has type ‘size_t’
Waf: Leaving directory `/Users/awainzin/libwebsite/build'
Build failed
 -> task failed (exit status 1): 
    {task 4317562000: c search.c -> search.c.0.o}
['/usr/bin/gcc', '-std=c99', '-I/Users/awainzin/libwebsite/build/src', '-I/Users/awainzin/libwebsite/src', '-I/Users/awainzin/libwebsite/include', '', '../src/search.c', '-c', '-o', '', 'src/search.c.0.o']
 -> task failed (exit status 1): 
    {task 4317561872: c core.c -> core.c.0.o}
['/usr/bin/gcc', '-std=c99', '-I/Users/awainzin/libwebsite/build/src', '-I/Users/awainzin/libwebsite/src', '-I/Users/awainzin/libwebsite/include', '', '../src/core.c', '-c', '-o', '', 'src/core.c.0.o']

And on Kubuntu (Linux) like...

$ python3 ./waf build
Waf: Entering directory `/home/xevix/libwebsite/build'
[1/3] c: src/core.c -> build/src/core.c.0.o
../src/core.c:18:25: fatal error: openssl/sha.h: No such file or directory                                                           
compilation terminated.
Waf: Leaving directory `/home/xevix/libwebsite/build'
Build failed
 -> task failed (exit status 1):                                                                                                     
        {task 148102444: c core.c -> core.c.0.o}                                                                                     
['/usr/lib/ccache/gcc', '-std=c99', '-I/home/xevix/libwebsite/build/src', '-I/home/xevix/libwebsite/src', '-I/home/xevix/libwebsite/include', '', '../src/core.c', '-c', '-o', '', 'src/core.c.0.o']

I do have openssl installed on Kubuntu, and I have malloc on OS X since I have XCode (with gcc etc.).

End of headers not detected if they come in more than one read()

Steps to reproduce: attempt to GET something via telnet.

Patch:

diff --git src/core.c src/core.c
index 7a4271d..bfab215 100644
--- src/core.c
+++ src/core.c
@@ -811,8 +811,8 @@ static int ws_body_slice(ws_request_t *req, int size) {

 static int ws_head_slice(ws_request_t *req, int size) {
     req->bufposition += size;
-    char *e1 = memmem(req->headers_buf, size, "\r\n\r\n", 4);
-    char *e2 = memmem(req->headers_buf, size, "\n\n", 2);
+    char *e1 = memmem(req->headers_buf, req->bufposition, "\r\n\r\n", 4);
+    char *e2 = memmem(req->headers_buf, req->bufposition, "\n\n", 2);
     size_t reallen;
     if(e2 < e1 && e2 || !e1) {
         e1 = e2;

network timeout prematurely closes connection

But probably should work as idle timeout instead

ImportError: No module named waflib.Build

Something seems wrong with "waf". Am I missing something? Running OS X 10.7, Python 3.1.

$ python3 ./wscript configure --prefix=/usr
Traceback (most recent call last):
File "./wscript", line 3, in
from waflib.Build import BuildContext
ImportError: No module named waflib.Build

Leak of repy_body

req->reply_body is not freed for regular http request/responses. Since the consumer of the API doesn't know when the reply completes, this must be handled by libwebsite.

diff --git a/src/core.c b/src/core.c
index a500225..c34723f 100644
--- a/src/core.c
+++ b/src/core.c
@@ -114,6 +114,7 @@ static ws_request_t *ws_request_new(ws_connection_t *conn) {
 
 int ws_request_free(ws_request_t *req) {
     obstack_free(&req->pieces, NULL);
+    free(req->reply_body);
     free(req);
 }