fukamachi/woo

A fast non-blocking HTTP server on top of libev

Woo

Woo is a fast non-blocking HTTP server built on top of libev. Although Woo is written in Common Lisp, it aims to be the fastest web server written in any programming language.

Warning

This software is still BETA quality.

How fast?

See benchmark.md for the detail.

Usage

Start a server

(ql:quickload :woo)

(woo:run
  (lambda (env)
    (declare (ignore env))
    '(200 (:content-type "text/plain") ("Hello, World"))))

Start with Clack

(ql:quickload :clack)

(clack:clackup
  (lambda (env)
    (declare (ignore env))
    '(200 (:content-type "text/plain") ("Hello, World")))
  :server :woo
  :use-default-middlewares nil)

Cluster

(woo:run
  (lambda (env)
    (declare (ignore env))
    '(200 (:content-type "text/plain") ("Hello, World")))
  :worker-num 4)

Signal handling

When the master process gets these signals, it kills worker processes and quits afterwards.

• QUIT: graceful shutdown, waits for all requests are finished.
• INT/TERM: shutdown immediately.

Benchmarks

See benchmark.md.

Installation

Woo has switched the backend from cl-async to libev after the latest Quicklisp dist release. If you're gonna run the benchmarks by your own, please use the latest one.

Requirements

• UNIX (GNU Linux, Mac, *BSD)
• SBCL
• libev

Installing via Quicklisp

(ql:quickload :woo)

See Also

• libev
• Clack
• Wookie

Author

• Eitaro Fukamachi ([email protected])

Copyright

Copyright (c) 2014 Eitaro Fukamachi & contributors

License

Licensed under the MIT License.

woo's Issues

Better worker mechanism

Something like lparallel, but more lightweight.

Can we have some documentation about this?

I m interested on this webserver but i don't have any documentation about it, do yo have any reference?

Does not work with ningle.

[13:43:28] woo - fast-http error: Callback Error: the message-complete callback failed
#NINGLE.APP:<APP {E51DFA9}> fell through ETYPECASE expression.
Wanted one of (FUNCTION SYMBOL).

This ningle: http://8arrow.org/ningle/

Toot and Wookie and Hunchentoot works well.

SO_REUSEPORT support on Linux 3.9 or above

Linux 3.9 has SO_REUSEPORT to run processes sharing the same port.
Taking advantage of it may improve the multiple process performance.

benchmark uses old version of Go

Looks like the benchmark page indicates that it is using Go 1.6. Could you please update the Go version to use the latest one?

Tornado is using only one IOLoop

Please consider using:

    server.bind(__port__, __host__, 0, __backlog__)
    server.start(0) # this will start one process per core 
    tornado.ioloop.IOLoop.current().start()

Memory fault on libev FFI

Running a Woo server to benchmark with wrk tool causes memory fault on libev FFI. It happens to wrk more than once in my trials on OSX. I get similar condition on Ubuntu on Amazon EC2.

REPL:

> (ql:quickload :woo)
> (woo:run
    (lambda (env)
      (declare (ignore env))
      '(200 (:content-type "text/plain") ("Hello, World"))))

Shell:

$ wrk -c 10 -t 1 -d 10 http://localhost:5000
$ wrk -c 10 -t 1 -d 10 http://localhost:5000  # again

Caused memory fault:

Unhandled memory fault at #xFFFFFFFC002043C0.
   [Condition of type SB-SYS:MEMORY-FAULT-ERROR]

Restarts:
 0: [RETRY] Retry SLIME REPL evaluation request.
 1: [*ABORT] Return to SLIME's top level.
 2: [REMOVE-FD-HANDLER] Remove #<SB-IMPL::HANDLER INPUT on descriptor 9: #<CLOSURE (LABELS SWANK/SBCL::RUN :IN SWANK/BACKEND:ADD-FD-HANDLER) {100716E6EB}>>
 3: [ABORT] Exit debugger, returning to top level.

Backtrace:
  0: (SB-SYS:MEMORY-FAULT-ERROR)
  1: ("foreign function: call_into_lisp")
  2: ("foreign function: post_signal_tramp")
  3: ("foreign function: ev_io_stop")
  4: (LEV:EV-IO-STOP #.(SB-SYS:INT-SAP #X00204510) #.(SB-SYS:INT-SAP #X00204420))
  5: (WOO.EV.SOCKET:FREE-WATCHERS #S(WOO.EV.SOCKET:SOCKET :WATCHERS #(#.(SB-SYS:INT-SAP #X002043F0) #.(SB-SYS:INT-SAP #X00204420) #.(SB-SYS:INT-SAP #X00204450)) :LAST-ACTIVITY 1.43118499171181d9 :FD 23 :RE..
  6: (WOO.EV.SOCKET:CLOSE-SOCKET #S(WOO.EV.SOCKET:SOCKET :WATCHERS #(#.(SB-SYS:INT-SAP #X002043F0) #.(SB-SYS:INT-SAP #X00204420) #.(SB-SYS:INT-SAP #X00204450)) :LAST-ACTIVITY 1.43118499171181d9 :FD 23 :REM..
  7: (WOO.EV.TCP::TCP-READ-CB #<unavailable argument> #.(SB-SYS:INT-SAP #X002043F0) #<unavailable argument>)
  8: ((LAMBDA (SB-ALIEN::ARGS-POINTER SB-ALIEN::RESULT-POINTER FUNCTION) :IN "/Users/mtakagi/.cache/common-lisp/sbcl-1.2.9-macosx-x64/Users/mtakagi/Lisp/quicklisp/dists/quicklisp/software/woo-20150302-git/..
  9: ("foreign function: call_into_lisp")
 10: ("foreign function: funcall3")
 11: ("foreign function: #x20100C95")
 12: ("foreign function: #x204510")
 13: ("foreign function: ev_run")
 14: (LEV:EV-RUN #.(SB-SYS:INT-SAP #X00204510) 0)
 15: ((FLET WOO::START-SERVER :IN WOO:RUN))
 16: (WOO:RUN #<FUNCTION (LAMBDA (ENV)) {1009A9CEDB}> :DEBUG T :PORT 5000 :ADDRESS "0.0.0.0" :BACKLOG NIL :FD NIL :WORKER-NUM NIL)
 17: (SB-INT:SIMPLE-EVAL-IN-LEXENV (WOO:RUN (LAMBDA (ENV) (DECLARE #) (QUOTE #))) #<NULL-LEXENV>)
 18: (EVAL (WOO:RUN (LAMBDA (ENV) (DECLARE #) (QUOTE #))))

Environments

OSX:

$ uname -a
Darwin MacBookPro.local 13.3.0 Darwin Kernel Version 13.3.0: Tue Jun  3 21:27:35 PDT 2014; root:xnu-2422.110.17~1/RELEASE_X86_64 x86_64
$ sbcl --version
SBCL 1.2.9
$ brew list libev
/usr/local/Cellar/libev/4.15/include/ (2 files)
/usr/local/Cellar/libev/4.15/lib/libev.4.dylib
/usr/local/Cellar/libev/4.15/lib/ (2 other files)
/usr/local/Cellar/libev/4.15/share/man/man3/ev.3
$ sbcl --noinform --eval '(progn (ql:system-apropos "woo") (ql:system-apropos "lev") (quit))'
...
#<SYSTEM woo / woo-20150302-git / quicklisp 2015-05-05>
...
#<SYSTEM lev / lev-20150505-git / quicklisp 2015-05-05>
...

Amazon EC2:

$ uname -a
Linux ip-172-30-2-79 3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
$ aws ec2 describe-instances
...
                    "ImageId": "ami-d05e75b8",
...
$ sbcl --version
SBCL 1.1.14.debian
$ apt-show-versions -p libev-dev
libev-dev:amd64/trusty 1:4.15-3 uptodate
$ sbcl --noinform --eval '(progn (ql:system-apropos "woo") (ql:system-apropos "lev") (quit))'
...
#<SYSTEM woo / woo-20150302-git / quicklisp 2015-05-05>
...
#<SYSTEM lev / lev-20150505-git / quicklisp 2015-05-05>
...

woo does not read whole query before parsing on big upload

This bug is only on Linux and not OS X.

Use program from this bugreport to reproduce.
fukamachi/fast-http#27

Issue with serving images (png?) in caveman2 with woo

When I load my project (https://github.com/ahungry/sluglisp) via:

 sbcl --eval '(ql:quickload :sluglisp)' --eval '(sluglisp:start :port 60333 :server :woo)

Then visit my page at http://sluglisp.ahungry.com

The second png image (http://sluglisp.ahungry.com/images/made-with-lisp-small.png) fails to load and the browser sits there timing out.

When I cancel the http request, then re-request the page, sbcl reports this error:

debugger invoked on a TYPE-ERROR in thread
#<THREAD "clack-handler-woo" RUNNING {1005BEE223}>:
  The value -60356 is not of type (UNSIGNED-BYTE 64).

It only happens when running with :server :woo, using hunchentoot causes no issue.

The environment is CentOS 6.7 (final), with libev 4.0.0 present on the system.

All dependencies are up to date with Quicklisp latest.

This error happens across the board with any of my recent caveman2 projects when running under woo.

Error in response when loading woo in lispworks

I'm getting the following error when attempting to load woo in lispworks enterprise 7.1.2 (64-bit) on osx catalina 10.15.2. I am having no problem loading woo in sbcl or clozure (ccl) on the same machine.

Here are the errors:

CL-USER 3 > (ql:quickload :woo)
To load "woo":
Load 1 ASDF system:
woo
; Loading "woo"
.
.
[package woo.response]
.
.

Error: Variable DATE-HEADER was declared type (SIMPLE-ARRAY CHARACTER (29)) but is being set to value "Thu, 01 Jan 1970 00:00:00 GMT".

Here is the error when executing that code from the response.lisp file.

CL-USER 5 > (declaim (type (simple-array character (29)) date-header))
T

CL-USER 6 > (defvar date-header "Thu, 01 Jan 1970 00:00:00 GMT")

Error: Variable DATE-HEADER was declared type (SIMPLE-ARRAY CHARACTER (29)) but is being set to value "Thu, 01 Jan 1970 00:00:00 GMT".
1 (abort) Return to top loop level 0.

Type :b for backtrace or :c to proceed.
Type :bug-form "" for a bug report template or :? for other options.

Lispworks wants base-char I think... but of course this doesn't work in sbcl although it does work in ccl... at least for setting the variable...

CL-USER 31 > (declaim (type (simple-array base-char (29)) date-header))
T

CL-USER 32 > (defvar date-header "Thu, 01 Jan 1970 00:00:00 GMT")
DATE-HEADER

The alien function "ev_loop_new" is undefined

I am getting this error when trying to run Woo. Originally, I tried to use it through Clack or Lack, but it would not work.

Hence, I opted for doing just these two things in a fresh REPL:

(ql:quickload :woo)

(woo:run
  (lambda (env)
    (declare (ignore env))
    '(200 (:content-type "text/plain") ("Hello, World"))))

I immediately get the following error:

The alien function "ev_loop_new" is undefined.
[Condition of type SB-KERNEL::UNDEFINED-ALIEN-FUNCTION-ERROR]

Restarts:
0: [RETRY] Retry SLIME REPL evaluation request.
1: [*ABORT] Return to SLIME's top level.
2: [ABORT] abort thread (#<THREAD "repl-thread" RUNNING {10022D8003}>)

Backtrace:
0: ("undefined function")
1: (LEV:EV-LOOP-NEW 41051748)
2: (SB-INT:SIMPLE-EVAL-IN-LEXENV (WOO:RUN (LAMBDA (ENV) (DECLARE #) (QUOTE #))) #)
3: (EVAL (WOO:RUN (LAMBDA (ENV) (DECLARE #) (QUOTE #))))
--more--

Does anyone know what could be the cause?

P.S. Is there a reason why (ql:quickload "woo") does not work? Is there another package with similar name?

Where is the usage document?

HPE_INVALID_CONSTANT error occurred in Node.js client when receive empty string

Server:

(ql:quickload :clack)

(defvar *hander*
  (clack:clackup
   (lambda (env)
     (declare (ignore env))
     '(200 (:content-type "text/plain") ("")))
   :port 3000
   :server :woo))

Client in Node.js with request library:

const request = require('request');

request.get('http://localhost:3000/', (err, res, body) => console.log([err, res, body]));

Run this client then get following result:

[ { Error: Parse Error
    at Socket.socketOnData (_http_client.js:440:20)
    at emitOne (events.js:116:13)
    at Socket.emit (events.js:211:7)
    at addChunk (_stream_readable.js:263:12)
    at readableAddChunk (_stream_readable.js:250:11)
    at Socket.Readable.push (_stream_readable.js:208:10)
    at TCP.onread (net.js:594:20) bytesParsed: 115, code: 'HPE_INVALID_CONSTANT' },
  undefined,
  undefined ]

Node.js version: v10.15.0

I guess woo returns invalid response. 🤔

Metadata for Quicklisp

Please consider adding :description, :author and :license information to your ASDF system. This will greatly help Quicklisp users and make it easier for them to report bugs.

More information:

http://blog.quicklisp.org/2015/05/looking-for-more-metadata.html
https://www.quicklisp.org

HTTP/2 support

Uncatchable exception when trying to serve a nonexistent file.

(woo:run
    (lambda (env)
       `(200 (:content-type "text/html")
           #P"/file/that/does/not-exist.html")))

One would hope that this would result in a 404 error being returned to the Web client, but no. Instead, Woo attempts to open this using POSIX, and it doesn't even bother to check the return value (which is -1, with errno = 2 (ENOENT/"No such file or directory")). Instead, the error doesn't get caught until Woo calls (fd-file-size fd) with fd = -1. That invokes SBCL's binding to fstat, which properly checks for fstat's error code and signals a condition. The error comes up not in the browser, but in the Lisp debugger.

I guess I also wouldn't get a 403 error if I tried to serve a file that Woo didn't have permission to read.

The icing on the cake is that this error happens outside of the user-provided request handler, so it
goes straight to the debugger and there is no way to catch it except by modifying Woo's source
files.

Binary executable created with Roswell cannot accept tcp connection

I created a simple script with Roswell which runs the hello world code similar to README. When I run this script from command line without building, it runs fine and sends the html response to browser. But when I uses ros build to build the binary and run it instead, I get this error on Mac:

Error: The value # is not of the expected type MACPTR.
While executing: WOO.LLSOCKET:ACCEPT, in process toplevel(3).
Type :POP to abort, :R for a list of available restarts.
Type :? for other options.

And on Ubuntu, I get this:

<ERROR> [09:50:45] woo.ev.tcp - Can't accept connection (Code: 22)

When I use (ql:quickload "woo"), there is a problem on "static-vectors"

Error while trying to load definition for system static-vectors
from pathname
d:/Fire_Workspace/emacs_24/home/clozure/quicklisp/dists/quicklisp/software/static-vectors-1.6/static-vectors.asd:

   You need ASDF >= 2.32 to load this system correctly.
   [Condition of type ASDF:LOAD-SYSTEM-DEFINITION-ERROR]

But anyhow, I had updated asdf.lisp by (ql:update-all-dists)

Abandon ware?

This is abandon ware right?

The value 3375638364 is not of type (SIGNED-BYTE 32)

This is an issue with the call to inet_ntoa. The cffi defun declares the type of the argument as an int64. This is incorrect. The actual call to inet_ntoa takes in a struct, not a pointer to a struct. The value inside of that struct is a single uint32. You can, thus, declare the type as a uint32, but an int64 is wrong. Also, one should be using inet_ntop and inet_pton instead of inet_ntoa and inet_aton.

For references to the solution to this issue see:
http://www.retran.com/beej/inet_ntoaman.html
http://man7.org/linux/man-pages/man7/ip.7.html

Add woo to TechEmpower's Web Framework Benchmarks would be great

TechEmpower's Web Framework Benchmarks is a performance comparison of many web application frameworks executing fundamental tasks such as JSON serialization, database access, and server-side template composition. Each framework is operating in a realistic production configuration. Results are captured on Amazon EC2 and on physical hardware. The test implementations are largely community-contributed and all source is available at the GitHub repository.

There is no Common Lisp frameworks in the benchmarks now, so it will be great if woo become the first one :)

Type error, followed by memory-fault-error in SBCL

Steps to reproduce:

 CL-USER> (ql:quickload :woo)
 To load "woo":
   Load 1 ASDF system:
     woo
 ; Loading "woo"

 (:WOO)
 CL-USER> 
 (woo:run
   (lambda (env)
     (declare (ignore env))
     '(200 (:content-type "text/plain") ("Hello, World"))))

In a browser, navigate to localhost:5000.

Expected:

See a plain text

 Hello, World

page in browser

Observed:

  <ERROR> [21:05:14] woo - The value 2779321106 is not of type (UNSIGNED-BYTE 8).

appears in REPL. The browser shows a loading icon and never displays content. If the stop button is clicked in browser, the REPL produces the following error

 Unhandled memory fault at #x7FFF28004E60.
    [Condition of type SB-SYS:MEMORY-FAULT-ERROR]

 Restarts:
  0: [RETRY] Retry SLIME REPL evaluation request.
  1: [*ABORT] Return to SLIME's top level.
  2: [ABORT] Abort thread (#<THREAD "repl-thread" RUNNING {10048D8083}>)

 Backtrace:
   0: (SB-SYS:MEMORY-FAULT-ERROR)
   1: ("foreign function: call_into_lisp")
   2: ("foreign function: post_signal_tramp")
   3: ("foreign function: #x20220F8764")
   4: (WOO.EV.SOCKET:CLOSE-SOCKET #S(WOO.EV.SOCKET:SOCKET :WATCHERS #(#.(SB-SYS:INT-SAP #X7FFFE8005300) #.(SB-SYS:INT-SAP #X7FFFE8005340) #.(SB-SYS:INT-SAP #X7FFFE8005380)) :LAST-ACTIVITY 1.423707477550414d..
   5: (WOO.EV.TCP::TCP-READ-CB #<unavailable argument> #.(SB-SYS:INT-SAP #X7FFFE8005300) #<unavailable argument>)
   6: ((LAMBDA (SB-ALIEN::ARGS-POINTER SB-ALIEN::RESULT-POINTER FUNCTION) :IN "/home/inaimathi/.cache/common-lisp/sbcl-1.1.15.debian-linux-x64/home/inaimathi/quicklisp/dists/quicklisp/software/woo-20150113-..
   7: ("foreign function: call_into_lisp")
   8: ("foreign function: funcall3")
   9: ("foreign function: #x20101085")
  10: ("foreign function: #x2014A118DE")
  11: (WOO:RUN #<FUNCTION (LAMBDA (ENV)) {100EFC697B}> :DEBUG T :PORT 5000 :ADDRESS "0.0.0.0" :BACKLOG NIL :FD NIL :WORKER-NUM NIL)
  12: (SB-INT:SIMPLE-EVAL-IN-LEXENV (WOO:RUN (LAMBDA (ENV) (DECLARE (IGNORE ENV)) (QUOTE (200 (:CONTENT-TYPE "text/plain") ("Hello, World"))))) #<NULL-LEXENV>)
  13: (EVAL (WOO:RUN (LAMBDA (ENV) (DECLARE (IGNORE ENV)) (QUOTE (200 (:CONTENT-TYPE "text/plain") ("Hello, World"))))))
  14: (SWANK::EVAL-REGION " ..)
  15: ((LAMBDA NIL :IN SWANK-REPL::REPL-EVAL))
  16: (SWANK-REPL::TRACK-PACKAGE #<CLOSURE (LAMBDA NIL :IN SWANK-REPL::REPL-EVAL) {100EFA414B}>)
  17: (SWANK::CALL-WITH-RETRY-RESTART "Retry SLIME REPL evaluation request." #<CLOSURE (LAMBDA NIL :IN SWANK-REPL::REPL-EVAL) {100EFA408B}>)
  18: (SWANK::CALL-WITH-BUFFER-SYNTAX NIL #<CLOSURE (LAMBDA NIL :IN SWANK-REPL::REPL-EVAL) {100EFA406B}>)
  19: (SWANK-REPL::REPL-EVAL " ..)
  20: (SB-INT:SIMPLE-EVAL-IN-LEXENV (SWANK-REPL:LISTENER-EVAL " ..)
  21: (EVAL (SWANK-REPL:LISTENER-EVAL " ..)
  22: (SWANK:EVAL-FOR-EMACS (SWANK-REPL:LISTENER-EVAL " ..)
  23: (SWANK::PROCESS-REQUESTS NIL)
  24: ((LAMBDA NIL :IN SWANK::HANDLE-REQUESTS))
  25: ((LAMBDA NIL :IN SWANK::HANDLE-REQUESTS))
  26: (SWANK/SBCL::CALL-WITH-BREAK-HOOK #<FUNCTION SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA NIL :IN SWANK::HANDLE-REQUESTS) {10048E00EB}>)
  27: ((FLET SWANK/BACKEND:CALL-WITH-DEBUGGER-HOOK :IN "/home/inaimathi/quicklisp/dists/quicklisp/software/slime-2.12/swank/sbcl.lisp") #<FUNCTION SWANK:SWANK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA NIL :IN SWANK:..
  28: (SWANK::CALL-WITH-BINDINGS ((*STANDARD-OUTPUT* . #1=#<SWANK/GRAY::SLIME-OUTPUT-STREAM {10048B9653}>) (*STANDARD-INPUT* . #2=#<SWANK/GRAY::SLIME-INPUT-STREAM {1002B00183}>) (*TRACE-OUTPUT* . #1#) (*ERR..
  29: (SWANK::HANDLE-REQUESTS #<SWANK::MULTITHREADED-CONNECTION {1003A42AF3}> NIL)
  30: ((FLET #:WITHOUT-INTERRUPTS-BODY-1117 :IN SB-THREAD::INITIAL-THREAD-FUNCTION-TRAMPOLINE))
  31: ((FLET SB-THREAD::WITH-MUTEX-THUNK :IN SB-THREAD::INITIAL-THREAD-FUNCTION-TRAMPOLINE))
  32: ((FLET #:WITHOUT-INTERRUPTS-BODY-537 :IN SB-THREAD::CALL-WITH-MUTEX))
  33: (SB-THREAD::CALL-WITH-MUTEX #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK :IN SB-THREAD::INITIAL-THREAD-FUNCTION-TRAMPOLINE) {7FFFF488EC6B}> #<SB-THREAD:MUTEX "thread result lock" owner: #<SB-THREAD:THR..
  34: (SB-THREAD::INITIAL-THREAD-FUNCTION-TRAMPOLINE #<SB-THREAD:THREAD "repl-thread" RUNNING {10048D8083}> #S(SB-THREAD:SEMAPHORE :NAME "Thread setup semaphore" :%COUNT 0 :WAITCOUNT 0 :MUTEX #<SB-THREAD:MU..
  35: ("foreign function: call_into_lisp")
  36: ("foreign function: new_thread_trampoline")

Build Info:

This is woo-20150113-git installed using quicklisp, running on 64-bit Debian. Observed in SBCL 1.1.15.debian and SBCL 1.2.8.nixos. The Debian version is using libev directly out of the Jesse repos (so looks like 1:4.15-3), while the nix version is running libev-4.15 and libevdev-1.3.1.

Installation in Aquamacs

Cheers! I am new to CL and this server. When trying to use, I get the following error in Aquamacs after running (ql:quickload :woo). Any idea how I could proceed?

`CL-USER> (ql:quickload :woo)
To load "cffi-grovel":
Load 1 ASDF system:
cffi-grovel
; Loading "cffi-grovel"
; ((UIOP/OS:OS-UNIX-P) "o")
;
; caught ERROR:
; illegal function call

; ((UIOP/OS:OS-WINDOWS-P)
; (IF (UIOP/OS:FEATUREP '(:OR :MINGW32 :MINGW64))
; "o"
; "obj"))
;
; caught ERROR:
; illegal function call

; ((UIOP/OS:OS-UNIX-P) "a")
;
; caught ERROR:
; illegal function call

; ((UIOP/OS:OS-WINDOWS-P)
; (IF (UIOP/OS:FEATUREP '(:OR :MINGW32 :MINGW64))
; "a"
; "lib"))
;
; caught ERROR:
; illegal function call

; ((UIOP/OS:OS-MACOSX-P) "dylib")
;
; caught ERROR:
; illegal function call

; ((UIOP/OS:OS-UNIX-P) "so")
;
; caught ERROR:
; illegal function call

; ((UIOP/OS:OS-WINDOWS-P) "dll")
;
; caught ERROR:
; illegal function call

; ((UIOP/OS:OS-UNIX-P) NIL)
;
; caught ERROR:
; illegal function call

; ((UIOP/OS:OS-WINDOWS-P) "exe")
;
; caught ERROR:
; illegal function call`

Use sendfile(2) for sending a static file faster

Using sendfile(2) would be better for sending a static file because it doesn't load file contents to user space.

The value 3369575493 is not of type (SIGNED-BYTE 32) (in clack-handler-woo thread)

I receive the following error:

debugger invoked on a TYPE-ERROR in thread
#<THREAD "clack-handler-woo" RUNNING {10059706B3}>:
  The value 3369575493 is not of type (SIGNED-BYTE 32).

When attempting to load a freshly created caveman2 project on my Centos6 server (well, it quickloads, and then displays this error on the first http request).

You can see the project here (let me know if the generated asd file should be modified maybe?)

https://github.com/ahungry/slugelisp

It doesn't happen on my other caveman2 web instance (created back in January) that runs on the default port 5000 (only on the port 60333 - no idea if the port is related in any way).

Hunchentoot and Wookie both seem to work fine (although Hunchentoot has its own issues with running out of threads and displaying a 503 server error after being up for around 24 hours).

Can't catch some errors thrown from worker threads.

One common error that happens is when a client opens a connection but
never sends a request. Eventually, that produces a timeout error, which bubbles
up to the debugger. But that's hard to reproduce. Here's an easier one, generated
by simply sending invalid output:

(woo:run (lambda (env)
                 `(200 (:content-type "application/octet-stream")
                      (#(1 2 3 4 5))))
    :num-workers 2)

The FAST-HTTP.ERROR:CB-MESSAGE-COMPLETE this throws isn't catchable anywhere because the error happens in a worker thread. It has nowhere to go but the debugger. This is pretty convenient for development, but a deal-killer for production code.

The woo:run function could accept an error-handling function that the worker thread would install
with HANDLER-BIND, like this:

(defun make-worker (process-fn when-died error-handler)
  (let* ((dequeue-async (cffi:foreign-alloc '(:struct lev:ev-async)))
         (stop-async (cffi:foreign-alloc '(:struct lev:ev-async)))
         (worker (%make-worker :dequeue-async dequeue-async
                               :stop-async stop-async
                               :process-fn process-fn))
         (worker-lock (bt:make-lock)))
    (lev:ev-async-init dequeue-async 'worker-dequeue)
    (lev:ev-async-init stop-async 'worker-stop)
    (setf (worker-thread worker)
          (bt:make-thread
           (lambda ()
             (tagbody
                begin
                (restart-case
                    (handler-bind ((t error-handler))
                      (bt:acquire-lock worker-lock)
                      (let ((*worker* worker))
                        (wev:with-sockaddr
                          (unwind-protect			       
                               (wev:with-event-loop ()
                                 (setf (worker-evloop worker) *evloop*)
                                 (bt:release-lock worker-lock)
                                 (lev:ev-async-start *evloop* dequeue-async)
                                 (lev:ev-async-start *evloop* stop-async))
                            (unless (eq (worker-status worker) :stopping)
                              (vom:debug "[~D] Worker has died" (worker-id worker))
                              (funcall when-died worker))
                            (finalize-worker worker)
                            (vom:debug "[~D] Bye." (worker-id worker))))))
                  (abort-worker-thread () :report "Abort the Woo worker")
                  (restart-worker () :report "Restart the worker"
                                  (go begin)))))
           :initial-bindings (default-thread-bindings)
           :name "woo-worker"))
    (sleep 0.1)
    (bt:acquire-lock worker-lock)
    worker))

Then, the app's error handler could invoke the abort-worker-thread restart to
abort the worker thread, or restart-worker to start the worker up again.

Submit benchmarking code to Techempower benchmarks.

The Techempower web framework benchmarks[1] are quite popular. Would love to some common lisp especially woo on there.

[1] https://www.techempower.com/benchmarks/#section=data-r10

Outputting stream of data in a handler doesn't work on woo

I wrote a simple demo app with ningle and clack running on woo server serving PNG images (gist: here).
When I output the stream of created image directly in the handler, the server fails on woo with the following error:

debugger invoked on a FAST-HTTP.ERROR:CB-MESSAGE-COMPLETE in thread

<THREAD "clack-handler-woo" RUNNING {1006C9B2B3}>:

Callback Error: the message-complete callback failed
The value #(137 80 78 71 13 10 26 10 0 0 0 13 ...)
is not of type
(SIMPLE-ARRAY (UNSIGNED-BYTE 8) (*)).

The code works fine on hunchentoot. The only way i figured this working is to coerce a stream into a simple-array:
(coerce output '(simple-array (unsigned-byte 8) (*))

That just looks weird to me - woo should be able to handle outputting to stream so it seems to me I'm doing something wrong.

format-timestamp is bottleneck on sbcl

mainly every single ~d format directive causes a make-string-output-stream call; I mentioned this on IRC to an sbcl dev. In the meantime, jasom/woo@b3f7b5e increases the benchmark rate by about 25% on my machine.

I didn't format this as a pull request because the only testing I did of the format-rfc-1123-timestamp was to call it on (now) and see that it looked sane.

Writing direclty to the output stream

Hi,

I'd like to benchmark woo in my use case, but I need to be able to write directly to the output stream (as the json data that I return can be large).

Is there a way to access this output stream (as in hunchentoot) ?

Thanks,

guillaume

Why not show the multiple-threads hunchentoot benchmark ? it's really fast.

On my MBP( i7cpu 8core), it's over 22k requests/s

$ wrk -c 100 -t 4 -d 10 http://127.0.0.1:5000                                                              
Running 10s test @ http://127.0.0.1:5000
  4 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    13.22ms   84.65ms   1.72s    97.82%
    Req/Sec     5.91k     1.90k   19.82k    77.95%
  226515 requests in 10.03s, 33.92MB read
  Socket errors: connect 0, read 135, write 0, timeout 20
Requests/sec:  22573.23
Transfer/sec:      3.38MB

#10 threads
$ wrk -c 100 -t 10 -d 10 http://127.0.0.1:5000                                                             
Running 10s test @ http://127.0.0.1:5000
  10 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    16.16ms   98.85ms   1.95s    97.60%
    Req/Sec     2.48k     1.17k   10.52k    73.37%
  229549 requests in 10.05s, 34.37MB read
  Socket errors: connect 0, read 151, write 0, timeout 15
Requests/sec:  22851.01
Transfer/sec:      3.42MB

This will make others surprised!

SSL support

crash on hacking attempt via URL

A hacker was probing for vulnerability and with this url brought the system to a stop. Is there a way to not go into debugger on strange url's? I've been crashed several times this way.

185.100.87.246 - [16/Jan/2019:12:01:39 -09:00] "GET /nice%20ports%2C/Tri%6Eity.txt%2ebak HTTP/1.0" 302 38 "-" "-"

debugger invoked on a FAST-HTTP.ERROR:CB-MESSAGE-COMPLETE in thread
#<THREAD "clack-handler-woo" RUNNING {100980E2C3}>:
Callback Error: the message-complete callback failed
(A SIMPLE-ERROR was caught when trying to print DEBUG-CONDITION when entering
the debugger. Printing was aborted and the SIMPLE-ERROR was stored in
SB-DEBUG::NESTED-DEBUG-CONDITION.)

Cannot send JSON

I do not have a lot of experience with Common Lisp, but I would like to learn it a bit better. I am having trouble with :content-type "application/json". This is running with Docker, using Roswell. Here is the stack trace:

woo: Unhandled FAST-HTTP.ERROR:CB-MESSAGE-COMPLETE in thread #<SB-THREAD:THREAD
woo:                                                           "main thread" RUNNING
woo:                                                            {1002D7FEA3}>:
woo:   Callback Error: the message-complete callback failed
woo:   The value TO-JSON is not of type STRING.
woo: Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {1002D7FEA3}>
woo: 0: (TRIVIAL-UTF-8:UTF-8-BYTE-LENGTH TO-JSON) [tl,external]
woo: 1: (TRIVIAL-UTF-8:STRING-TO-UTF-8-BYTES #<unavailable argument> :NULL-TERMINATE #<unavailable argument>)
woo: 2: (WOO::HANDLE-NORMAL-RESPONSE #<unavailable argument> #<unavailable argument> #<unavailable argument>)
woo: 3: (WOO::HANDLE-RESPONSE #S(FAST-HTTP.HTTP:HTTP-REQUEST :METHOD :GET :MAJOR-VERSION 1 :MINOR-VERSION 1 :STATUS 0 :CONTENT-LENGTH NIL :CHUNKED-P NIL :UPGRADE-P NIL :HEADERS #<HASH-TABLE :TEST EQUAL :COUNT 3 {1003068DD3}> :HEADER-READ 0 :MARK 83 :STATE 3 :RESOURCE "/json") #S(WOO.EV.SOCKET:SOCKET :WATCHERS #(#.(SB-SYS:INT-SAP #X006502E0) #.(SB-SYS:INT-SAP #X00650490) #.(SB-SYS:INT-SAP #X006504D0)) :LAST-ACTIVITY 1.5445727350415163d9 :FD 9 :REMOTE-ADDR "172.18.0.4" :REMOTE-PORT 34485 :DATA #<CLOSURE (LAMBDA (FAST-HTTP::DATA &KEY (FAST-HTTP::START 0) FAST-HTTP::END) :IN FAST-HTTP:MAKE-PARSER) {100306899B}> :TCP-READ-CB WOO.EV.TCP::TCP-READ-CB :READ-CB #<FUNCTION WOO::READ-CB> :WRITE-CB NIL :OPEN-P T :BUFFER #S(FAST-IO::OUTPUT-BUFFER :VECTOR #(58 32 87 111 111 13 10 84 114 97 110 115 ...) :FILL 37 :LEN 153 :QUEUE #1=(#(72 84 84 80 47 49 46 49 32 50 48 48 ...) #(10 68 97 116 101 58 32 84 117 101 44 32 ...) #(71 77 84 13 10 67 111 110 110 101 99 116 ...)) :LAST #1# :OUTPUT NIL)) (200 (:CONTENT-TYPE "application/json" :SERVER "Woo") (TO-JSON (QUOTE (:MESSAGE "Hello, World!")))))
woo: 4: ((LAMBDA (FAST-HTTP.HTTP:HTTP) :IN FAST-HTTP:MAKE-PARSER) #<unavailable argument>)
woo: 5: (FAST-HTTP.PARSER::PARSE-BODY #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument> #<unavailable argument>)
woo: 6: (FAST-HTTP.PARSER:PARSE-REQUEST #<unavailable argument> #<unavailable argument> #<unavailable argument> :START #<unavailable argument> :END #<unavailable argument>)
woo: 7: ((LAMBDA (FAST-HTTP::DATA &KEY (FAST-HTTP::START 0) FAST-HTTP::END) :IN FAST-HTTP:MAKE-PARSER) #<unavailable argument> :START #<unavailable argument> :END #<unavailable argument>)
woo: 8: (WOO::READ-CB #S(WOO.EV.SOCKET:SOCKET :WATCHERS #(#.(SB-SYS:INT-SAP #X006502E0) #.(SB-SYS:INT-SAP #X00650490) #.(SB-SYS:INT-SAP #X006504D0)) :LAST-ACTIVITY 1.5445727350415163d9 :FD 9 :REMOTE-ADDR "172.18.0.4" :REMOTE-PORT 34485 :DATA #<CLOSURE(LAMBDA (FAST-HTTP::DATA &KEY (FAST-HTTP::START 0) FAST-HTTP::END) :IN FAST-HTTP:MAKE-PARSER) {100306899B}> :TCP-READ-CB WOO.EV.TCP::TCP-READ-CB :READ-CB #<FUNCTION WOO::READ-CB> :WRITE-CB NIL :OPEN-P T :BUFFER #S(FAST-IO::OUTPUT-BUFFER :VECTOR #(58 32 87 111 111 13 10 84 114 97 110 115 ...) :FILL 37 :LEN 153 :QUEUE #1=(#(72 84 84 80 47 49 46 49 32 50 48 48 ...) #(10 68 97 116 101 58 32 84 117 101 44 32 ...) #(71 77 84 13 10 67 111 110 110 101 99 116 ...)) :LAST #1# :OUTPUT NIL)) #(71 69 84 32 47 106 115 111 110 32 72 84 ...) :START 0 :END 83)
woo: 9: (WOO.EV.TCP::TCP-READ-CB #<unavailable argument> #.(SB-SYS:INT-SAP #X006502E0) #<unavailable argument>)
woo: 10: ((LAMBDA (SB-ALIEN::ARGS-POINTER SB-ALIEN::RESULT-POINTER FUNCTION) :IN "/root/.cache/common-lisp/sbcl-1.2.12-linux-x64/1.2.12/root/.roswell/impls/ALL/ALL/quicklisp/dists/quicklisp/software/woo-20150608-git/src/ev/socket.fasl") 7036863545102470368635451020 #<FUNCTION (LAMBDA (WOO.EV.TCP::EVLOOP WOO.EV.TCP::WATCHER WOO.EV.TCP::EVENTS) :IN "/root/.roswell/impls/ALL/ALL/quicklisp/dists/quicklisp/software/woo-20150608-git/src/ev/tcp.lisp") {10067ABD4B}>)
woo: 11: ("foreign function: call_into_lisp")
woo: 12: ("foreign function: funcall3")
woo: 13: ("foreign function: callback_wrapper_trampoline")
woo: 14: ("foreign function: #x20100D1C")
woo: 15: ((FLET WOO::START-SERVER-MULTI :IN WOO:RUN))
woo: 16: (WOO:RUN #<FUNCTION (LAMBDA (ENV) :IN MAIN) {100306525B}> :DEBUG NIL :PORT 8080 :ADDRESS "0.0.0.0" :BACKLOG NIL :FD NIL :WORKER-NUM 4)
woo: 17: (SB-INT:SIMPLE-EVAL-IN-LEXENV (APPLY (QUOTE MAIN) ROS:*ARGV*) #<NULL-LEXENV>)
woo: 18: (EVAL-TLF (APPLY (QUOTE MAIN) ROS:*ARGV*) NIL #<NULL-LEXENV>)
woo: 19: ((LABELS SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (APPLY (QUOTE MAIN) ROS:*ARGV*) NIL)
woo: 20: (SB-INT:LOAD-AS-SOURCE #<CONCATENATED-STREAM :STREAMS NIL {1005EB81B3}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
woo: 21: ((FLET SB-FASL::LOAD-STREAM :IN LOAD) #<CONCATENATED-STREAM :STREAMS NIL {1005EB81B3}> NIL)
woo: 22: (LOAD #<CONCATENATED-STREAM :STREAMS NIL {1005EB81B3}> :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT:DEFAULT)
woo: 23: (ROS:SCRIPT :SCRIPT "./woo.ros" "--worker" "4" "--port" "8080")
woo: 24: (ROS:RUN ((:SCRIPT "./woo.ros" "--worker" "4" "--port" "8080") (:QUIT NIL)))
woo: 25: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ROS:RUN (QUOTE ((:SCRIPT "./woo.ros" "--worker" "4" "--port" "8080") (:QUIT NIL)))) #<NULL-LEXENV>)
woo: 26: (EVAL (ROS:RUN (QUOTE ((:SCRIPT "./woo.ros" "--worker" "4" "--port" "8080") (:QUIT NIL)))))
woo: 27: (SB-IMPL::PROCESS-EVAL/LOAD-OPTIONS ((:EVAL . "(progn #-ros.init(cl:load \"/usr/local/share/common-lisp/source/roswell/init.lisp\"))") (:EVAL . "(ros:quicklisp)") (:EVAL . "(ros:run '((:script \"./woo.ros\"\"--worker\"\"4\"\"--port\"\"8080\")(:quit ())))")))
woo: 28: (SB-IMPL::TOPLEVEL-INIT)
woo: 29: ((FLET #:WITHOUT-INTERRUPTS-BODY-83 :IN SAVE-LISP-AND-DIE))
woo: 30: ((LABELS SB-IMPL::RESTART-LISP :IN SAVE-LISP-AND-DIE))
woo: unhandled condition in --disable-debugger mode, quitting

Source

Could you add a license?

I was just hoping you could add a license to the project.

Thanks for your work on Woo, it really is an amazing effort.

uiop:run-program with string output

Hi,

I was looking at how different projects on quicklisp use uiop:run-program and came across this snippet in woo:

          (with-output-to-string (s)
            (uiop:run-program "uname -r"
                              :output s
                              :ignore-error-status t))))

I wanted to let you know that passing a stream as uiop:run-program's :output argument, especially one that isn't a file stream, is not very portable. But you can also get string output directly from uiop:run-program (that's far more portable because it internally uses a file) as follows:

(uiop:run-program '("uname" "-r") :output '(:string :stripped t))

Problem with cffi-grovel if woo is installed by qlot from the github

Here is a repository with a minimal example to reproduce this problem: https://github.com/svetlyak40wt/test-woo-cffi-grovel-error

I remember, that some time ago, I successful installed a woo using Qlot, but now it does not work.

Qlot tries to load all asd and fails with "Component "cffi-grovel" not found" error.

Here is a full traceback:

Installing dist "woo".
Calculating project dependencies...
While evaluating the form starting at line 1, column 0
  of #P"/Users/art/projects/lisp/test-woo/quicklisp/dists/woo/software/woo-52bf0ede5c2798cb170e0c298e834ee67d79c3d8/woo.asd":
Unhandled MISSING-COMPONENT in thread #<SB-THREAD:THREAD "main thread" RUNNING {10005205B3}>: Component "cffi-grovel" not found

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {10005205B3}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK Component "cffi-grovel" not found #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK SB-EXT:*INVOKE-DEBUGGER-HOOK* Component "cffi-grovel" not found)
2: (INVOKE-DEBUGGER Component "cffi-grovel" not found)
3: (ERROR MISSING-COMPONENT :REQUIRES "cffi-grovel")
4: ((:METHOD OPERATE (SYMBOL T)) LOAD-OP "cffi-grovel" :VERBOSE NIL) [fast-method]
5: ((SB-PCL::EMF OPERATE) #<unused argument> #<unused argument> LOAD-OP "cffi-grovel" :VERBOSE NIL)
6: ((LAMBDA NIL :IN OPERATE))
7: ((:METHOD OPERATE :AROUND (T T)) LOAD-OP "cffi-grovel" :VERBOSE NIL) [fast-method]
8: (LOAD-SYSTEM "cffi-grovel" :VERBOSE NIL)
9: (QUICKLISP-CLIENT::CALL-WITH-MACROEXPAND-PROGRESS #<CLOSURE (LAMBDA NIL :IN QUICKLISP-CLIENT::APPLY-LOAD-STRATEGY) {10035DD75B}>)
10: (QUICKLISP-CLIENT::AUTOLOAD-SYSTEM-AND-DEPENDENCIES "cffi-grovel" :PROMPT NIL)
11: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION (T T)) #<unused argument> #<CLOSURE (FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {10032CA12B}>) [fast-method]
12: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION :AROUND (QL-IMPL:SBCL T)) #<QL-IMPL:SBCL {1005FAB703}> #<CLOSURE (FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {10032CA12B}>) [fast-method]
13: ((:METHOD QUICKLISP-CLIENT:QUICKLOAD (T)) "cffi-grovel" :PROMPT NIL :SILENT T :VERBOSE NIL) [fast-method]
14: (QL-DIST::CALL-WITH-CONSISTENT-DISTS #<CLOSURE (LAMBDA NIL :IN QUICKLISP-CLIENT:QUICKLOAD) {10032BACBB}>)
15: ((FLET "H0" :IN QLOT/INSTALL::APPLY-QLFILE-TO-QLHOME) Component "cffi-grovel" not found, required by NIL)
16: (SB-KERNEL::%SIGNAL Component "cffi-grovel" not found, required by NIL)
17: (ERROR MISSING-DEPENDENCY :REQUIRED-BY NIL :REQUIRES "cffi-grovel")
18: (ASDF/FIND-COMPONENT:RESOLVE-DEPENDENCY-NAME NIL "cffi-grovel" NIL)
19: ((LAMBDA NIL :IN ASDF/PARSE-DEFSYSTEM:REGISTER-SYSTEM-DEFINITION))
20: (SB-INT:SIMPLE-EVAL-IN-LEXENV (DEFSYSTEM "woo" :VERSION "0.11.5" :AUTHOR "Eitaro Fukamachi" :LICENSE "MIT" :DEFSYSTEM-DEPENDS-ON ("cffi-grovel") :DEPENDS-ON ("lev" "clack-socket" "swap-bytes" "cffi" "static-vectors" "bordeaux-threads" "fast-http" "quri" "fast-io" "smart-buffer" "trivial-utf-8" "vom" ...) ...) #<NULL-LEXENV>)
21: (SB-EXT:EVAL-TLF (DEFSYSTEM "woo" :VERSION "0.11.5" :AUTHOR "Eitaro Fukamachi" :LICENSE "MIT" :DEFSYSTEM-DEPENDS-ON ("cffi-grovel") :DEPENDS-ON ("lev" "clack-socket" "swap-bytes" "cffi" "static-vectors" "bordeaux-threads" "fast-http" "quri" "fast-io" "smart-buffer" "trivial-utf-8" "vom" ...) ...) 0 NIL)
22: ((LABELS SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (DEFSYSTEM "woo" :VERSION "0.11.5" :AUTHOR "Eitaro Fukamachi" :LICENSE "MIT" :DEFSYSTEM-DEPENDS-ON ("cffi-grovel") :DEPENDS-ON ("lev" "clack-socket" "swap-bytes" "cffi" "static-vectors" "bordeaux-threads" "fast-http" "quri" "fast-io" "smart-buffer" "trivial-utf-8" "vom" ...) ...) 0)
23: ((LAMBDA (SB-KERNEL:FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) (DEFSYSTEM "woo" :VERSION "0.11.5" :AUTHOR "Eitaro Fukamachi" :LICENSE "MIT" :DEFSYSTEM-DEPENDS-ON ("cffi-grovel") :DEPENDS-ON ("lev" "clack-socket" "swap-bytes" "cffi" "static-vectors" "bordeaux-threads" "fast-http" "quri" "fast-io" "smart-buffer" "trivial-utf-8" "vom" ...) ...) :CURRENT-INDEX 0)
24: (SB-C::%DO-FORMS-FROM-INFO #<CLOSURE (LAMBDA (SB-KERNEL:FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) {1003291A4B}> #<SB-C::SOURCE-INFO {1003291A03}> SB-C::INPUT-ERROR-IN-LOAD)
25: (SB-INT:LOAD-AS-SOURCE #<SB-INT:FORM-TRACKING-STREAM for "file /Users/art/projects/lisp/test-woo/quicklisp/dists/woo/software/woo-52bf0ede5c2798cb170e0c298e834ee67d79c3d8/woo.asd" {100328EBC3}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
26: ((FLET SB-FASL::THUNK :IN LOAD))
27: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<CLOSURE (FLET SB-FASL::THUNK :IN LOAD) {21FD28B}> #<SB-INT:FORM-TRACKING-STREAM for "file /Users/art/projects/lisp/test-woo/quicklisp/dists/woo/software/woo-52bf0ede5c2798cb170e0c298e834ee67d79c3d8/woo.asd" {100328EBC3}>)
28: ((FLET SB-FASL::LOAD-STREAM :IN LOAD) #<SB-INT:FORM-TRACKING-STREAM for "file /Users/art/projects/lisp/test-woo/quicklisp/dists/woo/software/woo-52bf0ede5c2798cb170e0c298e834ee67d79c3d8/woo.asd" {100328EBC3}> NIL)
29: (LOAD #P"/Users/art/projects/lisp/test-woo/quicklisp/dists/woo/software/woo-52bf0ede5c2798cb170e0c298e834ee67d79c3d8/woo.asd" :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT :UTF-8)
30: (CALL-WITH-MUFFLED-CONDITIONS #<CLOSURE (LAMBDA NIL :IN LOAD*) {100328B06B}> ("Overwriting already existing readtable ~S." #(#:FINALIZERS-OFF-WARNING :ASDF-FINALIZERS)))
31: ((FLET "THUNK" :IN PERFORM))
32: (SB-IMPL::%WITH-STANDARD-IO-SYNTAX #<CLOSURE (FLET "THUNK" :IN PERFORM) {21FD5DB}>)
33: ((:METHOD PERFORM (DEFINE-OP SYSTEM)) #<DEFINE-OP > #<ASDF/SYSTEM:UNDEFINED-SYSTEM "woo">) [fast-method]
34: ((SB-PCL::EMF PERFORM) #<unused argument> #<unused argument> #<DEFINE-OP > #<ASDF/SYSTEM:UNDEFINED-SYSTEM "woo">)
35: ((LAMBDA NIL :IN ASDF/ACTION:CALL-WHILE-VISITING-ACTION))
36: ((:METHOD PERFORM-WITH-RESTARTS :AROUND (T T)) #<DEFINE-OP > #<ASDF/SYSTEM:UNDEFINED-SYSTEM "woo">) [fast-method]
37: ((:METHOD PERFORM-PLAN (T)) #<SEQUENTIAL-PLAN {100326FD53}>) [fast-method]
38: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
39: ((:METHOD PERFORM-PLAN :AROUND (T)) #<SEQUENTIAL-PLAN {100326FD53}>) [fast-method]
40: ((:METHOD OPERATE (OPERATION COMPONENT)) #<DEFINE-OP > #<ASDF/SYSTEM:UNDEFINED-SYSTEM "woo"> :PLAN-CLASS NIL :PLAN-OPTIONS NIL) [fast-method]
41: ((SB-PCL::EMF OPERATE) #<unused argument> #<unused argument> #<DEFINE-OP > #<ASDF/SYSTEM:UNDEFINED-SYSTEM "woo">)
42: ((LAMBDA NIL :IN OPERATE))
43: ((:METHOD OPERATE :AROUND (T T)) #<DEFINE-OP > #<ASDF/SYSTEM:UNDEFINED-SYSTEM "woo">) [fast-method]
44: ((LAMBDA NIL :IN FIND-SYSTEM))
45: (ASDF/SESSION:CONSULT-ASDF-CACHE (FIND-SYSTEM "woo") #<CLOSURE (LAMBDA NIL :IN FIND-SYSTEM) {100323CA0B}>)
46: ((:METHOD FIND-COMPONENT (STRING T)) "woo" (NIL) :REGISTERED NIL) [fast-method]
47: ((:METHOD OPERATE (OPERATION T)) #<DEFINE-OP > ("woo")) [fast-method]
48: ((SB-PCL::EMF OPERATE) #<unused argument> #<unused argument> #<DEFINE-OP > ("woo"))
49: ((LAMBDA NIL :IN OPERATE))
50: ((:METHOD OPERATE :AROUND (T T)) #<DEFINE-OP > ("woo")) [fast-method]
51: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<CLOSURE (LAMBDA NIL :IN OPERATE) {100323BB0B}> :OVERRIDE T :KEY NIL :OVERRIDE-CACHE T :OVERRIDE-FORCING NIL)
52: ((LAMBDA NIL :IN OPERATE))
53: ((:METHOD OPERATE :AROUND (T T)) #<DEFINE-OP > #<ASDF/SYSTEM:UNDEFINED-SYSTEM "woo">) [fast-method]
54: ((LAMBDA NIL :IN LOAD-ASD))
55: ((LAMBDA NIL :IN FIND-SYSTEM))
56: (ASDF/SESSION:CONSULT-ASDF-CACHE (FIND-SYSTEM "woo") #<CLOSURE (LAMBDA NIL :IN FIND-SYSTEM) {10018DEBAB}>)
57: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<CLOSURE (LAMBDA NIL :IN FIND-SYSTEM) {10018DEBAB}> :OVERRIDE NIL :KEY (FIND-SYSTEM "woo") :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
58: ((LABELS QLOT/INSTALL::SYSTEM-DEPENDENCIES :IN QLOT/INSTALL::APPLY-QLFILE-TO-QLHOME) "woo")
59: ((LAMBDA NIL :IN QLOT/INSTALL::APPLY-QLFILE-TO-QLHOME))
60: (QLOT/UTIL:CALL-IN-LOCAL-QUICKLISP #<CLOSURE (LAMBDA NIL :IN QLOT/INSTALL::APPLY-QLFILE-TO-QLHOME) {1005F18E3B}> #P"/Users/art/projects/lisp/test-woo/quicklisp/" :SYSTEMS (#P"/Users/art/projects/lisp/test-woo/test.asd") :CENTRAL-REGISTRY (#P"/Users/art/.roswell/local-projects/fukamachi/qlot/"))
61: (QLOT/INSTALL::APPLY-QLFILE-TO-QLHOME #P"/Users/art/projects/lisp/test-woo/qlfile" #P"/Users/art/projects/lisp/test-woo/quicklisp/" :IGNORE-LOCK NIL :PROJECTS NIL)
62: (QLOT/INSTALL:INSTALL-QLFILE #P"/Users/art/projects/lisp/test-woo/qlfile" :QUICKLISP-HOME #P"/Users/art/projects/lisp/test-woo/quicklisp/")
63: (ROS/SCRIPT/QLOT::MAIN "install")
64: (SB-INT:SIMPLE-EVAL-IN-LEXENV (APPLY (QUOTE ROS/SCRIPT/QLOT::MAIN) ROSWELL:*ARGV*) #<NULL-LEXENV>)
65: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ROSWELL:QUIT (APPLY (QUOTE ROS/SCRIPT/QLOT::MAIN) ROSWELL:*ARGV*)) #<NULL-LEXENV>)
66: (SB-EXT:EVAL-TLF (ROSWELL:QUIT (APPLY (QUOTE ROS/SCRIPT/QLOT::MAIN) ROSWELL:*ARGV*)) NIL NIL)
67: ((LABELS SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (ROSWELL:QUIT (APPLY (QUOTE ROS/SCRIPT/QLOT::MAIN) ROSWELL:*ARGV*)) NIL)
68: (SB-INT:LOAD-AS-SOURCE #<CONCATENATED-STREAM :STREAMS NIL {1002E0E853}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
69: ((FLET SB-FASL::THUNK :IN LOAD))
70: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<CLOSURE (FLET SB-FASL::THUNK :IN LOAD) {21FF53B}> #<CONCATENATED-STREAM :STREAMS NIL {1002E0E853}>)
71: ((FLET SB-FASL::LOAD-STREAM :IN LOAD) #<CONCATENATED-STREAM :STREAMS NIL {1002E0E853}> NIL)
72: (LOAD #<CONCATENATED-STREAM :STREAMS NIL {1002E0E853}> :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT :DEFAULT)
73: ((FLET ROSWELL::BODY :IN ROSWELL:SCRIPT) #<SB-SYS:FD-STREAM for "file /Users/art/.roswell/bin/qlot" {1002E0A043}>)
74: (ROSWELL:SCRIPT "/Users/art/.roswell/bin/qlot" "install")
75: (ROSWELL:RUN ((:EVAL "(ros:asdf)") (:EVAL "(ros:quicklisp)") (:SCRIPT "/Users/art/.roswell/bin/qlot" "install") (:QUIT NIL)))
76: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ROSWELL:RUN (QUOTE ((:EVAL "(ros:asdf)") (:EVAL "(ros:quicklisp)") (:SCRIPT "/Users/art/.roswell/bin/qlot" "install") (:QUIT NIL)))) #<NULL-LEXENV>)
77: (EVAL (ROSWELL:RUN (QUOTE ((:EVAL "(ros:asdf)") (:EVAL "(ros:quicklisp)") (:SCRIPT "/Users/art/.roswell/bin/qlot" "install") (:QUIT NIL)))))
78: (SB-IMPL::PROCESS-EVAL/LOAD-OPTIONS ((:EVAL . "(progn #-ros.init(cl:load \"/Users/art/usr/roswell/etc/roswell/init.lisp\"))") (:EVAL . "(ros:run '((:eval\"(ros:asdf)\")(:eval\"(ros:quicklisp)\")(:script \"/Users/art/.roswell/bin/qlot\"\"install\")(:quit ())))")))
79: (SB-IMPL::TOPLEVEL-INIT)
80: ((FLET SB-UNIX::BODY :IN SB-EXT:SAVE-LISP-AND-DIE))
81: ((FLET "WITHOUT-INTERRUPTS-BODY-34" :IN SB-EXT:SAVE-LISP-AND-DIE))
82: ((LABELS SB-IMPL::RESTART-LISP :IN SB-EXT:SAVE-LISP-AND-DIE))

unhandled condition in --disable-debugger mode, quitting
;
; compilation unit aborted
;   caught 1 fatal ERROR condition

But with woo installed from Quicklisp (I've tried 2019-01-07 version), qlot install works fine.

Probably, this should be fixed in the Qlot? What do you think, Eitaro?

(ql:quickload "woo") on lipspworks 7.0 failed.

Error: Error while trying to load definition for system woo from pathname /home/hft/quicklisp/dists/quicklisp/software/woo-20160318-git/woo.asd:
COMPILE-FILE-ERROR while compiling #<CL-SOURCE-FILE "cffi-toolchain" "toolchain" "asdf-compat">
1 (continue) Retry compiling #<CL-SOURCE-FILE "cffi-toolchain" "toolchain" "asdf-compat">.
2 Continue, treating compiling #<CL-SOURCE-FILE "cffi-toolchain" "toolchain" "asdf-compat"> as having been successful.
3 (abort) Give up on "cffi-grovel"
4 Try loading /home/hft/quicklisp/dists/quicklisp/software/woo-20160318-git/woo.asd again.
5 Give up loading /home/hft/quicklisp/dists/quicklisp/software/woo-20160318-git/woo.asd.
6 Try loading another file instead of /home/hft/quicklisp/dists/quicklisp/software/woo-20160318-git/woo.asd.
7 Retry ASDF operation.
8 Retry ASDF operation after resetting the configuration.
9 Give up on "woo"
10 Return to level 0.
11 Return to top loop level 0.

Support IPv6

Is it hard to support binding to IPv6 in Woo?

I found woo is the only handler for Clack which accepts :address argument, but when I pass "::0" it is failing. And looking at the code seems it only support IPv4 for now.

I have a bunch os IPv6 only backends and want to run ningle based microservice on them. Having support for IPv6 in Woo whould be wonderful!

P.S. — found a great tutorial Porting applications to IPv6 HowTo . May be will be able to fix woo myself. Anyway, any guidance from you will be highly appreciated! Thank you for great libraries!

Can you please tell us what each of them does?

Hi Fukamashi,

I m very interested in your packages but i m wondering what is the role they play?

I came across fast-http, clack, and woo as well.

I would like to develop a robust social networking applicaiton with image processing and gaming oriented, with this you can imagine moments of surge or requests.

what packages of yours or what could you recommend to use?

which among your packages provides a micro-style of web framework similar to python-flask? rest framework, streaming capability for 2-way video streaming for communications?

can you please advise?

compile error solaris..

; Loading "woo"
...; cc -m64 -o /root/.cache/common-lisp/sbcl-1.3.12-solaris-x64/root/quicklisp/dists/quicklisp/software/woo-20170227-git/src/llsocket/grovel__grovel-tmpGHU3ALSV -I/root/quicklisp/dists/quicklisp/software/cffi_0.18.0/ /root/.cache/common-lisp/sbcl-1.3.12-solaris-x64/root/quicklisp/dists/quicklisp/software/woo-20170227-git/src/llsocket/grovel__grovel.c
In file included from /root/.cache/common-lisp/sbcl-1.3.12-solaris-x64/root/quicklisp/dists/quicklisp/software/woo-20170227-git/src/llsocket/grovel__grovel.c:11:0:
/root/.cache/common-lisp/sbcl-1.3.12-solaris-x64/root/quicklisp/dists/quicklisp/software/woo-20170227-git/src/llsocket/grovel__grovel.c: In function 'main':
/root/quicklisp/dists/quicklisp/software/cffi_0.18.0/grovel/common.h:10:62: error: 'struct msghdr' has no member named 'msg_control'
#define offsetof(type, slot) ((long) ((char *) &(((type *) 0)->slot)))
^
/root/.cache/common-lisp/sbcl-1.3.12-solaris-x64/root/quicklisp/dists/quicklisp/software/woo-20170227-git/src/llsocket/grovel__grovel.c:1208:56: note: in expansion of macro 'offsetof'
fprintf(output, " :offset %lli)", (long long signed) offsetof(struct msghdr, msg_control));
^
/root/quicklisp/dists/quicklisp/software/cffi_0.18.0/grovel/common.h:10:62: error: 'struct msghdr' has no member named 'msg_controllen'
#define offsetof(type, slot) ((long) ((char *) &(((type *) 0)->slot)))
^
/root/.cache/common-lisp/sbcl-1.3.12-solaris-x64/root/quicklisp/dists/quicklisp/software/woo-20170227-git/src/llsocket/grovel__grovel.c:1213:56: note: in expansion of macro 'offsetof'
fprintf(output, " :offset %lli)", (long long signed) offsetof(struct msghdr, msg_controllen));
^
/root/quicklisp/dists/quicklisp/software/cffi_0.18.0/grovel/common.h:10:62: error: 'struct msghdr' has no member named 'msg_flags'
#define offsetof(type, slot) ((long) ((char *) &(((type *) 0)->slot)))
^
/root/.cache/common-lisp/sbcl-1.3.12-solaris-x64/root/quicklisp/dists/quicklisp/software/woo-20170227-git/src/llsocket/grovel__grovel.c:1218:56: note: in expansion of macro 'offsetof'
fprintf(output, " :offset %lli)", (long long signed) offsetof(struct msghdr, msg_flags));
^

debugger invoked on a CFFI-GROVEL:GROVEL-ERROR: Subprocess (:PROCESS #<SB-IMPL::PROCESS :EXITED 1>)
with command ("cc" "-m64" "-o" "/root/.cache/common-lisp/sbcl-1.3.12-solaris-x64/root/quicklisp/dists/quicklisp/software/woo-20170227-git/src/llsocket/grovel__grovel-tmpGHU3ALSV" "-I/root/quicklisp/dists/quicklisp/software/cffi_0.18.0/" "/root/.cache/common-lisp/sbcl-1.3.12-solaris-x64/root/quicklisp/dists/quicklisp/software/woo-20170227-git/src/llsocket/grovel__grovel.c")
exited with error code 1

HTTP/2 support?

Is there any idea of supporting HTTP/2?

socket is not closed after the client request ended

the socket fd is not closed after the client request ended.
reproduce procedure:

use the benchmark/woo.lisp
$sbcl --script benchmark/woo.lisp
Running at http://localhost:5000/

in another terminal,
$ ps -A|grep sbcl
12374 pts/3 00:00:00 sbcl
$ ls -l /proc/12374/fd
total 0
lrwx------ 1 64 May 20 19:11 0 -> /dev/pts/3
lrwx------ 1 64 May 20 19:11 1 -> /dev/pts/3
lrwx------ 1 64 May 20 19:10 2 -> /dev/pts/3
lr-x------ 1 64 May 20 19:11 3 -> /usr/lib/sbcl/sbcl.core
lr-x------ 1 64 May 20 19:11 4 -> /usr/lib/sbcl/sbcl.core
lrwx------ 1 64 May 20 19:11 5 -> /dev/tty
lr-x------ 1 64 May 20 19:11 6 -> /woo/benchmark/woo.lisp
lrwx------ 1 64 May 20 19:11 7 -> anon_inode:[eventpoll]
lrwx------ 1 64 May 20 19:11 8 -> socket:[4047374]

$ http http://localhost:5000
HTTP/1.1 200 OK
Connection: keep-alive
Date: Wed, 20 May 2015 23:12:02 GMT
Transfer-Encoding: chunked

Hello, World

$ ls -l /proc/12374/fd
total 0
lrwx------ 1 64 May 20 19:11 0 -> /dev/pts/3
lrwx------ 1 64 May 20 19:11 1 -> /dev/pts/3
lrwx------ 1 64 May 20 19:10 2 -> /dev/pts/3
lr-x------ 1 64 May 20 19:11 3 -> /usr/lib/sbcl/sbcl.core
lr-x------ 1 64 May 20 19:11 4 -> /usr/lib/sbcl/sbcl.core
lrwx------ 1 64 May 20 19:11 5 -> /dev/tty
lr-x------ 1 64 May 20 19:11 6 -> /woo/benchmark/woo.lisp
lrwx------ 1 64 May 20 19:11 7 -> anon_inode:[eventpoll]
lrwx------ 1 64 May 20 19:11 8 -> socket:[4047374]
lrwx------ 1 64 May 20 19:12 9 -> socket:[4047376] ------> note this one added

the newly added socket fd will be kept forever.

this causes process error: too many files opened.

Make all header values be strings

The content-length header in the hash-table is not a string as the other fields, when it should be. While you would certainly want to use the content-length as a number after being parsed, this can make parsing raw headers more difficult.

Example: https://github.com/fukamachi/woo/blob/master/src/woo.lisp#L328
You also expose content-length as a plist value: https://github.com/fukamachi/woo/blob/master/src/woo.lisp#L244
Maybe the latter could be changed to be the parsed value (therefore a number or nil), while the header remains untouched?

For a counter-example, hunchentoot exposes content-length as string.

I have not looked more intensely at the source code, but i believe content-length is the only header field that gets parsed.

Parsing multipart/form-data with file upload fails

Data in a POST with multipart/form-data seems to not be consistently handled in the manner one would expect.

(defvar *app* (make-instance 'ningle:<app>))

(setf (ningle:route *app* "/x" :method :post)
      #'(lambda (params)
          (declare (ignore params))
      "Hey!"))

(clack:clackup
 *app*
 :server :woo
 :port 8083
 :use-default-middlewares nil)

Trying to post with a file upload triggers "Callback Error: the message-complete callback failed".

echo 'testing123' > /tmp/x.txt
curl -F 'a="1234"' -F 'file0=@"/tmp/x.txt"' --trace-ascii /tmp/curl.out http://127.0.0.1:8083/x

I tried digging around a bit and it seems that perhaps the data provided to FAST-HTTP.PARSER's PARSE-REQUEST does not reflect the original content of the POST request. Compare the trace provided by curl and the data that is provided to PARSE-REQUEST:

The trace for curl:

== Info:   Trying 127.0.0.1...
== Info: Connected to 127.0.0.1 (127.0.0.1) port 8083 (#0)
=> Send header, 209 bytes (0xd1)
0000: POST /x HTTP/1.1
0012: Host: 127.0.0.1:8083
0028: User-Agent: curl/7.47.0
0041: Accept: */*
004e: Content-Length: 291
0063: Expect: 100-continue
0079: Content-Type: multipart/form-data; boundary=--------------------
00b9: ----929b14fd4ca5ce6b
00cf: 
== Info: Done waiting for 100-continue
=> Send data, 232 bytes (0xe8)
0000: --------------------------929b14fd4ca5ce6b
002c: Content-Disposition: form-data; name="a"
0056: 
0058: "1234"
0060: --------------------------929b14fd4ca5ce6b
008c: Content-Disposition: form-data; name="file0"; filename="x.txt"
00cc: Content-Type: text/plain
00e6: 
=> Send data, 11 bytes (0xb)
0000: testing123.
=> Send data, 48 bytes (0x30)
0000: 
0002: --------------------------929b14fd4ca5ce6b--
<= Recv header, 26 bytes (0x1a)
0000: HTTP/1.1 400 Bad Request
<= Recv header, 37 bytes (0x25)
0000: Date: Thu, 28 Apr 2016 15:38:59 GMT
<= Recv header, 19 bytes (0x13)
0000: Connection: close
<= Recv header, 20 bytes (0x14)
0000: Content-Length: 15
<= Recv header, 2 bytes (0x2)
0000: 
<= Recv data, 15 bytes (0xf)
0000: 400 Bad Request
== Info: Closing connection 0

The data that PARSE-REQUEST sees:

testing123

--------------------------929b14fd4ca5ce6b--
tion: form-data; name="a"

"1234"
--------------------------929b14fd4ca5ce6b
Content-Disposition: form-data; name="file0"; filename="x.txt"
Content-Type: text/plain

Apologies in advance if I'm missing something obvious...

QURI.ERROR:URL-DECODING-ERROR

Hi!

SERVER> server
#S(CLACK.HANDLER::HANDLER
:SERVER :WOO
:ACCEPTOR #<SB-THREAD:THREAD "clack-handler-woo" RUNNING {102E4C3983}>)

curl "http://127.0.0.1:2222/local:1%love"

Callback Error: the message-complete callback failed
Condition QURI.ERROR:URL-DECODING-ERROR was signalled.
[Condition of type FAST-HTTP.ERROR:CB-MESSAGE-COMPLETE]

Restarts:
0: [ABORT] abort thread (#<THREAD "clack-handler-woo" RUNNING {102E4C3983}>)

Backtrace:
0: ((FLET "H0" :IN FAST-HTTP.PARSER::PARSE-BODY) #)
1: (SB-KERNEL::%SIGNAL #<QURI.ERROR:URL-DECODING-ERROR {102F43A173}>)
2: (ERROR QURI.ERROR:URL-DECODING-ERROR)
3: (QURI.DECODE::HEXDIGIT-TO-INTEGER #)
4: (URL-DECODE "local:1%love" :ENCODING # :START # :END # :LENIENT T)
5: (WOO::HANDLE-REQUEST # #S(WOO.EV.SOCKET:SOCKET :WATCHERS #(#.(SB-SYS:INT-SAP #X7FFFEC011A60) #.(SB-SYS:INT-SAP #X7FFFEC011A20) #.(SB-SYS:INT-SAP #X7FFFEC0119E0)) :LAST-ACTIVITY 1..
6: ((LAMBDA NIL :IN WOO::SETUP-PARSER))
7: ((LAMBDA (FAST-HTTP.HTTP:HTTP) :IN FAST-HTTP:MAKE-PARSER) #)
8: (FAST-HTTP.PARSER::PARSE-BODY #S(FAST-HTTP.HTTP:HTTP-REQUEST :METHOD :GET :MAJOR-VERSION 1 :MINOR-VERSION 1 :STATUS 0 :CONTENT-LENGTH NIL :CHUNKED-P NIL ...) # #<unavailable argu..
9: (FAST-HTTP.PARSER:PARSE-REQUEST #S(FAST-HTTP.HTTP:HTTP-REQUEST :METHOD :GET :MAJOR-VERSION 1 :MINOR-VERSION 1 :STATUS 0 :CONTENT-LENGTH NIL :CHUNKED-P NIL ...) #S(FAST-HTTP.PARSER:CALLBACKS :MESSAGE-B..
10: ((LAMBDA (FAST-HTTP::DATA &KEY :START :END) :IN FAST-HTTP:MAKE-PARSER) #(71 69 84 32 47 208 ...) :START # :END #)
11: (WOO::READ-CB #S(WOO.EV.SOCKET:SOCKET :WATCHERS #(#.(SB-SYS:INT-SAP #X7FFFEC011A60) #.(SB-SYS:INT-SAP #X7FFFEC011A20) #.(SB-SYS:INT-SAP #X7FFFEC0119E0)) :LAST-ACTIVITY 1.5263082135862458e9 :FD 47 :REM..
12: ((LAMBDA (WOO.EV.TCP::EVLOOP WOO.EV.TCP::WATCHER WOO.EV.TCP::EVENTS) :IN "/home/sun/quicklisp/dists/quicklisp/software/woo-20170830-git/src/ev/tcp.lisp") # # #<u..
13: ((LAMBDA (SB-ALIEN::ARGS-POINTER SB-ALIEN::RESULT-POINTER FUNCTION) :IN "/home/sun/quicklisp/dists/quicklisp/software/woo-20170830-git/src/ev/socket.lisp") # #<unavailable argume..
14: ("foreign function: funcall_alien_callback")
15: ("foreign function: callback_wrapper_trampoline")
16: ("foreign function: #x201014CC")
17: ("foreign function: #x406832E919B63900")
18: ("foreign function: ev_run")
19: (LEV:EV-RUN # #)

please add an error handler.
thanks.

Woo won't start on Ubuntu Trusty, SBCL 1.5.6 and gcc 4.8.4-2ubuntu1~14.04.4

When I start the Clack with Woo handler, it tries to compile a grovel file, output the following
commands to stdout:

; gcc -o /root/.cache/common-lisp/sbcl-1.5.6-linux-x64/root/finances/.qlot/dists/ultralisp/software/fukamachi-woo\
-20190913121619/src/syscall/types__grovel-tmp4TM0ME1F.o -c -g -Wall -Wundef -Wsign-compare -Wpointer-arith -O3 -D\
_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Wunused-parameter -fno-omit-frame-pointer -momit-\
leaf-frame-pointer -fno-pie -fPIC -I/root/finances/.qlot/dists/quicklisp/software/cffi_0.20.1/ /root/.cache/commo\
n-lisp/sbcl-1.5.6-linux-x64/root/finances/.qlot/dists/ultralisp/software/fukamachi-woo-20190913121619/src/syscall\
/types__grovel.c
; gcc -o /root/.cache/common-lisp/sbcl-1.5.6-linux-x64/root/finances/.qlot/dists/ultralisp/software/fukamachi-woo\
-20190913121619/src/syscall/types__grovel-tmpU2IL9N4 -g -Wl,--export-dynamic -no-pie /root/.cache/common-lisp/sbc\
l-1.5.6-linux-x64/root/finances/.qlot/dists/ultralisp/software/fukamachi-woo-20190913121619/src/syscall/types__gr\
ovel.o

And raises a condition:

Subprocess #<UIOP/LAUNCH-PROGRAM::PROCESS-INFO {1002C79A13}>
 with command ("gcc" "-o"
               "/root/.cache/common-lisp/sbcl-1.5.6-linux-x64/root/finances/.qlot/dists/ultralisp/software/fukamachi-woo-20190913121619/src/syscall/types__grovel-tmpU2IL9N4"
               "-g" "-Wl,--export-dynamic" "-no-pie"
               "/root/.cache/common-lisp/sbcl-1.5.6-linux-x64/root/finances/.qlot/dists/ultralisp/software/fukamachi-woo-20190913121619/src/syscall/types__grovel.o")
 exited with error code 1
   [Condition of type CFFI-GROVEL:GROVEL-ERROR]

Restarts:
 0: [RETRY] Retry #<PROCESS-OP > on #<GROVEL-FILE "woo" "src" "syscall" "types">.
 1: [ACCEPT] Continue, treating #<PROCESS-OP > on #<GROVEL-FILE "woo" "src" "syscall" "types"> as having been successful.
 2: [RETRY] Retry ASDF operation.
 3: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration.
 4: [RETRY] Retry ASDF operation.
 5: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the configuration.
 --more--

Backtrace:
 0: (CFFI-GROVEL:GROVEL-ERROR "~a" #<UIOP/RUN-PROGRAM:SUBPROCESS-ERROR {1002C7B193}>)
 1: ((FLET "THUNK" :IN CFFI-GROVEL:PROCESS-GROVEL-FILE))
 2: (SB-IMPL::%WITH-STANDARD-IO-SYNTAX #<CLOSURE (FLET "THUNK" :IN CFFI-GROVEL:PROCESS-GROVEL-FILE) {7FA9D395C43B}>)
 3: (CFFI-GROVEL:PROCESS-GROVEL-FILE #P"/root/finances/.qlot/dists/ultralisp/software/fukamachi-woo-20190913121619/src/syscall/types.lisp" #P"/root/.cache/common-lisp/sbcl-1.5.6-linux-x64/root/finances/.q..
 4: ((:METHOD ASDF/ACTION:PERFORM (CFFI-GROVEL::PROCESS-OP CFFI-GROVEL:GROVEL-FILE)) #<CFFI-GROVEL::PROCESS-OP > #<CFFI-GROVEL:GROVEL-FILE "woo" "src" "syscall" "types">) [fast-method]
 5: ((SB-PCL::EMF ASDF/ACTION:PERFORM) #<unused argument> #<unused argument> #<CFFI-GROVEL::PROCESS-OP > #<CFFI-GROVEL:GROVEL-FILE "woo" "src" "syscall" "types">)
 6: ((LAMBDA NIL :IN ASDF/ACTION:CALL-WHILE-VISITING-ACTION))
 7: ((:METHOD ASDF/ACTION:PERFORM :AROUND (CFFI-GROVEL::PROCESS-OP CFFI-GROVEL::CC-FLAGS-MIXIN)) #<CFFI-GROVEL::PROCESS-OP > #<CFFI-GROVEL:GROVEL-FILE "woo" "src" "syscall" "types">) [fast-method]
 8: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS :AROUND (T T)) #<CFFI-GROVEL::PROCESS-OP > #<CFFI-GROVEL:GROVEL-FILE "woo" "src" "syscall" "types">) [fast-method]
 9: ((:METHOD ASDF/PLAN:PERFORM-PLAN (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1005D461B3}>) [fast-method]
10: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
11: ((:METHOD ASDF/PLAN:PERFORM-PLAN :AROUND (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1005D461B3}>) [fast-method]
12: ((:METHOD ASDF/OPERATE:OPERATE (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "clack-handler-woo"> :PLAN-CLASS NIL :PLAN-OPTIONS NIL) [fast-meth..
13: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "clack-handler-woo"> :VERBOSE NIL)
14: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
15: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "clack-handler-woo"> :VERBOSE NIL) [fast-method]
16: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> ASDF/LISP-ACTION:LOAD-OP "clack-handler-woo" :VERBOSE NIL)
17: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
18: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP "clack-handler-woo" :VERBOSE NIL) [fast-method]
19: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<CLOSURE (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {1005D36DCB}> :OVERRIDE T :KEY NIL :OVERRIDE-CACHE T :OVERRIDE-FORCING NIL)
20: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
21: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<CLOSURE (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {1005D2C05B}> :OVERRIDE NIL :KEY NIL :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
22: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP "clack-handler-woo" :VERBOSE NIL) [fast-method]
23: (ASDF/OPERATE:LOAD-SYSTEM "clack-handler-woo" :VERBOSE NIL)
24: (QUICKLISP-CLIENT::CALL-WITH-MACROEXPAND-PROGRESS #<CLOSURE (LAMBDA NIL :IN QUICKLISP-CLIENT::APPLY-LOAD-STRATEGY) {1005D2BFAB}>)
25: (QUICKLISP-CLIENT::AUTOLOAD-SYSTEM-AND-DEPENDENCIES "clack-handler-woo" :PROMPT NIL)
26: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION (T T)) #<unused argument> #<CLOSURE (FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {1005D0B14B}>) [fast-method]
27: ((:METHOD QL-IMPL-UTIL::%CALL-WITH-QUIET-COMPILATION :AROUND (QL-IMPL:SBCL T)) #<QL-IMPL:SBCL {1003B96C53}> #<CLOSURE (FLET QUICKLISP-CLIENT::QL :IN QUICKLISP-CLIENT:QUICKLOAD) {1005D0B14B}>) [fast-me..
28: ((:METHOD QUICKLISP-CLIENT:QUICKLOAD (T)) "clack-handler-woo" :PROMPT NIL :SILENT T :VERBOSE NIL) [fast-method]
29: (QL-DIST::CALL-WITH-CONSISTENT-DISTS #<CLOSURE (LAMBDA NIL :IN QUICKLISP-CLIENT:QUICKLOAD) {1005CFE7AB}>)
30: (LACK.UTIL::LOAD-WITH-QUICKLISP "clack-handler-woo")
31: (LACK.UTIL:FIND-PACKAGE-OR-LOAD "CLACK.HANDLER.WOO")
32: (CLACK.UTIL:FIND-HANDLER :WOO)
33: (CLACK:CLACKUP #<CLOSURE (LAMBDA (LACK.MIDDLEWARE.SESSION::ENV) :IN "/root/finances/.qlot/dists/ultralisp/software/fukamachi-lack-20190831205316/src/middleware/session.lisp") {1005CFCBEB}> :ADDRESS "l..
34: (WEBLOCKS/SERVER::START-SERVER #<SERVER port=8080 stopped> :DEBUG T)
35: ((LAMBDA (#:G0 &REST #:G1) :IN WEBLOCKS/SERVER:START) NIL)
36: (WEBLOCKS/SERVER:START :DEBUG T :PORT 8080 :INTERFACE "localhost" :SERVER-TYPE :WOO)
37: (FINANCES/SERVER:START :PORT 8080)
38: (SB-INT:SIMPLE-EVAL-IN-LEXENV (FINANCES/SERVER:START :PORT 8080) #<NULL-LEXENV>)
39: (EVAL (FINANCES/SERVER:START :PORT 8080))
40: ((LAMBDA NIL :IN SLYNK-MREPL::MREPL-EVAL-1))
41: (SLYNK::CALL-WITH-RETRY-RESTART "Retry SLY mREPL evaluation request." #<CLOSURE (LAMBDA NIL :IN SLYNK-MREPL::MREPL-EVAL-1) {1005CF80CB}>)
42: ((LAMBDA NIL :IN SLYNK-MREPL::MREPL-EVAL-1))
43: ((LAMBDA NIL :IN SLYNK::CALL-WITH-LISTENER))
44: (SLYNK::CALL-WITH-BINDINGS ((*PACKAGE* . #<PACKAGE "COMMON-LISP-USER">) (*) (** . #1=#<SB-THREAD:THREAD "telegram-bot" RUNNING {10047BEE03}>) (***) (/ NIL) (// #1#) ...) #<CLOSURE (LAMBDA NIL :IN SLYN..
45: (SLYNK-MREPL::MREPL-EVAL-1 #<SLYNK-MREPL::MREPL mrepl-1-1> "(finances/server:start :port 8080 )")
46: (SLYNK-MREPL::MREPL-EVAL #<SLYNK-MREPL::MREPL mrepl-1-1> "(finances/server:start :port 8080 )")
47: (SLYNK::PROCESS-REQUESTS NIL)
48: ((LAMBDA NIL :IN SLYNK::SPAWN-CHANNEL-THREAD))
49: ((LAMBDA NIL :IN SLYNK::SPAWN-CHANNEL-THREAD))
50: (SLYNK-SBCL::CALL-WITH-BREAK-HOOK #<FUNCTION SLYNK:SLYNK-DEBUGGER-HOOK> #<CLOSURE (LAMBDA NIL :IN SLYNK::SPAWN-CHANNEL-THREAD) {1003CF00DB}>)
51: ((FLET SLYNK-BACKEND:CALL-WITH-DEBUGGER-HOOK :IN "/root/finances/.qlot/dists/ultralisp/software/joaotavora-sly-20190915232901/slynk/backend/sbcl.lisp") #<FUNCTION SLYNK:SLYNK-DEBUGGER-HOOK> #<CLOSURE ..
52: ((LAMBDA NIL :IN SLYNK::CALL-WITH-LISTENER))
53: (SLYNK::CALL-WITH-BINDINGS ((*PACKAGE* . #<PACKAGE "COMMON-LISP-USER">) (*) (** . #1=#<SB-THREAD:THREAD "telegram-bot" RUNNING {10047BEE03}>) (***) (/ NIL) (// #1#) ...) #<CLOSURE (LAMBDA NIL :IN SLYN..
54: ((LAMBDA NIL :IN SLYNK::SPAWN-CHANNEL-THREAD))
55: ((FLET SB-UNIX::BODY :IN SB-THREAD::NEW-LISP-THREAD-TRAMPOLINE))
56: ((FLET "WITHOUT-INTERRUPTS-BODY-4" :IN SB-THREAD::NEW-LISP-THREAD-TRAMPOLINE))
57: ((FLET SB-THREAD::WITH-MUTEX-THUNK :IN SB-THREAD::NEW-LISP-THREAD-TRAMPOLINE))
58: ((FLET "WITHOUT-INTERRUPTS-BODY-1" :IN SB-THREAD::CALL-WITH-MUTEX))
59: (SB-THREAD::CALL-WITH-MUTEX #<CLOSURE (FLET SB-THREAD::WITH-MUTEX-THUNK :IN SB-THREAD::NEW-LISP-THREAD-TRAMPOLINE) {7FA9D395ED9B}> #<SB-THREAD:MUTEX "thread result lock" owner: #<SB-THREAD:THREAD "sly..
60: (SB-THREAD::NEW-LISP-THREAD-TRAMPOLINE #<SB-THREAD:THREAD "sly-channel-1-mrepl-remote-1" RUNNING {1003CCB4F3}> NIL #<CLOSURE (LAMBDA NIL :IN SLYNK::SPAWN-CHANNEL-THREAD) {1003CCAADB}> NIL)
61: ("foreign function: call_into_lisp")
62: ("foreign function: new_thread_trampoline")

Seems that gcc does not like a -no-pie option. Because first command have -fno-pie and compiles without an error.

Moreover, when I tried to execute second command, gcc says:

# gcc -o /root/.cache/common-lisp/sbcl-1.5.6-linux-x64/root/finances/.qlot/dists/ultralisp/software/fukamachi-woo-20190913121619/src/syscall/types__grovel-tmp6XOIRI3N -g -Wl,--export-dynamic -no-pie /root/.cache/common-lisp/sbcl-1.5.6-linux-x64/root/finances/.qlot/dists/ultralisp/software/fukamachi-woo-20190913121619/src/syscall/types__grovel.o
gcc: error: unrecognized command line option ‘-no-pie’

And replacing it with -fno-pie, makes this command pass.

Probably, this issue should be reported to the grovel's tracker?

Managing the temporary files internally?

The temporary files made by smart-buffers will stick around after we're done with them, so manual removal or using a script to clear out the /tmp folder is needed. It is also very bad if we accidentally remove one that's still in use. If the temporary files could be exposed, cleanup can be done internally.

Undefined aliens on CentOS 7

When deploying woo in an centos7 container, i get these messages:

STYLE-WARNING: Undefined alien: "ev_default_loop_uc_"
STYLE-WARNING: Undefined alien: "ev_is_default_loop"
STYLE-WARNING: Undefined alien: "ev_loop"
STYLE-WARNING: Undefined alien: "ev_unloop"
STYLE-WARNING: Undefined alien: "ev_default_destroy"
STYLE-WARNING: Undefined alien: "ev_default_fork"
STYLE-WARNING: Undefined alien: "ev_loop_count"
STYLE-WARNING: Undefined alien: "ev_loop_depth"
STYLE-WARNING: Undefined alien: "ev_loop_verify"

and:
[14:04:14] woo.ev.tcp - Unexpected error (Code: 104)

Are there any constraints on the libev version? libev-4.15 is installed.
Apart from the error message, it seems to work fine.

Limiting maximum size for receiving large post data

When we POST data like a large file upload, woo downloads it all to a temporary file via smart-buffers. If we could possibly limit the maximum size, it would be useful for enforcing a hard file upload limit. Would this be a desirable feature or should it be handled independently via client-side/proxy-server?

I know PHP likes to download a file before checking the filesize. However nginx has something like this: https://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size, so I'm wondering what stance woo has.