unbit/uwsgi

uWSGI application server container

uwsgi's Issues

suspicious code in core/async.c reported by scan-build

scan-build reports that timeout set in line 259 is never read, so is this a dead store or the if at line 261 is supposed to be an else if?

256 else {
257 min_timeout = uwsgi_min_rb_timer(uwsgi.rb_async_timeouts);
258 if (uwsgi.async_runqueue_cnt) {
259 timeout = 0;

Value stored to 'timeout' is never read
260 }
261 if (min_timeout) {
262 timeout = min_timeout->key - uwsgi_now();
263 if (timeout <= 0) {
264 async_expire_timeouts();
265 timeout = 0;
266 }
267 }
268 else {
269 timeout = -1;
270 }
271 }

Stacked cheaper algos

I wanted to add new backlog handling feature to cheaper subsystem and this change would go the cheaper_busyness plugin, but then I though that we could have much better cheaper subsystem if we could stack multiple cheaper algos. This way all the backlog monitoring and worker count adjustments would go to separate plugin and there would be no need for single fat super-plugin reimplementing features that are already there.
In uwsgi_calc_cheaper() we could call a list of algos and then pick the biggest number and use that as needed_workers value.
This way cheaper_busyness can concentrate on busyness tracking, busyness might not indicate that we need more workers but backlog monitoring might want to start few.
It would be nice if each algo could have separate interval since backlog should be checked every master cycle, while busyness needs bigger intervals for smooth operation.

spooler-max-tasks not respawning spooler in uwsgi 1.3

with uwsgi 1.3 it looks like spooler-max-tasks most of the times does not work as advertised:

[spooler /foo/spooler pid: 4185] done with task uwsgi_spoolfile_on_foo_29150_3_0_1350930600_53133 after 120 seconds
[spooler /foo/spooler pid: 4185] managing request uwsgi_spoolfile_on_foo_29150_3_0_1350930600_53912 ...

I've seen one time it respawning fine against ten not respawning.

This is the relevant conf:

<spooler>%d/spooler</spooler> <spooler-max-tasks>1</spooler-max-tasks>

Reloading configuration without restart

Right now any change in vassal config will cause its restart but some options could be changed without that, for example --max-requests or --cron. I guess that this would require big changes in options handling (like adding requires restart flag to option definition or so) and I don't really need it right away, but it would be great if uWSGI could eventually have such feature.

Standalone with HTTPS support in 1.3 stable responds in cleartext

Using 1.3 stable as standalone server with HTTPS support, I can send requests to the server in HTTPS, but the responses come in cleartext. The logs report a successful HTTP/1.1 200, but tcpdump shows cleartext bodies.

My cert is self signed. I've also tried many different config variations. Using straight HTTP mode with everything else left the same works like a charm.

This generates a -- (Error code: ssl_error_rx_record_too_long) -- in firefox.

My config: http://pastebin.com/VGFuSNKN
Server debug log: http://pastebin.com/F8N9EHZb
Openssl s_client test results: http://pastebin.com/rGhD7SnF

Allows defining custom metrics

Currently uWSGI exports in its various stats subsystems only a static set of metrics.

It would be great to allow for custom defined metrics.

Metrics could be created with

metric = foo,bar,x,y,x

Then you can increase/decrese with

uwsgi.metric_inc('foo')
uwsgi.metric_dec('foo')
uwsgi.metric_set('foo', 0)

metric are 64bit unsigned values (or it is better to allow for negative values ?)

metrics could be 'attached' to loglines, so for example it could be interesting to count the number of error 500:

log-metric = foo (HTTP/1.\d 500)

this will increase 'foo' metric automagically.

Metric are automatically exported to stats server and external plugin like the carbon one

Other ideas ?

Feature: Distributed SSL cache

A nice feature for the new https router would be the ability to cache SSL sessions, and share this cache across workers.

The next step would be a way to share this cache across uWSGI instances, which can reside in different machines.

I think this would have a big impact on performance.

Storage plugins

It is very common to want to store 'static resources' out of the filesystem. An example is gridfs or zookper trees. Another common scenario is wanting to store objects in uWSGI plugins in some persistent way independent by the underlying architecture.

My idea is adding the concept of storage plugin.

A storage plugin can support the following "methods"

"get" -> retrieve the body of an object (given a key)
"set" -> store an object with the supplied content
"stat" -> check for availability of an object and its metadata
"rm" -> destroy an object (given a key)
"attr" -> add metadata to an object (an empty value destroy the metadata)

Storage instances are created by config:

storage = foobar gridfs:127.0.0.1:4041,mycollection

and can be referred by other components by instance names:

static-map = /images foobar:/static/images

or plugins can refer to them with

uwsgi_storage_stat("foobar", "/images/foo.png");

Issues:

• the api has to support streamed content, for example a set can be build in a 'transactioned' way:

1. create the object
2. add 4k of body
3. add 8k of body
4. add ...k of body
5. commit/save

otherwise you should store in memory the whole body.

• Multithread support

not all of the library can be used in a non-blocking way. GridFS for example can only be used in multithread modes, so the api must be versatile enough to support both.

SIGSEGV when running tracebacker on loaded workers

DETAILS

per discussion on ML, i've found a repro case. this uses the dumbloop, as it matrches the exact problem i'm dealing with (and makes it easy to keep worker loaded).

example simulates a loaded AMQP worker by publishing to the loopback on an artificial memory:// transport (see queue_worker.py for more detail), and then rapidly connects to the tracebacker using a socat busy loop; should segfault in approx 5-10 seconds.

REPRODUCE

1. get uWSGI HEAD
2. add dumbloop to build
3. build uWSGI
4. get Kombu and deps
5. run queue_worker.py
6. die in ~5s :)

COMMANDS

export PIP_REQUIRE_VIRTUALENV=
export PYTHON=$(type -p python2.7 python2.6 python2 python | head -n1)

git clone --depth 1 https://github.com/unbit/uwsgi __bug &&
cd __bug &&
sed -i 's/python/dumbloop,&/' buildconf/default.ini &&
"$PYTHON" uwsgiconfig.py --build &&
"$PYTHON" -m pip install --target eggs kombu &&
wget https://raw.github.com/gist/3934532/c9c1e3b9822e006ae0ce3bcc895fa07be5ef1cf7/queue_worker.py &&
(

    ./uwsgi --master --pp=eggs --loop=dumb --dumbloop-code=queue_worker.py \
        --dumbloop-function=main --enable-threads --py-tracebacker=trace &
    sleep 3
    while socat UNIX-CLIENT:trace1 FILE:/dev/null; do :; done
    kill -INT %%

)

Emperor Mode: Unix socket not removed on termination

I'm running uwsgi 1.4.2 in emperor mode.

/usr/bin/uwsgi --emperor /etc/uwsgi.d --daemonize /dev/null --disable-logging --lazy --protocol uwsgi --pidfile /var/run/uwsgi/uwsgi.pid

And this is the PHP Vassal config:

[uwsgi]
plugins = php54
uid = nginx
gid = nginx
processes = 15
cheap = true
socket = /tmp/php.sock
php-allowed-docroot=/var/www
php-allowed-ext=php
php-index=index.php
idle = 5
chdir=/var/www

Take a serious note on the socket parameter, it's a Unix socket and not a TCP socket.
When I start uwsgi, no troubles. It works.

The problem comes when SIGQUIT is issued to the parent process.
It doesn't remove the Unix socket (viz /tmp/php.sock) on termination.

This problem doesn't occur when using TCP sockets obviously, because no process is listening on the TCP socket.

The disadvantage of the Unix socket not being removed is, when you restart, say you made some changes to the startup parameters, the socket has to be removed manually or uwsgi fails to start the vassal because the socket is already bound!

It starts throwing errors of socket is already bound and throttles the vassal. When you remove the socket file manually (with or without terminating uwsgi parent process), everything is back to normal.

HEAD and GET in static document serving

In HTTP/HTTPS standalone server mode, It looks like issuing a HEAD to a document under static root returns the full document, and omits Content-type. Issuing a GET to a static doc also omits Content-type.

This causes problems with IE as it rejects stylesheets not served with text/css as the mime-type unless in quirks/compatibility mode. In my situation, I need standards mode so canvas will work.

A normal request (non-static)

Escape character is '^]'.
HEAD / HTTP/1.0

HTTP/1.0 302 FOUND
Vary: Cookie
Content-Type: text/html; charset=utf-8
Location: http://dev.shieldcore.com/logout/

A static HEAD request

Escape character is '^]'.
HEAD /static/css/style.css HTTP/1.0

HTTP/1.0 200 OK
Content-Length: 19123
Last-Modified: Sat, 27 Oct 2012 02:02:45 GMT

/* Color Schemes */

.H2O1804 a { color: #FF9A00; }

A static GET request

GET /static/css/style.css HTTP/1.0

[pid: 26280|app: 0|req: 23/44] 192.168.56.101 () {20 vars in 248 bytes} [Fri Oct 26 19:27:39 2012] GET /static/css/style.css => generated 19123 bytes in 0 msecs via sendfile() (HTTP/1.0 200) 2 headers in 88 bytes (0 switches on core 0)
HTTP/1.0 200 OK
Content-Length: 19123
Last-Modified: Sat, 27 Oct 2012 02:02:45 GMT

/* Color Schemes */

.H2O1804 a { color: #FF9A00; }

The log entry:

[pid: 26280|app: 0|req: 17/37] 192.168.56.101 () {20 vars in 249 bytes} [Fri Oct 26 19:22:35 2012] HEAD /static/css/style.css => generated 19123 bytes in 0 msecs via sendfile() (HTTP/1.0 200) 2 headers in 88 bytes (0 switches on core 0)

RFC: more advanced app clustering

Most apps running production code are run on several nodes and there are many cases when some kind of coordination between those nodes is required (for example to ensure that only one app at a time does some specific work), most of the time this is done using database (by creating locks or writing some metadata). uWSGI can be used to run some code (like daemons or crons) but AFAIK it lacks any kind of cluster wide cooperation, if I want to run cron task in my cluster than should only run on single node at any given time, than I need to ensure than no other node will execute it, this can be achieved either by:

• adding cron entry only to one node config - but in case of node failure I need to manually move this cron task to another node
• using some wrapper that creates a lock and executes task, or skip execution if lock is already present - this provides HA but requires setting up additional clustering software (like corosync or similar tools to provide reliable locking, or just redis if high reliability isn't required)

I believable that we can add clustered and coordinated cron execution across cluster of nodes without too much work, pieces required:

• cluster locking so that uWSGI can create lock across cluster, we can also call that remote locking and use redis with SETNX command to provide locks (or even maybe create locking subsystem with pluggable lockers: redis, mongodb, whatever), such locks could be added to API

• with cluster wide locks we can promote one node to cluster master, in case of redis that would be the first node that got lock (locks should expire and master should update them and reset ttl before they will expire)

• with one node being master we can coordinate what node will execute what task but now we need to have the list of all cron commands but since cron settings doesn't need to be the same on all nodes than we should be able to define cron tasks in few different ways:

  • --cron %(interval) %(cmd) - cron will only be executed on local node, it won't use cluster lock features
  • --cluster-cron %(interval) %(cmd) - cron task will be executed on any node that contains this option

  Every node should communicate with master and send him the list of defined --cluster-cron options.
  Master than would would setup schedules for next round of each cron executions, pseudo metadata:

  crons {
    'mycmd1': {last_run: 123456,  last_node: 192.168.1.10:3000, next_run: 12445, next_node: 192.168.1.111:3555, last_rcode: 0}
  }
  

• now each node would read metadata from master every minute and check if it should run any command, master should provide desired cron distribution algorithm so that one node will not be hammered

• beside cron tasks we could add cluster wide singleton lock for daemon execution, so we could run daemon that should only be executed on single node and if this node dies it should be restarted on another node (for example celery beat - periodic task scheduler for celery tasks)

This is general idea, it might not be the best one but it provides starting point. Questions to be answered first:

• Should such functionality be included in uWSGI itself? Right now I'm using python script wrapper for my cron tasks, it uses redis to lock task execution to single node but I would like to move this to uWSGI itself.
• I don't think that uWSGI should provide advanced cluster tools, I would opt for using remote databases like redis or mongodb for providing locks, storing metadata and all other cluster wide informations that should be stored reliably, is this right direction?

I wanted to discuss this before I start hacking any code, all comments are welcome.

Retry failed requests in FastRouter

I have 1 FastRouter and 3 subscribed nodes.
If I run ab -c 6 -n 1000000 -r http://myapp.com/login and during this ab run I will stop 2 nodes I often get at least 1 error.

Concurrency Level:      6
Time taken for tests:   315.490 seconds
Complete requests:      18924
Failed requests:        5
   (Connect: 0, Receive: 0, Length: 5, Exceptions: 0)

This probably happens if FastRouter forwards a request to a node that is stopping or just dies. It's the race between node failing/stopping and marking node as such.
Would it be possible for FastRouter to retry request on connection errors?

SNI support

SNI is a technique to host multiple domains under https on the same ip. Basically it is vhost over https.

As always i want to support it in the most 'automagic' way, allowing admins to add SNI hosts on the fly

(like passing the key/certificate pair in a crypted subscription packet)

static files issues

I have varnish -> nginx -> uWSGI fastrouter -> uWSGI backends
I've noticed issue below with static files serving when I connect directly to nginx

Oct 19 12:48:09 localhost myapp: sendfile(): Broken pipe [core/sendfile.c line 251]
Oct 19 12:48:24 localhost myapp: SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /interface/build/script/site.js (ip ) !!!
Oct 19 12:48:24 localhost myapp: SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /interface/build/script/site.js (ip ) !!!

About 3 on every 10 requests results in sendfile error, files are being served from local disk, there are no errors in nginx, I have no network issues and when I try to fetch those files over varnish it always works. It seems weird that I'm getting sendfile(): Broken pipe errors even when connecting to nginx from localhost, maybe there is some bug with file descriptor handling on static files?
Varnish seems to hide this bug in some way.

nginx config:

user www-data;
worker_processes 2;
worker_rlimit_nofile 200000;
pid /var/run/nginx.pid;

events {
    use epoll;
    multi_accept on;
    worker_connections 65536;
}

http {

    sendfile on;
    tcp_nopush  on;
    tcp_nodelay on;
    gzip off;
    keepalive_timeout  120;

    access_log off;
    error_log /var/log/nginx/error.log;

    server {
    listen   8080;

        include /etc/nginx/conf.d/*.uwsgi;

        location / {
            include uwsgi_params;
            uwsgi_buffering on;
            uwsgi_buffers 8 16k;
            uwsgi_buffer_size 32k;
            uwsgi_param REDIRECT_STATUS 200;
            uwsgi_param UWSGI_FASTROUTER_KEY $host;
            uwsgi_param SCRIPT_NAME "";
            uwsgi_pass  unix:///var/run/uwsgi.socket;
        }
    }

    include /etc/nginx/conf.d/*.conf;

}

fastrouter config:

[uwsgi]
logger = syslog
threaded-logger = true
fastrouter = /var/run/uwsgi.socket
chmod-socket = 666
fastrouter-subscription-server = :2626
fastrouter-stats = :2580
fastrouter-processes = 2
subscription-tolerance = 60
master = true
listen = 1000
fastrouter-events = 256
fastrouter-timeout = 60
subscription-dotsplit = true
subscription-algo = lrc

vassal config:

[uwsgi]
autoload = true
plugins-dir = /usr/lib/uwsgi/plugins
socket = 192.168.10.87:3055
stats = :4055
add-header = X-SDK-Cachable: 0
pidfile = /var/run/emperor-site_staging.pid
uid = www-data
gid = www-data
master = true
vacuum = true
no-orphans = true
reload-mercy = 300
post-buffering = 4096
memory-report = true
auto-procname = true
procname-prefix-spaced = [site_staging@2970]
processes = 5
cheaper-algo = busyness
cheaper = 1
cheaper-step = 1
cheaper-initial = 1
cheaper-overload = 15
cheaper-busyness-multiplier = 20
cheaper-busyness-penalty = 3
cheaper-busyness-min = 20
cheaper-busyness-max = 50
cheaper-busyness-backlog-multiplier = 5
cheaper-busyness-backlog-step = 1
max-requests = 10000
reload-on-rss = 256
harakiri = 120
harakiri-verbose = true
namespace = /sudoku/site_staging-2970:a87
chdir = /home/site_staging/
cgroup = /cgroup/sudoku.site_staging.55
cgroup-opt = memory.limit_in_bytes=2684354560
cgroup-opt = memory.memsw.limit_in_bytes=2684354560
cgroup-opt = cpu.shares=4096
ksm = 20

if-exists = /sudoku/site_staging-2970/etc/dropbear/banner
exec-as-root = chown -R www-data /etc/dropbear
namespace-keep-mount = /dev/pts
attach-daemon = /usr/sbin/dropbear -b /etc/dropbear/banner -w -s -j -k -p 5055 -E -F -I 300
endif =

check-static = /mfs # MooseFS shared fs
namespace-keep-mount = /mnt/mfs/55:/mfs
env = RAILS_ENV=staging
plugins = rack
rack = /home/site_staging/config.ru
remap-modifier = 7:0
ruby-gc-freq = 100
env = GEM_HOME=/home/site_staging/vendor/bundle/ruby/1.9.1
check-static = /home/site/public
static-index = index.html

unsubscribe-on-graceful-reload = true
subscribe-freq = 3

subscribe-to = 192.168.10.115:2626:my.app.com

logger = rsyslog:127.0.0.1:514,myapp
threaded-logger = true
log-x-forwarded-for = true
disable-logging = true
log-slow = 1000
log-4xx = true
log-5xx = true

timer decorator and cheaped workers

If I use timer decorator and all my workers will be cheaped than also timer will stop working since it is executed in first worker. Docs say that timer can be executed using spooler instead of worker, but it doesn't say anything about cheaping workers with --idle option.
This should be either:

1. documented - so that users will be warned (provided that they will read docs) and maybe print big fat warning that using --idle with timer signal (and possibly other) does not mix
2. disable --idle when signals are used - since user wants to handle some signals and we need worker to do so it's better to disable one minor feature than to break other that is potentially important
3. fix it in some other way

timerapp.ini:

[uwsgi]
autoload = true
plugins-dir = /usr/lib/uwsgi/plugins
plugins = python
socket = :0
master = true
uid = nobody
gid = nogroup
processes = 1
wsgi-file = timerapp.py
idle = 5

timerapp.py:

import uwsgi
from uwsgidecorators import rbtimer


@rbtimer(1)
def timer(sig):
    print('Ping!')


def application(env, start_response):
    start_response('200 OK', [('Content-Type','text/html')])
    return 'uWSGI is alive!'

app log:

[uWSGI] getting INI configuration from timerapp.ini
*** Starting uWSGI 1.4.1 (64bit) on [Thu Nov 22 22:23:06 2012] ***
compiled with version: 4.4.3 on 13 November 2012 09:26:57
os: Linux-3.0.0-27-virtual #44~lucid1-Ubuntu SMP Fri Oct 19 16:05:20 UTC 2012
nodename: sudoku-dev-backend1
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /home/app
detected binary path: /usr/bin/uwsgi
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 bound to TCP address :26411 (port auto-assigned) fd 3
Python version: 2.6.5 (r265:79063, Oct  1 2012, 22:16:31)  [GCC 4.4.3]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1b6f5d0
your server socket listen backlog is limited to 100 connections
mapped 144784 bytes (141 KB) for 1 cores
*** Operational MODE: single process ***
[uwsgi-signal] signum 0 registered (wid: 0 modifier1: 0 target: default, any worker)
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x1b6f5d0 pid: 9480 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 9480)
spawned uWSGI worker 1 (pid: 9481, cores: 1)
Ping!
Ping!
Ping!
Ping!
Ping!
Ping!
workers have been inactive for more than 5 seconds (1353619393-1353619387)
cheap mode enabled: waiting for socket connection...

touch-reload style for clearing the cache

In a project (my company website) i have added

touch-reload = .git/index

to trigger an instance reload whenever the git repository changes. This is mainly to clear the uWSGI cache as each page (after generated) is put there and served using the 'cache internal router'

Reloading the whole server (only to clear the cache) seems overkill, so a

--touch-cache-clear

would be a great (and cheap) solution

add UWSGI_REQUEST_ID concept

When there are multiple players in the game, like nginx, fastrouter and various instances, identifying a request during log analysis is pretty hard.

For example we are currently facing an SSL issue hard to debug as matching errors in the https router to the ones in the instances is almost impossibile.

Generating a unique id in the foremost player and passing it to the lower layers would be useful, and such id should be reported in all of the requests error log lines.

The id generator should be pluggable as some users could need something like uuid while other may need other combinations (or they could already have some internal generator).

log-master fails on log messages larger than 2k in Mac OS X

I can't use --log-master on Mac OS X to pass python logging output (from StreamHandler) through the master, because log messages larger than 2k fail to emit, with "IOError: [Errno 40] Message too long".

Also, messages close to exactly 2k in length sometimes fail with "IOError: [Errno 55] No buffer space available"

Changing setlinebuf(stdout); to setbuf(stdout, (char*)malloc(4096)); in uwsgi.c:1736 makes the former problem (Errno 40) go away; all messages get through. But long messages still get truncated at either 1024, 2048, 3072 or 4096 bytes (and that same "IOError: [Errno 55] No buffer space available" error and stack trace gets printed right after). Making that stdout buffer bigger (even much bigger) does not help.

So it seems like there are two separate problems, both seeming to involve the OS X pipe buffer, and that setlinebuf → setbuf change fixes works around one of them.

off by one(?)... malloc(): Cannot allocate memory... [section] loading in INI files

loading by section, and in general, seems to have regressed a bit in 1.4.1:

# git clone https://github.com/unbit/uwsgi.git
# cd uwsgi
# python2 uwsgiconfig.py --build
# echo -e '[uwsgi]\nprint=uwsgi\n[section]\nprint=section' > dead.ini

Pass(P)/Fail(-) ...

P ./uwsgi --ini dead.ini
- ./uwsgi --ini dead.ini:section
P ./uwsgi --ini ./dead.ini:section
- ./uwsgi --ini - < dead.ini
P ./uwsgi --ini fd://0 < dead.ini (but complains about missing file?)

realpath() of - failed: No such file or directory [core/utils.c line 4263]
malloc(): Cannot allocate memory [core/utils.c line 2206]
!!! tried memory allocation of 18446744073680636930 bytes !!!
*** backtrace of 5549 ***
./uwsgi(uwsgi_backtrace+0x25) [0x444b05]
./uwsgi(uwsgi_malloc+0x55) [0x41b945]
./uwsgi(uwsgi_concat2n+0x3d) [0x41c39d]
./uwsgi(config_magic_table_fill+0x7a) [0x44372a]
./uwsgi(uwsgi_opt_load_ini+0x11) [0x4439c1]
./uwsgi(add_exported_option+0x2af) [0x41cbdf]
./uwsgi(uwsgi_commandline_config+0x4a) [0x43972a]
./uwsgi(main+0x759) [0x4192e9]
/usr/lib/libc.so.6(__libc_start_main+0xf5) [0x7fa7f8489725]
./uwsgi() [0x41a1d5]
*** end of backtrace ***

...and likely some others.

RELATED: inherit/include cannot handle [sections] because they expect an extension (.js, .ini, .xml, ...) at the end

RELATED: howto reference a different [section] of the same file??? %o:section and other flavors usually work, but fail for unknown reasons at times (i suspect when the path has /../ or other dots, but unconfirmed)

[emperor dir:// relpaths] chdir(): No such file or directory [core/emperor.c line 146]

uwsgi_imperial_monitor_directory fails to properly handle relative paths (simple directory only... glob handler is ok); uwsgi_imperial_monitor_directory_init properly sets up, but uwsgi_imperial_monitor_directory later fails to account for a relpath (since it's already in uwsgi.emperor_absolute_dir and not uwsgi.cwd)

likely simple fix... but i just noticed it before bed... however, on quick scan, it looks like core/emperor.c line 145 just needs to use uwsgi.emperor_absolute_dir... untested though.

examples are simplest case (ie. no master proc, etc)

INIT

# CFLAGS='-ggdb -O0' python2 uwsgiconfig.py --build
# mkdir vassals-example/

PASS

# ./uwsgi --emperor $(pwd)/vassals-example/

FAIL

# ./uwsgi --emperor dir://./vassals-example/
# ./uwsgi --emperor dir://vassals-example/
# ./uwsgi --emperor ./vassals-example/
# ./uwsgi --emperor vassals-example/

SystemError raised when access to `listen_queue` method of uwsgi in FreeBSD

When call uwsgi.listen_queue() in FreeBSD, it raised:

SystemError: error return without exception set

sample code:

#! /usr/bin/env python
import uwsgi

def application(env, start_response):
    start_response('200 OK', [('Content-Type','text/html')])
    listen_queue = uwsgi.listen_queue()
    return "listen queue: {0}".format(listen_queue)

Python errors in Django app that only occur when using uWSGI

I'm getting a couple python errors described here: pandas-dev/pandas#1741

When I run my app with Django's dev server (manage.py runserver) or Gunicorn, I don't get these errors and everything works fine, which leads me to believe there is some uWSGI-specific issue here.

Anyone here have any idea what this could be? Thanks for any help in advance!

Bad permissions on emperor directory file cause segfault

I war running emperor with --uid www-data --gid www-data --ini /some/dir

but the directory permissions were d-wxrw---x

this caused the emperor to segfault!

Invalid offload() log messages

{address space usage: 19288064 bytes/18MB} {rss usage: 7012352 bytes/6MB} [pid: 5054|app: 0|req: -1/1] 192.168.10.92 () {26 vars in 433 bytes} [Fri Dec  7 12:33:48 2012] GET /URL => generated 0 bytes in 7 msecs via offload() (HTTP/1.0 -30) 2 headers in 92 bytes (0 switches on core 0)

7 msecs this file is big (~600MB) so it wasn't fetched so fast, this is probably the time between request start and sending it to offload thread

(HTTP/1.0 -30) - since offload is taking care of this file, than I think it's safe to just hardcode it to 200 (or 304 if static offloading takes care of that)

FastRouter hangs on request with non-subscribed domain

If I send a request with Host header containing some random domain that is not subscribed by any node then it just hangs. We should have 503 instead.
IIt used to work fine so I guess that this issue is a result of last FastRouter modifications.

curl -vI gdjfdhfdhfdhfbd.domain.com
* About to connect() to gdjfdhfdhfdhfbd.domain.com port 80 (#0)
*   Trying 172.16.200.53...
* connected
* Connected to gdjfdhfdhfdhfbd.domain.com (172.16.200.53) port 80 (#0)
> HEAD / HTTP/1.1
> User-Agent: curl/7.27.0
> Host: gdjfdhfdhfdhfbd.domain.com
> Accept: */*
> 
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0
* additional stuff not fine transfer.c:1037: 0 0

worker continue die with gevent loop engine

uWSGI 1.4.2
gevent 1.0rc1

INI file:
[uwsgi]
gid = pgy
uid = pgy
socket = 127.0.0.1:3131
master = true
processes = 1
loop = gevent
async = 50
enable-threads = true
harakiri = 60
socket-timeout = 30
disable-logging = true
daemonize = /home/pgy/log/uwsgi/%n.log
virtualenv = /home/pgy/venv
module = runserver
callable = app

uwsgi log:
*** Starting uWSGI 1.4.2 (64bit) on [Thu Nov 29 00:11:56 2012] ***
compiled with version: 4.6.3 on 28 November 2012 10:45:02
os: Linux-3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012
nodename: pgy-web1
machine: x86_64
clock source: unix
detected number of CPU cores: 8
current working directory: /home/pgy/pgysocial
detected binary path: /usr/local/bin/uwsgi
your processes number limit is 63631
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 10240

• async cores set to 50 - fd table size: 10240
  lock engine: pthread robust mutexes
  uwsgi socket 0 bound to TCP address 127.0.0.1:3131 fd 3
  Python version: 2.7.3 (default, Aug 1 2012, 05:25:23) [GCC 4.6.3]
  Set PythonHome to /home/pgy/venv
  Python main interpreter initialized at 0x241e910
  python threads support enabled
  your server socket listen backlog is limited to 100 connections
  mapped 1117728 bytes (1091 KB) for 50 cores
  *** Operational MODE: async ***
  WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0x241e910 pid: 6932 (default app)
  *** uWSGI is running in multiple interpreter mode ***
  spawned uWSGI master process (pid: 6932)
  spawned uWSGI worker 1 (pid: 6939, cores: 50)
  *** running gevent loop engine [addr:0x45d140] ***
  !!! uWSGI process 6939 got Segmentation Fault !!!
  *** backtrace of 6939 ***
  uwsgi(uwsgi_backtrace+0x25) [0x442bb5]
  uwsgi(uwsgi_segfault+0x21) [0x442c91]
  /lib/x86_64-linux-gnu/libc.so.6(+0x364a0) [0x7fe4251e94a0]
  /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x3a) [0x7fe4256b2e6a]
  /usr/lib/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47) [0x7fe4256b37d7]
  uwsgi(python_call+0x24) [0x44e6f4]
  uwsgi(uwsgi_gevent_current_wsgi_req+0x1d) [0x45e89d]
  uwsgi(warn_pipe+0xa) [0x43fdea]
  /lib/x86_64-linux-gnu/libc.so.6(+0x364a0) [0x7fe4251e94a0]
  /lib/x86_64-linux-gnu/libpthread.so.0(send+0x22) [0x7fe4267cc282]
  /usr/lib/libpython2.7.so.1.0(+0x81ea7) [0x7fe4255f3ea7]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x4f18) [0x7fe4256033b8]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fe4255ce605]
  /usr/lib/libpython2.7.so.1.0(+0x5c7bd) [0x7fe4255ce7bd]
  /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fe4256b2e83]
  /usr/lib/libpython2.7.so.1.0(+0xd5047) [0x7fe425647047]
  /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fe4256b2e83]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x3fcd) [0x7fe42560246d]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fe4255ce605]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5420) [0x7fe4256038c0]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fe4256045eb]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fe4256045eb]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fe4256045eb]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fe4255ce605]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5420) [0x7fe4256038c0]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fe4256045eb]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fe4256045eb]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fe4256045eb]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fe4256045eb]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fe4256045eb]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fe4256045eb]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fe4256045eb]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fe4255ce605]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5420) [0x7fe4256038c0]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fe4255ce605]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5420) [0x7fe4256038c0]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fe4255ce605]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5420) [0x7fe4256038c0]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fe4255ce605]
  /usr/lib/libpython2.7.so.1.0(+0x5c7bd) [0x7fe4255ce7bd]
  /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fe4256b2e83]
  /usr/lib/libpython2.7.so.1.0(+0x1251cf) [0x7fe4256971cf]
  /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fe4256b2e83]
  /usr/lib/libpython2.7.so.1.0(PyObject_CallFunctionObjArgs+0xb0) [0x7fe4256f8a40]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x17fe) [0x7fe4255ffc9e]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fe4255ce605]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5420) [0x7fe4256038c0]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fe4255ce605]
  /usr/lib/libpython2.7.so.1.0(+0x5c7bd) [0x7fe4255ce7bd]
  /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fe4256b2e83]
  /usr/lib/libpython2.7.so.1.0(+0x1251cf) [0x7fe4256971cf]
  /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fe4256b2e83]
  /usr/lib/libpython2.7.so.1.0(+0x13136c) [0x7fe4256a336c]
  /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fe4256b2e83]
  /usr/lib/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47) [0x7fe4256b37d7]
  uwsgi(python_call+0x24) [0x44e6f4]
  uwsgi(uwsgi_request_wsgi+0xe1) [0x450a81]
  uwsgi(py_uwsgi_gevent_request+0xc2) [0x45eb12]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x4f18) [0x7fe4256033b8]
  /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fe4255ce605]
  /usr/lib/libpython2.7.so.1.0(+0x5c7bd) [0x7fe4255ce7bd]
  /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fe4256b2e83]
  /usr/lib/libpython2.7.so.1.0(+0x1251cf) [0x7fe4256971cf]
  /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fe4256b2e83]
  *** end of backtrace ***
  DAMN ! worker 1 (pid: 6939) died :( trying respawn ...
  Respawned uWSGI worker 1 (new pid: 7008)

Kill all daemon child processes on detach

With option below I can run dropbear instance inside my vassal chroot (vassal is using --namespace)

attach-daemon = /usr/sbin/dropbear -w -s -j -k -p 1022 -E -F -I 300

But if I will log in with ssh to this dropbear instance, than during vassal stop or reload uWSGI will hang until this child process is terminated.
To solve this we would need to kill all daemon children before or after killing deamon pid, but after a little googling I can't find any way to get all children pids of given parent pid. It seems that it requires /proc parsing and that would be limited to linux only, I have no idea how to implement it on *bsd.

I can leave without it since I can kill dropbear using helper scripts when I reload my vassals, but if it would be grate if uWSGI could handle it by itself. If not then maybe docs should warn about such issues.

uWSGI thinks worker is running but it is not

{
    "cwd": "/home/app", 
    "daemons": [
        {
            "cmd": "/usr/sbin/dropbear -b /etc/dropbear/banner -w -s -j -k -p 5054 -E -F -I 300", 
            "pid": 19855, 
            "respawns": 447
        }
    ], 
    "gid": 33, 
    "listen_queue": 5, 
    "listen_queue_errors": 0, 
    "load": 5, 
    "locks": [
        {
            "user 0": 0
        }, 
        {
            "signal": 0
        }, 
        {
            "filemon": 0
        }, 
        {
            "timer": 0
        }, 
        {
            "probe": 0
        }, 
        {
            "rbtimer": 0
        }, 
        {
            "cron": 0
        }, 
        {
            "rpc": 0
        }
    ], 
    "pid": 1, 
    "uid": 33, 
    "version": "1.3.1-dev-a3e9a28", 
    "workers": [
        {
            "apps": [
                {
                    "chdir": "", 
                    "exceptions": 0, 
                    "id": 0, 
                    "modifier1": 7, 
                    "mountpoint": "", 
                    "requests": 5, 
                    "startup_time": 12
                }
            ], 
            "avg_rt": 39, 
            "cores": [
                {
                    "id": 0, 
                    "in_request": 0, 
                    "requests": 294
                }
            ], 
            "delta_requests": 17, 
            "exceptions": 0, 
            "harakiri_count": 0, 
            "id": 1, 
            "last_spawn": 1350706679, 
            "pid": 11360, 
            "requests": 294, 
            "respawn_count": 38, 
            "rss": 226373632, 
            "running_time": 628263, 
            "signals": 0, 
            "status": "cheap", 
            "tx": 5178, 
            "vsz": 380694528
        }, 
        {
            "apps": [
                {
                    "chdir": "", 
                    "exceptions": 0, 
                    "id": 0, 
                    "modifier1": 7, 
                    "mountpoint": "", 
                    "requests": 0, 
                    "startup_time": 12
                }
            ], 
            "avg_rt": 46, 
            "cores": [
                {
                    "id": 0, 
                    "in_request": 0, 
                    "requests": 244
                }
            ], 
            "delta_requests": 7, 
            "exceptions": 0, 
            "harakiri_count": 0, 
            "id": 2, 
            "last_spawn": 1350752068, 
            "pid": 18535, 
            "requests": 244, 
            "respawn_count": 20, 
            "rss": 226373632, 
            "running_time": 9405, 
            "signals": 0, 
            "status": "idle", 
            "tx": 0, 
            "vsz": 380694528
        }
    ]
}

So I should have worker process with pid 18535 but there is no such process. My requests are queuing in backlog and cheaper busyness will finally spawn new worker.

I have those messages in my logs:

Oct 20 18:45:05 localhost app: [uwsgi-daemons] throttling "/usr/sbin/dropbear -b /etc/dropbear/banner -w -s -j -k -p 5054 -E -F -I 300" for 441 seconds
Oct 20 18:52:26 localhost app: [uwsgi-daemons] respawning "/usr/sbin/dropbear -b /etc/dropbear/banner -w -s -j -k -p 5054 -E -F -I 300"
Oct 20 18:52:26 localhost app: [18370] Oct 20 18:52:26 Failed listening on '5054': Error listening: Address already in use
Oct 20 18:52:26 localhost app: [18370] Oct 20 18:52:26 premature exit: No listening ports available.
Oct 20 18:52:27 localhost app: [uwsgi-daemons] respawning "/usr/sbin/dropbear -b /etc/dropbear/banner -w -s -j -k -p 5054 -E -F -I 300"
Oct 20 18:52:27 localhost app: [18372] Oct 20 18:52:27 Failed listening on '5054': Error listening: Address already in use
Oct 20 18:52:27 localhost app: [18372] Oct 20 18:52:27 premature exit: No listening ports available.
Oct 20 18:52:28 localhost app: [uwsgi-daemons] throttling "/usr/sbin/dropbear -b /etc/dropbear/banner -w -s -j -k -p 5054 -E -F -I 300" for 443 seconds
Oct 20 18:54:28 localhost app: DAMN ! worker 2 (pid: 16100) died, killed by signal 9 :( trying respawn ...
Oct 20 18:54:28 localhost app: Respawned uWSGI worker 2 (new pid: 18535)
Oct 20 18:54:28 localhost app: [uwsgi-daemons] respawning "/usr/sbin/dropbear -b /etc/dropbear/banner -w -s -j -k -p 5054 -E -F -I 300"
Oct 20 18:54:28 localhost app: [18537] Oct 20 18:54:28 Not backgrounding
Oct 20 18:56:52 localhost app: announcing my loyalty to the Emperor...
Oct 20 19:17:47 localhost app: DAMN ! worker 2 (pid: 18535) died, killed by signal 9 :( trying respawn ...
Oct 20 19:17:47 localhost app: Respawned uWSGI worker 2 (new pid: 19852)
Oct 20 19:17:48 localhost app: [uwsgi-daemons] respawning "/usr/sbin/dropbear -b /etc/dropbear/banner -w -s -j -k -p 5054 -E -F -I 300"
Oct 20 19:17:48 localhost app: [19854] Oct 20 19:17:48 Failed listening on '5054': Error listening: Address already in use
Oct 20 19:17:48 localhost app: [19854] Oct 20 19:17:48 premature exit: No listening ports available.

But dmesg tells me that my workers are being killed by OOM killer so maybe uWSGI fails to respaw them due to low memory in vassal cgroup? It looks that way since I have most OOM killer messages are doubled so it newly spawned worker is probably being killed just after starting (there are few ruby cron tasks attached and they probably eat all memory).

Oct 20 17:16:51 a178 kernel: [4442785.953510] Killed process 12610 (uwsgi) total-vm:371768kB, anon-rss:219004kB, file-rss:2060kB
Oct 20 17:16:51 a178 kernel: [4442786.042248] Killed process 12613 (uwsgi) total-vm:371768kB, anon-rss:219000kB, file-rss:232kB
Oct 20 17:37:06 a178 kernel: [4443998.129133] Killed process 12640 (uwsgi) total-vm:371768kB, anon-rss:219004kB, file-rss:2064kB
Oct 20 17:37:06 a178 kernel: [4443998.188021] Killed process 28489 (uwsgi) total-vm:371768kB, anon-rss:219000kB, file-rss:232kB
Oct 20 17:37:09 a178 kernel: [4444000.896209] Killed process 2002 (uwsgi) total-vm:371768kB, anon-rss:219000kB, file-rss:1864kB
Oct 20 18:08:41 a178 kernel: [4445888.114885] Killed process 2011 (uwsgi) total-vm:371768kB, anon-rss:219004kB, file-rss:2064kB
Oct 20 18:08:41 a178 kernel: [4445888.635278] Killed process 5014 (uwsgi) total-vm:371768kB, anon-rss:219000kB, file-rss:448kB
Oct 20 18:54:28 a178 kernel: [4448628.028641] Killed process 5017 (uwsgi) total-vm:371768kB, anon-rss:219004kB, file-rss:2064kB
Oct 20 18:54:28 a178 kernel: [4448628.145161] Killed process 21276 (uwsgi) total-vm:371768kB, anon-rss:219000kB, file-rss:232kB
Oct 20 19:17:47 a178 kernel: [4450024.092154] Killed process 22722 (uwsgi) total-vm:371768kB, anon-rss:219004kB, file-rss:2064kB
Oct 20 19:17:48 a178 kernel: [4450024.283838] Killed process 16056 (uwsgi) total-vm:371768kB, anon-rss:219000kB, file-rss:24kB

So the issue seems to be that uWSGI doesn't detect condition where there is no memory available to respawn worker that was killed by OOM, and new worker gets killed right away (?).

dropbear issues are probably separate thing, maybe uWSGI is trying to restart it while it shouldn't since the old process still runs fine

NONBLOCKing mode of connect_to_tcp breaks uwsgi_open_and_read

in f7399ef connect_to_tcp (a uwsgi_connect implementation) was changed to SOCK_NONBLOCK mode.

But in uwsgi_open_and_read in core/utils.c, when provided a http url, the read from the socket does not handle the EWOULDBLOCK case. Thus it would randomly fail.

ugreen + pycurl multi_curl: buffer overflow detected :-)

hi,

I'am using ugreen to wait answer from pycurl E_CALL_MULTI what looks something like this:

        while True:
            ret, num_handles = m.perform()
            if ret != pycurl.E_CALL_MULTI_PERFORM:
                break
        read, write, exc = m.fdset()
        if read:
            uwsgi.wait_fd_read(read[0], 1) 
            uwsgi.suspend()
        elif write:
            uwsgi.wait_fd_write(write[0], 1)
            uwsgi.suspend()

most times this code works ok, but as soon as I have more load (so more connections) uwsgi starts to crash with this backtrace

*** buffer overflow detected ***: uwsgi terminated
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(__fortify_fail+0x37)[0x7ff12eafa807]
/lib/x86_64-linux-gnu/libc.so.6(+0x109700)[0x7ff12eaf9700]
/lib/x86_64-linux-gnu/libc.so.6(+0x10a7be)[0x7ff12eafa7be]
/usr/lib/x86_64-linux-gnu/libcurl.so.4(curl_multi_fdset+0xa3)[0x7feec71a72f3]
/usr/local/lib/python2.7/dist-packages/pycurl.so(+0x711b)[0x7feec73d511b]
/usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x556f)[0x7ff12ee40a0f]
/usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855)[0x7ff12ee0b605]
/usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5420)[0x7ff12ee408c0]
/usr/lib/libpython2.7.so.1.0(+0x5b76b)[0x7ff12ee0a76b]
/usr/lib/libpython2.7.so.1.0(PyIter_Next+0xb)[0x7ff12ee6308b]
uwsgi(uwsgi_response_subhandler_wsgi+0x5a)[0x44cf1a]
uwsgi(uwsgi_request_wsgi+0x3df)[0x44bc0f]
uwsgi(u_green_request+0x19)[0x45e6e9]
/lib/x86_64-linux-gnu/libc.so.6(+0x48650)[0x7ff12ea38650]
======= Memory map: ========
00400000-0047d000 r-xp 00000000 08:01 37625315                           /usr/local/bin/uwsgi
0067c000-0067d000 r--p 0007c000 08:01 37625315                           /usr/local/bin/uwsgi
0067d000-00688000 rw-p 0007d000 08:01 37625315                           /usr/local/bin/uwsgi
00688000-0068c000 rw-p 00000000 00:00 0 
01e88000-02c03000 rw-p 00000000 00:00 0                                  [heap]
02c03000-0535e000 rw-p 00000000 00:00 0                                  [heap]
7feec2fcf000-7feec30d1000 rw-p 00000000 00:00 0 
7feec30d1000-7feec30da000 r-xp 00000000 08:01 38798499                   /lib/x86_64-linux-gnu/libcrypt-2.15.so
7feec30da000-7feec32da000 ---p 00009000 08:01 38798499                   /lib/x86_64-linux-gnu/libcrypt-2.15.so
7feec32da000-7feec32db000 r--p 00009000 08:01 38798499                   /lib/x86_64-linux-gnu/libcrypt-2.15.so
7feec32db000-7feec32dc000 rw-p 0000a000 08:01 38798499                   /lib/x86_64-linux-gnu/libcrypt-2.15.so
7feec32dc000-7feec330a000 rw-p 00000000 00:00 0 
7feec330a000-7feec33a8000 r-xp 00000000 08:01 37618266                   /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
7feec33a8000-7feec35a8000 ---p 0009e000 08:01 37618266                   /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
7feec35a8000-7feec35aa000 r--p 0009e000 08:01 37618266                   /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
7feec35aa000-7feec35ac000 rw-p 000a0000 08:01 37618266                   /usr/lib/x86_64-linux-gnu/libsqlite3.so.0.8.6
7feec35ac000-7feec35ad000 rw-p 00000000 00:00 0 
7feec35ad000-7feec35f2000 r-xp 00000000 08:01 37623540                   /usr/lib/x86_64-linux-gnu/libhx509.so.5.0.0
7feec35f2000-7feec37f2000 ---p 00045000 08:01 37623540                   /usr/lib/x86_64-linux-gnu/libhx509.so.5.0.0
7feec37f2000-7feec37f4000 r--p 00045000 08:01 37623540                   /usr/lib/x86_64-linux-gnu/libhx509.so.5.0.0
7feec37f4000-7feec37f6000 rw-p 00047000 08:01 37623540                   /usr/lib/x86_64-linux-gnu/libhx509.so.5.0.0
7feec37f6000-7feec37f7000 rw-p 00000000 00:00 0 
7feec37f7000-7feec3805000 r-xp 00000000 08:01 37623538                   /usr/lib/x86_64-linux-gnu/libheimbase.so.1.0.0
7feec3805000-7feec3a04000 ---p 0000e000 08:01 37623538                   /usr/lib/x86_64-linux-gnu/libheimbase.so.1.0.0
7feec3a04000-7feec3a05000 r--p 0000d000 08:01 37623538                   /usr/lib/x86_64-linux-gnu/libheimbase.so.1.0.0
7feec3a05000-7feec3a06000 rw-p 0000e000 08:01 37623538                   /usr/lib/x86_64-linux-gnu/libheimbase.so.1.0.0
7feec3a06000-7feec3a2e000 r-xp 00000000 08:01 37623568                   /usr/lib/x86_64-linux-gnu/libwind.so.0.0.0
7feec3a2e000-7feec3c2d000 ---p 00028000 08:01 37623568                   /usr/lib/x86_64-linux-gnu/libwind.so.0.0.0
7feec3c2d000-7feec3c2e000 r--p 00027000 08:01 37623568                   /usr/lib/x86_64-linux-gnu/libwind.so.0.0.0
7feec3c2e000-7feec3c2f000 rw-p 00028000 08:01 37623568                   /usr/lib/x86_64-linux-gnu/libwind.so.0.0.0
7feec3c2f000-7feec3c32000 r-xp 00000000 08:01 38798326                   /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7feec3c32000-7feec3e31000 ---p 00003000 08:01 38798326                   /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7feec3e31000-7feec3e32000 r--p 00002000 08:01 38798326                   /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7feec3e32000-7feec3e33000 rw-p 00003000 08:01 38798326                   /lib/x86_64-linux-gnu/libkeyutils.so.1.4
7feec3e33000-7feec3e36000 r-xp 00000000 08:01 38798324                   /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7feec3e36000-7feec4035000 ---p 00003000 08:01 38798324                   /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7feec4035000-7feec4036000 r--p 00002000 08:01 38798324                   /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7feec4036000-7feec4037000 rw-p 00003000 08:01 38798324                   /lib/x86_64-linux-gnu/libgpg-error.so.0.8.0
7feec4037000-7feec4048000 r-xp 00000000 08:01 37623552                   /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7feec4048000-7feec4247000 ---p 00011000 08:01 37623552                   /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7feec4247000-7feec4248000 r--p 00010000 08:01 37623552                   /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7feec4248000-7feec4249000 rw-p 00011000 08:01 37623552                   /usr/lib/x86_64-linux-gnu/libp11-kit.so.0.0.0
7feec4249000-7feec4259000 r-xp 00000000 08:01 37619106                   /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.12
7feec4259000-7feec4458000 ---p 00010000 08:01 37619106                   /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.12
7feec4458000-7feec4459000 r--p 0000f000 08:01 37619106                   /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.12
7feec4459000-7feec445a000 rw-p 00010000 08:01 37619106                   /usr/lib/x86_64-linux-gnu/libtasn1.so.3.1.12
7feec445a000-7feec446e000 r-xp 00000000 08:01 37623560                   /usr/lib/x86_64-linux-gnu/libroken.so.18.1.0
7feec446e000-7feec466d000 ---p 00014000 08:01 37623560                   /usr/lib/x86_64-linux-gnu/libroken.so.18.1.0
7feec466d000-7feec466e000 r--p 00013000 08:01 37623560                   /usr/lib/x86_64-linux-gnu/libroken.so.18.1.0
7feec466e000-7feec466f000 rw-p 00014000 08:01 37623560                   /usr/lib/x86_64-linux-gnu/libroken.so.18.1.0
7feec466f000-7feec46a0000 r-xp 00000000 08:01 37623537                   /usr/lib/x86_64-linux-gnu/libhcrypto.so.4.1.0
7feec46a0000-7feec48a0000 ---p 00031000 08:01 37623537                   /usr/lib/x86_64-linux-gnu/libhcrypto.so.4.1.0
7feec48a0000-7feec48a1000 r--p 00031000 08:01 37623537                   /usr/lib/x86_64-linux-gnu/libhcrypto.so.4.1.0
7feec48a1000-7feec48a2000 rw-p 00032000 08:01 37623537                   /usr/lib/x86_64-linux-gnu/libhcrypto.so.4.1.0
7feec48a2000-7feec48a3000 rw-p 00000000 00:00 0 
7feec48a3000-7feec493e000 r-xp 00000000 08:01 37623510                   /usr/lib/x86_64-linux-gnu/libasn1.so.8.0.0
7feec493e000-7feec4b3d000 ---p 0009b000 08:01 37623510                   /usr/lib/x86_64-linux-gnu/libasn1.so.8.0.0
7feec4b3d000-7feec4b3f000 r--p 0009a000 08:01 37623510                   /usr/lib/x86_64-linux-gnu/libasn1.so.8.0.0
7feec4b3f000-7feec4b43000 rw-p 0009c000 08:01 37623510                   /usr/lib/x86_64-linux-gnu/libasn1.so.8.0.0
7feec4b43000-7feec4bc3000 r-xp 00000000 08:01 37623543                   /usr/lib/x86_64-linux-gnu/libkrb5.so.26.0.0
7feec4bc3000-7feec4dc3000 ---p 00080000 08:01 37623543                   /usr/lib/x86_64-linux-gnu/libkrb5.so.26.0.0
7feec4dc3000-7feec4dc5000 r--p 00080000 08:01 37623543                   /usr/lib/x86_64-linux-gnu/libkrb5.so.26.0.0
7feec4dc5000-7feec4dc9000 rw-p 00082000 08:01 37623543                   /usr/lib/x86_64-linux-gnu/libkrb5.so.26.0.0
7feec4dc9000-7feec4dcf000 r-xp 00000000 08:01 37623539                   /usr/lib/x86_64-linux-gnu/libheimntlm.so.0.1.0
7feec4dcf000-7feec4fce000 ---p 00006000 08:01 37623539                   /usr/lib/x86_64-linux-gnu/libheimntlm.so.0.1.0
7feec4fce000-7feec4fcf000 r--p 00005000 08:01 37623539                   /usr/lib/x86_64-linux-gnu/libheimntlm.so.0.1.0
7feec4fcf000-7feec4fd0000 rw-p 00006000 08:01 37623539                   /usr/lib/x86_64-linux-gnu/libheimntlm.so.0.1.0
7feec4fd0000-7feec4fd7000 r-xp 00000000 08:01 37619119                   /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7feec4fd7000-7feec51d6000 ---p 00007000 08:01 37619119                   /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7feec51d6000-7feec51d7000 r--p 00006000 08:01 37619119                   /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7feec51d7000-7feec51d8000 rw-p 00007000 08:01 37619119                   /usr/lib/x86_64-linux-gnu/libkrb5support.so.0.1
7feec51d8000-7feec51db000 r-xp 00000000 08:01 38798313                   /lib/x86_64-linux-gnu/libcom_err.so.2.1
7feec51db000-7feec53da000 ---p 00003000 08:01 38798313                   /lib/x86_64-linux-gnu/libcom_err.so.2.1
7feec53da000-7feec53db000 r--p 00002000 08:01 38798313                   /lib/x86_64-linux-gnu/libcom_err.so.2.1
7feec53db000-7feec53dc000 rw-p 00003000 08:01 38798313                   /lib/x86_64-linux-gnu/libcom_err.so.2.1
7feec53dc000-7feec5401000 r-xp 00000000 08:01 37619112                   /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7feec5401000-7feec5601000 ---p 00025000 08:01 37619112                   /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7feec5601000-7feec5602000 r--p 00025000 08:01 37619112                   /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7feec5602000-7feec5603000 rw-p 00026000 08:01 37619112                   /usr/lib/x86_64-linux-gnu/libk5crypto.so.3.1
7feec5603000-7feec5604000 rw-p 00000000 00:00 0 
7feec5604000-7feec56c8000 r-xp 00000000 08:01 37619117                   /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7feec56c8000-7feec58c7000 ---p 000c4000 08:01 37619117                   /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7feec58c7000-7feec58d1000 r--p 000c3000 08:01 37619117                   /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7feec58d1000-7feec58d2000 rw-p 000cd000 08:01 37619117                   /usr/lib/x86_64-linux-gnu/libkrb5.so.3.3
7feec58d2000-7feec594c000 r-xp 00000000 08:01 38797330                   /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7feec594c000-7feec5b4c000 ---p 0007a000 08:01 38797330                   /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7feec5b4c000-7feec5b4d000 r--p 0007a000 08:01 38797330                   /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7feec5b4d000-7feec5b50000 rw-p 0007b000 08:01 38797330                   /lib/x86_64-linux-gnu/libgcrypt.so.11.7.0
7feec5b50000-7feec5c04000 r-xp 00000000 08:01 37619109                   /usr/lib/x86_64-linux-gnu/libgnutls.so.26.21.8
7feec5c04000-7feec5e04000 ---p 000b4000 08:01 37619109                   /usr/lib/x86_64-linux-gnu/libgnutls.so.26.21.8
7feec5e04000-7feec5e0a000 r--p 000b4000 08:01 37619109                   /usr/lib/x86_64-linux-gnu/libgnutls.so.26.21.8
7feec5e0a000-7feec5e0b000 rw-p 000ba000 08:01 37619109                   /usr/lib/x86_64-linux-gnu/libgnutls.so.26.21.8
7feec5e0b000-7feec5e0c000 rw-p 00000000 00:00 0 
7feec5e0c000-7feec5e46000 r-xp 00000000 08:01 37623534                   /usr/lib/x86_64-linux-gnu/libgssapi.so.3.0.0
7feec5e46000-7feec6046000 ---p 0003a000 08:01 37623534                   /usr/lib/x86_64-linux-gnu/libgssapi.so.3.0.0
7feec6046000-7feec6047000 r--p 0003a000 08:01 37623534                   /usr/lib/x86_64-linux-gnu/libgssapi.so.3.0.0
7feec6047000-7feec604a000 rw-p 0003b000 08:01 37623534                   /usr/lib/x86_64-linux-gnu/libgssapi.so.3.0.0
7feec604a000-7feec6064000 r-xp 00000000 08:01 37619132                   /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7feec6064000-7feec6263000 ---p 0001a000 08:01 37619132                   /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7feec6263000-7feec6264000 r--p 00019000 08:01 37619132                   /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7feec6264000-7feec6265000 rw-p 0001a000 08:01 37619132                   /usr/lib/x86_64-linux-gnu/libsasl2.so.2.0.25
7feec6265000-7feec627d000 r-xp 00000000 08:01 38798495                   /lib/x86_64-linux-gnu/libresolv-2.15.so
7feec627d000-7feec647d000 ---p 00018000 08:01 38798495                   /lib/x86_64-linux-gnu/libresolv-2.15.so
7feec647d000-7feec647e000 r--p 00018000 08:01 38798495                   /lib/x86_64-linux-gnu/libresolv-2.15.so
7feec647e000-7feec647f000 rw-p 00019000 08:01 38798495                   /lib/x86_64-linux-gnu/libresolv-2.15.so
7feec647f000-7feec6481000 rw-p 00000000 00:00 0 
7feec6481000-7feec6499000 r-xp 00000000 08:01 37623561                   /usr/lib/x86_64-linux-gnu/librtmp.so.0
7feec6499000-7feec6699000 ---p 00018000 08:01 37623561                   /usr/lib/x86_64-linux-gnu/librtmp.so.0
7feec6699000-7feec669a000 r--p 00018000 08:01 37623561                   /usr/lib/x86_64-linux-gnu/librtmp.so.0
7feec669a000-7feec669b000 rw-p 00019000 08:01 37623561                   /usr/lib/x86_64-linux-gnu/librtmp.so.0
7feec669b000-7feec66d6000 r-xp 00000000 08:01 37619115                   /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7feec66d6000-7feec68d6000 ---p 0003b000 08:01 37619115                   /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7feec68d6000-7feec68d7000 r--p 0003b000 08:01 37619115                   /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7feec68d7000-7feec68d9000 rw-p 0003c000 08:01 37619115                   /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2.2
7feec68d9000-7feec68e0000 r-xp 00000000 08:01 38798494                   /lib/x86_64-linux-gnu/librt-2.15.so
7feec68e0000-7feec6adf000 ---p 00007000 08:01 38798494                   /lib/x86_64-linux-gnu/librt-2.15.so
7feec6adf000-7feec6ae0000 r--p 00006000 08:01 38798494                   /lib/x86_64-linux-gnu/librt-2.15.so
7feec6ae0000-7feec6ae1000 rw-p 00007000 08:01 38798494                   /lib/x86_64-linux-gnu/librt-2.15.so
7feec6ae1000-7feec6b2c000 r-xp 00000000 08:01 37619136                   /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.1
7feec6b2c000-7feec6d2b000 ---p 0004b000 08:01 37619136                   /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.1
7feec6d2b000-7feec6d2d000 r--p 0004a000 08:01 37619136                   /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.1
7feec6d2d000-7feec6d2e000 rw-p 0004c000 08:01 37619136                   /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2.8.1
7feec6d2e000-7feec6d30000 rw-p 00000000 00:00 0 
7feec6d30000-7feec6d3d000 r-xp 00000000 08:01 37619140                   /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.1
7feec6d3d000-7feec6f3c000 ---p 0000d000 08:01 37619140                   /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.1
7feec6f3c000-7feec6f3d000 r--p 0000c000 08:01 37619140                   /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.1
7feec6f3d000-7feec6f3e000 rw-p 0000d000 08:01 37619140                   /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2.8.1
7feec6f3e000-7feec6f70000 r-xp 00000000 08:01 37623541                   /usr/lib/x86_64-linux-gnu/libidn.so.11.6.6
7feec6f70000-7feec716f000 ---p 00032000 08:01 37623541                   /usr/lib/x86_64-linux-gnu/libidn.so.11.6.6
7feec716f000-7feec7170000 r--p 00031000 08:01 37623541                   /usr/lib/x86_64-linux-gnu/libidn.so.11.6.6
7feec7170000-7feec7171000 rw-p 00032000 08:01 37623541                   /usr/lib/x86_64-linux-gnu/libidn.so.11.6.6
7feec7171000-7feec71cb000 r-xp 00000000 08:01 37623512                   /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7feec71cb000-7feec73cb000 ---p 0005a000 08:01 37623512                   /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7feec73cb000-7feec73cd000 r--p 0005a000 08:01 37623512                   /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7feec73cd000-7feec73ce000 rw-p 0005c000 08:01 37623512                   /usr/lib/x86_64-linux-gnu/libcurl.so.4.2.0
7feec73ce000-7feec73dd000 r-xp 00000000 08:01 37625382                   /usr/local/lib/python2.7/dist-packages/pycurl.so
7feec73dd000-7feec75dc000 ---p 0000f000 08:01 37625382                   /usr/local/lib/python2.7/dist-packages/pycurl.so
7feec75dc000-7feec75dd000 r--p 0000e000 08:01 37625382                   /usr/local/lib/python2.7/dist-packages/pycurl.so
7feec75dd000-7feec75df000 rw-p 0000f000 08:01 37625382                   /usr/local/lib/python2.7/dist-packages/pycurl.so
7feec75df000-7feec7661000 rw-p 00000000 00:00 0 
7feec7661000-7feec7664000 r-xp 00000000 08:01 37622669                   /usr/lib/python2.7/lib-dynload/_heapq.so
7feec7664000-7feec7863000 ---p 00003000 08:01 37622669                   /usr/lib/python2.7/lib-dynload/_heapq.so
7feec7863000-7feec7864000 r--p 00002000 08:01 37622669                   /usr/lib/python2.7/lib-dynload/_heapq.so
7feec7864000-7feec7866000 rw-p 00003000 08:01 37622669                   /usr/lib/python2.7/lib-dynload/_heapq.so
7feec7866000-7feec78e8000 rw-p 00000000 00:00 0 
7feec78e9000-7feec83f9000 rw-p 00000000 00:00 0 
7feec83f9000-7feecfc3f000 rw-s 00000000 00:04 264730989                  /dev/zero (deleted)
7feecfc3f000-7feed7237000 rw-s 00000000 00:04 264730988                  /dev/zero (deleted)
7feed7237000-7feed80f6000 rw-s 00000000 00:04 264730987                  /dev/zero (deleted)
7feed80f6000-7feeda0c2000 rw-s 00000000 00:04 264730986                  /dev/zero (deleted)
7feeda0c2000-7feeda0d2000 rw-s 00000000 00:04 264730985                  /dev/zero (deleted)
7feeda0d2000-7feee1918000 rw-s 00000000 00:04 264730984                  /dev/zero (deleted)
7feee1918000-7feee8f10000 rw-s 00000000 00:04 264730983                  /dev/zero (deleted)
7feee8f10000-7feee9dcf000 rw-s 00000000 00:04 264730982                  /dev/zero (deleted)
7feee9dcf000-7feeebd9b000 rw-s 00000000 00:04 264730981                  /dev/zero (deleted)
7feeebd9b000-7feeebdab000 rw-s 00000000 00:04 264730980                  /dev/zero (deleted)
7feeebdab000-7feef35f1000 rw-s 00000000 00:04 264730979                  /dev/zero (deleted)
7feef35f1000-7feefabe9000 rw-s 00000000 00:04 264730978                  /dev/zero (deleted)
7feefabe9000-7feefbaa8000 rw-s 00000000 00:04 264730977                  /dev/zero (deleted)
7feefbaa8000-7feefda74000 rw-s 00000000 00:04 264730976                  /dev/zero (deleted)
7feefda74000-7feefda84000 rw-s 00000000 00:04 264730975                  /dev/zero (deleted)
7feefda84000-7fef052ca000 rw-s 00000000 00:04 264730974                  /dev/zero (deleted)
7fef052ca000-7fef0c8c2000 rw-s 00000000 00:04 264730973                  /dev/zero (deleted)
7fef0c8c2000-7fef0d781000 rw-s 00000000 00:04 264730972                  /dev/zero (deleted)
7fef0d781000-7fef0f74d000 rw-s 00000000 00:04 264730971                  /dev/zero (deleted)
7fef0f74d000-7fef0f75d000 rw-s 00000000 00:04 264730970                  /dev/zero (deleted)
7fef0f75d000-7fef16fa3000 rw-s 00000000 00:04 264730969                  /dev/zero (deleted)
7fef16fa3000-7fef1e59b000 rw-s 00000000 00:04 264730968                  /dev/zero (deleted)
7fef1e59b000-7fef1f45a000 rw-s 00000000 00:04 264730967                  /dev/zero (deleted)
7fef1f45a000-7fef21426000 rw-s 00000000 00:04 264730966                  /dev/zero (deleted)
7fef21426000-7fef21436000 rw-s 00000000 00:04 264730965                  /dev/zero (deleted)
7fef21436000-7fef28c7c000 rw-s 00000000 00:04 264730964                  /dev/zero (deleted)
7fef28c7c000-7fef30274000 rw-s 00000000 00:04 264730963                  /dev/zero (deleted)
7fef30274000-7fef31133000 rw-s 00000000 00:04 264730962                  /dev/zero (deleted)
7fef31133000-7fef330ff000 rw-s 00000000 00:04 264730961                  /dev/zero (deleted)
7fef330ff000-7fef3310f000 rw-s 00000000 00:04 264730960                  /dev/zero (deleted)
7fef3310f000-7fef3a955000 rw-s 00000000 00:04 264730959                  /dev/zero (deleted)
7fef3a955000-7fef41f4d000 rw-s 00000000 00:04 264730958                  /dev/zero (deleted)
7fef41f4d000-7fef42e0c000 rw-s 00000000 00:04 264730957                  /dev/zero (deleted)
7fef42e0c000-7fef44dd8000 rw-s 00000000 00:04 264730956                  /dev/zero (deleted)
7fef44dd8000-7fef44de8000 rw-s 00000000 00:04 264730955                  /dev/zero (deleted)
7fef44de8000-7fef44dea000 rw-s 00000000 00:04 264730954                  /dev/zero (deleted)
7fef44dea000-7fef44fec000 r--p 00000000 08:01 37618640                   /usr/lib/locale/locale-archive

I tried uwsgi 1.2.x but situation is the same, I think that it started to happen when I moved to Python 2.7, but not sure.

I also tried greenlet instead of ugreen, but no success.

I'am using ubuntu 12.04 64 bit with python 2.7.3

any chance to see the fix in near future for this bug? or I should look for some other solution instead of uwsgi/pycurl combo?

SERVER_PORT does not work with zeromq

based on

#5

I think we have now the same issue in SERVER_PORT

STATS [TRACKERS]: allow write access to internal counters (if plugin allows)

(STUB: can/will expand as needed)

EXAMPLE USE CASE: updating the "request count" from an dumb-loop-style event loop... since app is always in control, and is not WSGI/HTTP related, all stats and similar are usually 0. as uWSGI continues expansion toward a "phenomenally useful general-purpose no-job-too-small-or-too-big application server" we need ways for non WSGI loops to insert their records for aggregation or .

basically, after i handle a message on the queue, i want to tell uwsgi that i just handle something -- then it can eventually apply timeouts properly (eg. harakiri) or automatic reloads, etc etc

first HEAD request is not handled properly

• I start my vassal and subscribe it to FastRouter
• I do HEAD request to my app (this will be the first request after app startup)
• app gets GET instead of HEAD
• every request after the first one works fine, app gets HEAD request

I've noticed this because I allow only GET and POST in my app, so HEAD should return 405.

~$ curl -I http://myapp.com/login
HTTP/1.1 200 OK
[...]

~$ curl -I http://myapp.com/login
HTTP/1.1 405 METHOD NOT ALLOWED
[...]

CACHE: named cache stores?

Proposal

various components can use the cache for sometimes non-related things (fastrouter keys, subscriptions keys, TLS share/balance, the APP itlsef, etc) would be nice to spawn (unlimited?) cache stores by name, then assign by name to discreet options.

UPDATE #92 (comment)

--cache2 name=X,max_items=n[,keysize=n,blocks=n,blocksize=n,hashsize=n,hash=S,store=S]

UPDATE #92 (comment)

// create...
struct uwsgi_cache *mycache = uwsgi_cache_create("name=pippo2,max_items=100");
// fetch...
struct uwsgi_cache *mycache = uwsgi_cache_by_name("pippo2");

Anti-dos policy in routers

As each router (http/fast/raw) maintains a very big list of statistics, we can improve things to support some anti-dos pattern.

The first feature could be a concurrent-requests limiter. Basically an hash table of ips maintains the number of active connections for the same ip.

The user can configure a "soft" policy, simply reporting a warning in the logs (that you can parse and eventually trigger an alarm) or a "hard" one, triggering a block of that ip.

The stats JSON blob should report the list of blocked ips too.

Sticky sessions in fastrouter/httprouter

When some kind of async database replica is in place, having the same client connecting to the same backend is the best solution to avoid incongruences in responses (mainly caused by sync gaps)

Sticky sessions are a common technique, but as always i want to give them to uWSGI users with little effort from a code-changing point of view.

My idea is setting a cookie in the fastrouter with the address of the backend (encrypted).

The encrypted blob should contain the ip address of the client, the backend node (obviously), a random IV and should support different kind of keys (from preshared keys to the same certificate used by the secured subscription system)

If the fastrouter find the cookie already set in the request it will simply decrypt it and use the supplied address.

Obviously an api function could be added (like uwsgi.sticky()) to generate the encrypted blob from the app only for the requests requiring it.

Incorrect work in emperor mode with php plugin

Vassal configuration:

[uwsgi]
plugins = php
socket = /tmp/testuser_example.com.sock
processes = 4
cheaper = 1
idle = 60
max-requests = 1000
vacuum = true
chmod-socket = 666
master = true
master-as-root = true
chroot = /fpanel/data/testuser
;disable-logging = true
uid = 2002
gid = 2002
harakiri = 30
harakiri-verbose = true
php-index = index.php

Nginx configuration:

location ~ .php$ {
    include uwsgi_params;
    uwsgi_modifier1 14;
    uwsgi_pass unix:///tmp/testuser_example.com.sock;
    uwsgi_param DOCUMENT_ROOT   /home/testuser/example.com/www;
}

Then I start uwsgi and it work as expected

# uwsgi --emperor /etc/uwsgi/vassals/
*** Starting uWSGI 1.2.6 (64bit) on [Sun Oct 21 14:03:57 2012] ***
compiled with version: 4.4.6 20120305 (Red Hat 4.4.6-4) on 02 October 2012 22:08:05
detected number of CPU cores: 2
current working directory: /home/dborisov
detected binary path: /usr/sbin/uwsgi
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) *** 
*** WARNING: you are running uWSGI without its master process manager ***
your memory page size is 4096 bytes
*** starting uWSGI Emperor ***
*** has_emperor mode detected (fd: 6) ***
[uWSGI] getting INI configuration from fpanel.ini
*** Starting uWSGI 1.2.6 (64bit) on [Sun Oct 21 14:03:58 2012] ***
compiled with version: 4.4.6 20120305 (Red Hat 4.4.6-4) on 02 October 2012 22:08:05
detected number of CPU cores: 2
current working directory: /etc/uwsgi/vassals
detected binary path: /usr/sbin/uwsgi
your memory page size is 4096 bytes
 *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers *** 
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 bound to UNIX address /tmp/testuser_example.com.sock fd 3
PHP 5.3.3 initialized
your server socket listen backlog is limited to 100 connections
*** Operational MODE: preforking ***
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 7126)
spawned uWSGI worker 1 (pid: 7127, cores: 1)

But when I change ini-file of vassal or simply touch it and uwsgi gracefilly reloads the config, I got an error

[emperor] reload the uwsgi instance fpanel.ini
Sun Oct 21 14:14:04 2012 - received message 1 from emperor
...gracefully killing workers...
binary reloading uWSGI...
chdir() to /etc/uwsgi/vassals
closing all non-uwsgi socket fds > 2 (max_fd = 1024)...
found fd 3 mapped to socket 0 (/tmp/testuser_example.com.sock)
running /usr/sbin/uwsgi
*** has_emperor mode detected (fd: 6) ***
[uWSGI] getting INI configuration from fpanel.ini
*** Starting uWSGI 1.2.6 (64bit) on [Sun Oct 21 14:14:04 2012] ***
compiled with version: 4.4.6 20120305 (Red Hat 4.4.6-4) on 02 October 2012 22:08:05
detected number of CPU cores: 2
current working directory: /etc/uwsgi/vassals
detected binary path: /usr/sbin/uwsgi
your memory page size is 4096 bytes
 *** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers *** 
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 inherited UNIX address /tmp/testuser_example.com.sock fd 3
PHP 5.3.3 initialized
your server socket listen backlog is limited to 100 connections
*** Operational MODE: preforking ***
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
gracefully (RE)spawned uWSGI master process (pid: 7126)
spawned uWSGI worker 1 (pid: 7139, cores: 1)
workers have been inactive for more than 60 seconds (1350814506-1350814445)
cheap mode enabled: waiting for socket connection...
Respawned uWSGI worker 1 (new pid: 7141)
realpath() of /home/testuser/example.com/www failed: No such file or directory [utils.c line 4302]
[pid: 7141|app: -1|req: -1/1] 192.168.1.102 () {44 vars in 813 bytes} [Sun Oct 21 14:15:16 2012] GET /index.php => generated 0 bytes in 0 msecs (HTTP/1.1 500) 0 headers in 0 bytes (0 switches on core 0)
announcing my loyalty to the Emperor...
*** vassal fpanel.ini is now loyal ***

Uwsgi thinks that this path is not in chroot, because I change DOCUMENT_ROOT var in nginx configuration. But without it any site in chroot will be broken.
Then I kill emperor process and start it again and it works perfectly until the next reload.
I think it is a bug.

PS: Without using chroot this problem is not presented. uwsgi 1.2.6. Sorry for my english

plugins declaration, blank space error

Using the 'plugins' option as a comma separated list, now you have to pay attention to blank spaces.

For example:
plugins = python273, router_rewrite
is incorrect, cause looks for " router_rewrite" starting with the blank space.

You should write
plugins = python273,router_rewrite without any blank space after the comma.

I think this behavior is not really user friendly and both declarations should be valid.

uwsgidecorators import in uwsgi configuration

I need uwsgidecorators in all my django projects.
It would be great to have a variable to import the module. For example:

uwsgidecorators = 1

or

import = uwsgidecorators

Odd timezone in worker during request

When I send SIGUSR2 to worker during request or harakiri happens, worker logs with timezone -0600, and my default timezone is set to UTC. Master logs in UTC, and workers log in UTC when not handling request and SIGUSR2 is sent. Changing default timezone doesn't have effect on "-0600" timezone. I'm testing with version 1.2.3-debian.

I've set log-date to %%Y-%%m-%%dT%%H:%%M:%%S%%z to log the timezone, and here are the example logs:

• idle worker:

2012-12-13T09:30:20+0000 - *** backtrace of 14457 ***
2012-12-13T09:30:20+0000 - uwsgi_python(uwsgi_backtrace+0x25) [0x430a55]
2012-12-13T09:30:20+0000 - uwsgi_python(what_i_am_doing+0x30) [0x430b90]
2012-12-13T09:30:20+0000 - /lib/x86_64-linux-gnu/libc.so.6(+0x364a0) [0x7fa0820784a0]
2012-12-13T09:30:20+0000 - /lib/x86_64-linux-gnu/libc.so.6(epoll_wait+0x33) [0x7fa082136353]
2012-12-13T09:30:20+0000 - uwsgi_python(event_queue_wait+0x23) [0x428813]
2012-12-13T09:30:20+0000 - uwsgi_python(wsgi_req_accept+0x3c) [0x40de3c]
2012-12-13T09:30:20+0000 - uwsgi_python(simple_loop+0x106) [0x42be06]
2012-12-13T09:30:20+0000 - uwsgi_python(uwsgi_ignition+0x18a) [0x430faa]
2012-12-13T09:30:20+0000 - uwsgi_python(uwsgi_start+0x2c36) [0x434586]
2012-12-13T09:30:20+0000 - uwsgi_python(main+0xed4) [0x40d6e4]
2012-12-13T09:30:20+0000 - /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fa08206376d]
2012-12-13T09:30:20+0000 - uwsgi_python() [0x40d781]
2012-12-13T09:30:20+0000 - *** end of backtrace ***

• the same worker handling request:

2012-12-13T03:49:15-0600 - *** backtrace of 14457 ***
2012-12-13T03:49:15-0600 - uwsgi_python(uwsgi_backtrace+0x25) [0x430a55]
2012-12-13T03:49:15-0600 - uwsgi_python(what_i_am_doing+0x30) [0x430b90]
2012-12-13T03:49:15-0600 - /lib/x86_64-linux-gnu/libc.so.6(+0x364a0) [0x7fa0820784a0]
2012-12-13T03:49:15-0600 - /lib/x86_64-linux-gnu/libc.so.6(__select+0x33) [0x7fa08212f023]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(+0xa3e0d) [0x7fa07e427e0d]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x53a5) [0x7fa07e415845]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fa07e3e0605]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5420) [0x7fa07e4158c0]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fa07e4165eb]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fa07e3e0605]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(+0x5c7bd) [0x7fa07e3e07bd]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fa07e4c4e83]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(+0x1251cf) [0x7fa07e4a91cf]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fa07e4c4e83]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(+0x13136c) [0x7fa07e4b536c]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fa07e4c4e83]
2012-12-13T03:49:15-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47) [0x7fa07e4c57d7]
2012-12-13T03:49:15-0600 - /usr/lib/uwsgi/plugins/python_plugin.so(python_call+0x24) [0x7fa07e890004]
2012-12-13T03:49:15-0600 - /usr/lib/uwsgi/plugins/python_plugin.so(uwsgi_request_wsgi+0x119) [0x7fa07e8922f9]
2012-12-13T03:49:15-0600 - uwsgi_python(wsgi_req_recv+0x7f) [0x40dd5f]
2012-12-13T03:49:15-0600 - uwsgi_python(simple_loop+0x112) [0x42be12]
2012-12-13T03:49:15-0600 - uwsgi_python(uwsgi_ignition+0x18a) [0x430faa]
2012-12-13T03:49:15-0600 - uwsgi_python(uwsgi_start+0x2c36) [0x434586]
2012-12-13T03:49:15-0600 - uwsgi_python(main+0xed4) [0x40d6e4]
2012-12-13T03:49:15-0600 - /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fa08206376d]
2012-12-13T03:49:15-0600 - uwsgi_python() [0x40d781]
2012-12-13T03:49:15-0600 - *** end of backtrace ***
2012-12-13T03:49:15-0600 - SIGUSR2: --- uWSGI worker 2 core 0 (pid: 14457) is managing request /tabs/i10l/ since Thu Dec 13 03:49:12 2012 ---

• another worker during harakiri (sleep in request):

2012-12-13T10:17:35+0000 - *** HARAKIRI ON WORKER 1 (pid: 15286) ***
2012-12-13T10:17:35+0000 - HARAKIRI: -- syscall> 23 0x0 0x0 0x0 0x0 0x7fff7ab28040 0x7fff7ab27de0 0x7fff7ab28008 0x7fa08212f023
2012-12-13T10:17:35+0000 - HARAKIRI: -- wchan> poll_schedule_timeout
2012-12-13T04:17:35-0600 - *** backtrace of 15286 ***
2012-12-13T04:17:35-0600 - uwsgi_python(uwsgi_backtrace+0x25) [0x430a55]
2012-12-13T04:17:35-0600 - uwsgi_python(what_i_am_doing+0x30) [0x430b90]
2012-12-13T04:17:35-0600 - /lib/x86_64-linux-gnu/libc.so.6(+0x364a0) [0x7fa0820784a0]
2012-12-13T04:17:35-0600 - /lib/x86_64-linux-gnu/libc.so.6(__select+0x33) [0x7fa08212f023]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(+0xa3e0d) [0x7fa07e427e0d]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x53a5) [0x7fa07e415845]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fa07e3e0605]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x5420) [0x7fa07e4158c0]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x614b) [0x7fa07e4165eb]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x855) [0x7fa07e3e0605]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(+0x5c7bd) [0x7fa07e3e07bd]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fa07e4c4e83]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(+0x1251cf) [0x7fa07e4a91cf]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fa07e4c4e83]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(+0x13136c) [0x7fa07e4b536c]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(PyObject_Call+0x53) [0x7fa07e4c4e83]
2012-12-13T04:17:35-0600 - /usr/lib/libpython2.7.so.1.0(PyEval_CallObjectWithKeywords+0x47) [0x7fa07e4c57d7]
2012-12-13T04:17:35-0600 - /usr/lib/uwsgi/plugins/python_plugin.so(python_call+0x24) [0x7fa07e890004]
2012-12-13T04:17:35-0600 - /usr/lib/uwsgi/plugins/python_plugin.so(uwsgi_request_wsgi+0x119) [0x7fa07e8922f9]
2012-12-13T04:17:35-0600 - uwsgi_python(wsgi_req_recv+0x7f) [0x40dd5f]
2012-12-13T04:17:35-0600 - uwsgi_python(simple_loop+0x112) [0x42be12]
2012-12-13T04:17:35-0600 - uwsgi_python(uwsgi_ignition+0x18a) [0x430faa]
2012-12-13T04:17:35-0600 - uwsgi_python(uwsgi_start+0x2c36) [0x434586]
2012-12-13T04:17:35-0600 - uwsgi_python(main+0xed4) [0x40d6e4]
2012-12-13T04:17:35-0600 - /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed) [0x7fa08206376d]
2012-12-13T04:17:35-0600 - uwsgi_python() [0x40d781]
2012-12-13T04:17:35-0600 - *** end of backtrace ***
2012-12-13T04:17:35-0600 - HARAKIRI: --- uWSGI worker 1 core 0 (pid: 15286) WAS managing request /tabs/i10l/ since Thu Dec 13 04:17:14 2012 ---
2012-12-13T10:17:37+0000 - DAMN ! worker 1 (pid: 15286) died, killed by signal 9 :( trying respawn ...
2012-12-13T10:17:37+0000 - Respawned uWSGI worker 1 (new pid: 6874)

uwsgi running by root, permission denied to read /proc/self/auxv

issue explained:
https://bugs.launchpad.net/libjpeg-turbo/+bug/1020872

Async mode break signals?

When async core > 1, workers can not get signal...

Possible issue with plugin loading

Steps to reproduce:

1. Use this buildconf to compile uWSGI

[uwsgi]
debug = true
main_plugin = 
plugins = dummy,corerouter
inherit = base

1. run this command

./uwsgi --master --plugin 0:dummy -s :0

gdb show this:

max space for custom process name = 1755
[uwsgi-logger] registered "syslog"
[uwsgi-logger] registered "rsyslog"
[uwsgi-logger] registered "socket"
[uwsgi-logger] registered "redislog"
[uwsgi-logger] registered "mongodblog"
[uwsgi-logger] registered "file"
executable name: /home/lukasz.mierzwa/uwsgi_test/uwsgi
[uWSGI DEBUG] loading plugin 0:dummy

Program received signal SIGSEGV, Segmentation fault.
uwsgi_load_plugin (modifier=-1, plugin=0x7021f9 "corerouter", has_option=0x0) at core/plugins.c:72
72      void *uwsgi_load_plugin(int modifier, char *plugin, char *has_option) {
(gdb) where
#0  uwsgi_load_plugin (modifier=-1, plugin=0x7021f9 "corerouter", has_option=0x0) at core/plugins.c:72
#1  0x0000000000436358 in uwsgi_plugin_parse_section (filename=<value optimized out>) at core/plugins.c:17
#2  0x0000000000435e80 in uwsgi_load_plugin (modifier=<value optimized out>, plugin=<value optimized out>, has_option=0x0) at core/plugins.c:140
#3  0x0000000000436358 in uwsgi_plugin_parse_section (filename=<value optimized out>) at core/plugins.c:17
#4  0x0000000000435e80 in uwsgi_load_plugin (modifier=<value optimized out>, plugin=<value optimized out>, has_option=0x0) at core/plugins.c:140
#5  0x0000000000436358 in uwsgi_plugin_parse_section (filename=<value optimized out>) at core/plugins.c:17
#6  0x0000000000435e80 in uwsgi_load_plugin (modifier=<value optimized out>, plugin=<value optimized out>, has_option=0x0) at core/plugins.c:140
#7  0x0000000000436358 in uwsgi_plugin_parse_section (filename=<value optimized out>) at core/plugins.c:17
[same thing repeating]
#7486 0x0000000000435e80 in uwsgi_load_plugin (modifier=<value optimized out>, plugin=<value optimized out>, has_option=0x0) at core/plugins.c:140
#7487 0x000000000043fe26 in uwsgi_opt_load_plugin (opt=<value optimized out>, value=<value optimized out>, none=<value optimized out>) at core/uwsgi.c:3475
#7488 0x000000000041a17a in add_exported_option (key=0x457d4e "plugin", value=0x682320 "0:dummy", configured=0) at core/utils.c:2165
#7489 0x00000000004352eb in uwsgi_commandline_config () at core/init.c:180
#7490 0x0000000000444264 in main (argc=6, argv=0x7fffffffe6b8, envp=<value optimized out>) at core/uwsgi.c:1798

EDIT:

I've added corerouter to plugins list in my buildconf only because for some reason with command line from step 2 uWSGI was trying to load it:

[uWSGI DEBUG] loading plugin 0:dummy
open("./corerouter_plugin.so"): No such file or directory [core/utils.c line 4799]
!!! UNABLE to load uWSGI plugin: ./corerouter_plugin.so: cannot open shared object file: No such file or directory !!!
optind:6 argc:6

Now I see that corerouter is included in embedded_plugins list, so I have no idea why was it trying to load it as a separate plugin. I think that segfault is caused by me compiling corerouter both as embedded and separate plugin, if so than possible issue is the UNABLE to load uWSGI plugin: ./corerouter_plugin.so error. Anyway please take a look and decide if this is real issue

REMOTE_ADDR does not work with zeromq

Hi,

I have now a working mongrel => uwsgi setup. thanks unbit in the irc.

I have still a issue about the REMOTE_ADDR.

When I take a look at the REMOTE_ADDR I get

}0:,

which couldn't be the remote address due to the fact that HTTP_X_FORWARDED_FOR shows the right address.

You can take a look about the output of the env variables here

http://external.none.at:6767/print_env.cgi

uwsgi call & mongrel2 conf http://pastebin.com/wunZZB3n

looks like the parsing of the tnetstring is the problem

Strace output at the end of http://pastebin.com/wunZZB3n

can not build alarm_xmpp

error:gloox.cc.c not found.

smart-attach-daemon errors when the pidfile is not available

With uwsgi-1.4 branch as 092767c with this configuration:

smart-attach-daemon = /proc/unbit/tmp/memcached.pid memcached -s %d/memcached.socket -d -P /proc/unbit/tmp/memcached.pid

if the pid file does not exists it loops here:

[uwsgi-daemons] found changed pidfile for "memcached -s %d/memcached.socket -d -P /proc/unbit/tmp/memcached.pid -u toret" (old_pid: 0 new_pid: -1)
open("/proc/unbit/tmp/memcached.pid"): No such file or directory [core/daemons.c line 96]
open("/proc/unbit/tmp/memcached.pid"): No such file or directory [core/daemons.c line 96]

Support one of stud's methods of passing the client IP to the backend

It would be great to support one of stud's methods of passing the client IP to the backend:

      --write-ip             Write 1 octet with the IP family followed by the IP
                             address in 4 (IPv4) or 16 (IPv6) octets little-endian
                             to backend before the actual data
                             (Default: off)
      --write-proxy          Write HaProxy's PROXY (IPv4 or IPv6) protocol line
                             before actual data
                             (Default: off)

Segfaults with --offload and 1.4 final

With 1.4 final I've notice a lot of offload related stuff, so I wanted to give it a try, sadly I'm getting segfaults:

Nov 12 12:31:56 localhost onion: *** glibc detected *** [onion@2517] uWSGI worker 1: double free or corruption (fasttop): 0x0000000001dc6d30 ***
Nov 12 12:31:56 localhost onion: ======= Backtrace: =========
Nov 12 12:31:56 localhost onion: /lib/libc.so.6(+0x78b06)[0x7fb4bfa02b06]
Nov 12 12:31:56 localhost onion: /lib/libc.so.6(cfree+0x73)[0x7fb4bfa093d3]
Nov 12 12:31:56 localhost onion: [onion@2517] uWSGI worker 1[0x43337a]
Nov 12 12:31:56 localhost onion: [onion@2517] uWSGI worker 1[0x4151d3]
Nov 12 12:31:56 localhost onion: /lib/libpthread.so.0(+0x69ca)[0x7fb4c1e599ca]
Nov 12 12:31:56 localhost onion: /lib/libc.so.6(clone+0x6d)[0x7fb4bfa7316d]

This happens with offload-threads = 128 and ab run using:

ab -r -c 100 -n 10000000 http://localhost:8080/static/js/jquery.qtip.js

It doesn't happen instantly, I need to wait few minutes to trigger it, I will try to narrow it down.

Installing 1.1.1

ImportError: No module named uwsgiconfig

 [exec] Downloading/unpacking uWSGI>=1.1
 [exec]   Downloading uWSGI-1.1.1.tar.gz
 [exec]   Running setup.py egg_info for package uWSGI
 [exec]     Traceback (most recent call last):
 [exec]       File "<string>", line 14, in <module>
 [exec]       File "/build/uWSGI/setup.py", line 3, in <module>
 [exec]         import uwsgiconfig as uc
 [exec]     ImportError: No module named uwsgiconfig
 [exec]     Complete output from command python setup.py egg_info:
 [exec]     Traceback (most recent call last):
 [exec] 
 [exec]   File "<string>", line 14, in <module>
 [exec] 
 [exec]   File "/build/uWSGI/setup.py", line 3, in <module>
 [exec] 
 [exec]     import uwsgiconfig as uc
 [exec] 
 [exec] ImportError: No module named uwsgiconfig
 [exec] 
 [exec] ----------------------------------------
 [exec] Command python setup.py egg_info failed with error code 1