bitly / simplehttp Goto Github PK
View Code? Open in Web Editor NEWa family of libraries and daemons for building scalable web infrastructure
Home Page: http://github.com/bitly/simplehttp
License: MIT License
a family of libraries and daemons for building scalable web infrastructure
Home Page: http://github.com/bitly/simplehttp
License: MIT License
n/t
because we don't currently explicitly set evhttp_connection_set_closecb
there is a bug where asynchronous handlers could dereference invalid request or connection objects when the remote hangs up before the request was finished.
simplehttp
could facilitate cleanly handling both success and failure cases.
they should also support a simple csv like txt format
I'm using libevent 2.0.12.
After "cd simplehttp ; make && make install", I switched to simplequeue to make install, but gcc began to complain, saying "../simplehttp/libsimplehttp.a(simplehttp.o): In function simplehttp_init': /home/lamusicoscos/Base/bitly-simplehttp-10406dc/simplehttp/simplehttp.c:152: undefined reference to
current_base'
collect2: ld returned 1 exit status make: *** [simplequeue] Error 1".
But being with libevent 1.4, everything's OK. I'm just wondering why it happened and how can I make it work with libevent 2.0.12+.
Hi all,
I've been trying to follow the tutorial found here: http://micha.gd/realtimestream/ .
I am using libevent-1.4.14 . I get to the point in the tutorial where I issue this command:
%> ps_to_http --destination-get-url="http://127.0.0.1:8081/put?data=%s" --pubsub-url="http://127.0.0.1:8080/sub?multipart=0"
It works fine for a few minutes [pub/ sub/ requests ] but the it dies with this error:
ERROR: request failed
Any ideas what I might be doing wrong?
add basic auth support to pubsubclient
and add command line options to ps_to_http
and ps_to_file
changed install -D to use the more compatable -d
from valgrind, during the included simpleleveldb tests:
==25024== 24 bytes in 1 blocks are indirectly lost in loss record 4 of 22
==25024== at 0x4C21430: calloc (vg_replace_malloc.c:418)
==25024== by 0x504341F: array_list_new (arraylist.c:31)
==25024== by 0x50439B4: json_object_new_array (json_object.c:491)
==25024== by 0x404FCE: fwmatch_cb (simpleleveldb.c:463)
==25024== by 0x40606E: generic_request_handler (simplehttp.c:132)
==25024== by 0x4E33F21: evhttp_get_body (http.c:1592)
==25024== by 0x4E2D6F7: event_base_loop (event.c:395)
==25024== by 0x40665A: simplehttp_run (simplehttp.c:300)
==25024== by 0x406685: simplehttp_main (simplehttp.c:308)
==25024== by 0x40453C: main (simpleleveldb.c:922)
according to http.c:1886
in libevent source we should be able to set outbound 'Connection: close' headers in order to circumvent the HTTP 1.1 related leaks we encounter.
json format responses of some simpleleveldb endpoints is not the way we'd prefer. We'd like to identify what currently uses the current structure and transition them to a new structure that puts all response data somewhere in the value of the "data" member of the root, and include useful keys and metadata.
got a err when compile simpleleveldb on ubuntu 12.04
cc -I. -I../simplehttp/.. -I/usr/local/include -I/usr/local/lib//include -Wall -g -O2 -MM -MT simpleleveldb -MF simpleleveldb.deps simpleleveldb.c str_list_set.c
cc -I. -I../simplehttp/.. -I/usr/local/include -I/usr/local/lib//include -Wall -g -O2 -o simpleleveldb simpleleveldb.c str_list_set.c -L. -L../simplehttp -L/usr/local/lib -L/usr/local/lib -L/usr/local/lib//lib -levent -ljson -lsimplehttp -lleveldb -lm -lstdc++ -lsnappy -lpthread
/tmp/cccCgPcZ.o: In functionfinalize_request': /temp/project/simplehttp/simpleleveldb/simpleleveldb.c:103: undefined reference to
json_object_new_string'
I have a pubsub instance not doing much and using 28135 connections, 573 of which are TCP and 27496 of which are (according to lsof):
pubsub 3392 root *462u sock 0,5 0t0 36281617 can't identify protocol
where the 462 is incrementing.
I think this is related to the HTTP/1.1 issue; when the HTTP/1.1 client goes away, that fd is never cleaned up. #44
pysimplehttp has been updated and BaseReader has been added in
queuereader lib should handle heartbeat messages and not pass them to the callback
simplehttp async connection pool resources are never all freed, even if there is no connection activity for a long time. This can contribute to memory fragmentation and pin large heap regions, so even if the application using simplehttp async connections naturally frees all its other resources, it still has an RSS near its maximum.
Maybe async connection pool resources should be freed if all connections they are related to are done and gone for more than a few seconds.
maybe it should check all input before writing to the db?
maybe it should return an indication of how many records it wrote successfully?
what the title says
http = httpclient.HTTPClient()
url = "http://localhost:13731/get"
params = urllib.urlencode({'key':'f00'})
url += '?' + params
resp = http.fetch(url)
print resp.headers.get('Content-Type', "No content type!") # prints: text/html; charset=ISO-8859-1
Should be application/json
or text/javascript
Currently ps_to_file will segfault if it does not have write permission on the file specified or if that directory does not exist.
test
Update simplequeue to use page based allocation. This will allow linux to reclaim free'd memory regardless of heap position.
Add a multiput call (mput) to simplequeue. Allows posting multiple items at once to the queue separated by either a default separator of newline or a user specified separator. The separator can be one or more characters.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.