quantmind / pulsar Goto Github PK
View Code? Open in Web Editor NEWEvent driven concurrent framework for Python
License: BSD 3-Clause "New" or "Revised" License
Event driven concurrent framework for Python
License: BSD 3-Clause "New" or "Revised" License
The websocket application works for Chrome and Firefox.
A new web socket middleware is needed to cover most web browsers.
The best solution seems the SocketIO protocol.
I'm getting the following error:
2013-01-13 07:05:44 [p=1516,t=140735145521536] [ERROR] [pulsar.iostream] Could not parse data Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pulsar/async/iostream.py", line 762, in request_data parsed_data, buffer = self.parser.decode(buffer) File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pulsar/apps/wsgi/server.py", line 327, in decode self.expect_continue() File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pulsar/apps/wsgi/server.py", line 345, in expect_continue if headers is not None and headers.get('Expect') == '100-continue': AttributeError: 'list' object has no attribute 'get'
The cause is line https://github.com/quantmind/pulsar/blob/0.4/pulsar/apps/wsgi/server.py#L345. (At least, the AttributeError, I'm not sure if the "Could not parse data) is or not.
Currently I use this
def is_stack_trace(trace):
if isinstance(trace,tuple) and len(trace) == 3:
return True
return False
But should I use
inspect.istraceback(trace)
Tests and docs needed.
The MultiDeferred
class has a recursive behaviour on Mapping
, list
, sets
and generators
.
Should this made optional, with default false
?
It caused a bug in the test suite returning a list of errors.
Changes of internals to accommodate pep 3156
The method needs to be safe in case the argument is a generator which raise an error during iteration.
Since the major refactoring the shell application does not work. Need fixing
The text output is random due to the asynchronous test suite.
Need tests for pause
, resume
, pause_writing
and resume_writing
for stream socket transports. These are PEP 3156 methods.
The exit
command does not work on the asynchronous pulsar shell. It hangs the whole execution.
CPU bound workers can take a long time to process a request.
In this case we should have a dedicated thread in the worker process which communicate with the arbiter.
This is to be implemented for workers on a separate process.
The function accept a config dictionary and return a stream object to write information without using the actor logger.
It is implemented in the pulsar.async.actor
module.
All print function calls should be removed from the code.
Generators can cause memory leaks, visible using the --show_leaks
test option.
In particular the async.actor
tests cause leaks the the mailbox _responde
generator
Currently to check test coverage we need to run tests in thread mode so that coverage can be collected.
We need a test coverage plugin using the coverage API in the same line as the profile test plugin.
Function Settings are not working in multiprocess mode.
They are already written but need to be included.
Fix them
the testApp
test in the pshell
example needs fixing.
Modify the current deferred implementation so that you can add error backs as well as callbacks. Exactly the same way as twisted.
Sometimes the test in taskqueue.local.test_check_next_run
fails. The test is currently partially switched off.
Some tasks are quite big, expecially periodic tasks which perform massive updates on data.
An idea could be to have periodic tasks which send tasks requests.
A map reduce framework.
At the moment task-queue workers process a task at a time in a synchronous way.
Propose to:
max_concurrent_tasks
configuration parameter.Write up a doc page documenting how best to contribute to pulsar development.
The page is in docs/source/contributing.rst
To add an exclude
option when running tests:
python runtests.py --exclude apps calculator
In the example the tests with labels apps
and calculator
won't run.
Setup configurable 404 and 500 error pages
The method is needed when flushing a task backend, i.e. removing all tasks and clearing the task queue.
Ctrl-C does not stop a running test case.
The command options of plugins which are not installed are available in the command line. They shouldn't.
The may_pool_task
method in the TaskBackend
periodically call itself every 1 second, to pull new tasks. Is this the best way?
Not needed, use
isinstance(obj, Deferred)
instead
When errors occurs, looping callbaks are not called and they keep getting added to the event loop slowing down the server response.
The taskqueue.redis
tests are causing issues with the test suite.
Some of them can cause the whole test suite to hang.
The pulsar.spawn
method should return a deferred rather than a proxy
. The deferred will receive the callback once the actor is fully operational. That is to say, the actor has registered its mailbox address with the ActorProxyMonitor
.
When returning a Transfer-Encoding
as chunked
there is an error in the client. This needs fixing.
Actor to have a thread pool implementation so that CPU type workers can have more than one thread to work on.
When a new monitor
is created with the arbiter
has already started, the other monitors are not linked with it.
This is the proposal:
not needed
Deferred returning deferred needs to have test put in place. Their behaviour is quite complex and difficult to follow.
Add functionality to profile a task run using the python profiler.
Importing modules using a pattern such as tests
or test_*
is working but all modules are imported first.
Modules not matching the pattern should not be imported.
Create postgresql asynchronous binding using psycopg2
asynchronous interface. There is a twisted project that does it for twisted
.
It does not shut down properly as several workers form test cases hangs.
This is also true when using concurrent testing (backlog 2
for example)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.