Giter Site home page Giter Site logo

quantmind / pulsar Goto Github PK

View Code? Open in Web Editor NEW
1.9K 92.0 162.0 46.52 MB

Event driven concurrent framework for Python

License: BSD 3-Clause "New" or "Revised" License

Python 99.20% HTML 0.09% JavaScript 0.01% CSS 0.11% C 0.12% Shell 0.19% Makefile 0.20% Batchfile 0.07%
python asyncio wsgi http test-framework multiprocessing rpc http-server greenlet

pulsar's People

Contributors

artemmus avatar bright-pan avatar cyberj avatar davidjfelix avatar dhutty avatar etataurov avatar eventh avatar fabiocerqueira avatar felippemr avatar keis avatar lsbardel avatar pvanderlinden avatar remcoder avatar robgil avatar ryankung avatar s-sokolko avatar sekrause avatar wilddom avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pulsar's Issues

SocketIO middleware

The websocket application works for Chrome and Firefox.
A new web socket middleware is needed to cover most web browsers.
The best solution seems the SocketIO protocol.

Problem with RPC example on Python3.3.

I'm getting the following error:

2013-01-13 07:05:44 [p=1516,t=140735145521536] [ERROR] [pulsar.iostream] Could not parse data Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pulsar/async/iostream.py", line 762, in request_data parsed_data, buffer = self.parser.decode(buffer) File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pulsar/apps/wsgi/server.py", line 327, in decode self.expect_continue() File "/Library/Frameworks/Python.framework/Versions/3.3/lib/python3.3/site-packages/pulsar/apps/wsgi/server.py", line 345, in expect_continue if headers is not None and headers.get('Expect') == '100-continue': AttributeError: 'list' object has no attribute 'get'

The cause is line https://github.com/quantmind/pulsar/blob/0.4/pulsar/apps/wsgi/server.py#L345. (At least, the AttributeError, I'm not sure if the "Could not parse data) is or not.

better method for is_trace_back

Currently I use this

def is_stack_trace(trace):
    if isinstance(trace,tuple) and len(trace) == 3:
        return True
    return False

But should I use

inspect.istraceback(trace)

recursive behaviour in MultiDeferred

The MultiDeferred class has a recursive behaviour on Mapping, list, sets and generators.
Should this made optional, with default false?
It caused a bug in the test suite returning a list of errors.

pep 3156

Changes of internals to accommodate pep 3156

Worker thread

CPU bound workers can take a long time to process a request.
In this case we should have a dedicated thread in the worker process which communicate with the arbiter.
This is to be implemented for workers on a separate process.

Better get_stream function

The function accept a config dictionary and return a stream object to write information without using the actor logger.
It is implemented in the pulsar.async.actor module.
All print function calls should be removed from the code.

Memory leaks in generators

Generators can cause memory leaks, visible using the --show_leaks test option.
In particular the async.actor tests cause leaks the the mailbox _responde generator

Coverage test plugin

Currently to check test coverage we need to run tests in thread mode so that coverage can be collected.
We need a test coverage plugin using the coverage API in the same line as the profile test plugin.

New deferred implementation

Modify the current deferred implementation so that you can add error backs as well as callbacks. Exactly the same way as twisted.

Tasks to create new tasks

Some tasks are quite big, expecially periodic tasks which perform massive updates on data.

An idea could be to have periodic tasks which send tasks requests.

A map reduce framework.

Asynchronous Tasks

At the moment task-queue workers process a task at a time in a synchronous way.
Propose to:

  • Allow for asynchronous behavoiur by yielding and releasing the event loop
  • Process a maximum number of task concurrently by using max_concurrent_tasks configuration parameter.

contributing page in docs

Write up a doc page documenting how best to contribute to pulsar development.
The page is in docs/source/contributing.rst

exclude tag option in tests

To add an exclude option when running tests:

python runtests.py --exclude apps calculator

In the example the tests with labels apps and calculator won't run.

test plugins options

The command options of plugins which are not installed are available in the command line. They shouldn't.

Fix may_pool_task

The may_pool_task method in the TaskBackend periodically call itself every 1 second, to pull new tasks. Is this the best way?

I/O loop callbacks not called

When errors occurs, looping callbaks are not called and they keep getting added to the event loop slowing down the server response.

spawn to return a deferred

The pulsar.spawn method should return a deferred rather than a proxy. The deferred will receive the callback once the actor is fully operational. That is to say, the actor has registered its mailbox address with the ActorProxyMonitor.

Thread pool

Actor to have a thread pool implementation so that CPU type workers can have more than one thread to work on.

Link newly created monitors

When a new monitor is created with the arbiter has already started, the other monitors are not linked with it.
This is the proposal:

  • When a new monitor is created, it should link with existing monitors.
  • Existing monitors should pass the information to their existing actors.
  • The new actors will be aware of the link automatically.

Deferred returning deferred

Deferred returning deferred needs to have test put in place. Their behaviour is quite complex and difficult to follow.

Profiling task

Add functionality to profile a task run using the python profiler.

pattern importing

Importing modules using a pattern such as tests or test_* is working but all modules are imported first.
Modules not matching the pattern should not be imported.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.