dotpot/conpig

A green threading library with automated concurrency for python based on a processes, signal handlers and greenlets

Conpig Threading Library

Conpig is a green threading library using green threads from Gevent which are based on greenlets. along side a signal based scheduler. This allows one true concurrency using green threads instead of OS processes or OS threads such that spawning a new thread is still cheap.

Want

• Fast, cheap, easy, traditional concurrency.

Have

• Cpython's GIL prevents OS threads from running concurrently and switching context for any reason but to perform IO.

• OS threads are bulky and slow. Processes are even worse.

• Current green thread implementations require manual context switches. This is tedious. This is also bug prone for programmers who are used to having a scheduler.

• Processes based libraries like parallel python prevent you from communicating between "threads" with the datastructures you'd usually use. Again, processes are also slow and buggy.

Solution

• Create a global scheduler using signals to schedule the next context switch.

Drawbacks & Limitations

• A solution to concurrency, not parallelism.

• Conpig threads still can only run on one core of a processor.

• Context switches still only occur every CPython tick.

• You still need cooperative yields if you want true fairness for programs that don't do as much IO.

Installing (pip to come)

• Need: Greenlet & Gevent.

1. Install Greenlet with pip.

2. Install Gevent

   • The old version depends on libevent. The new github version depends on libev
   • On OSX the install path for libevent is wrong even if you use Mac Ports.
   • The easiest install method is to download v.1 from github, and install from source using the provided "setup.py".

Usage

• You can run from a main method

import conpig

def test(arg):
    for i in range(0,4000):
        print arg


conpig.spawn(test, "X")    
conpig.spawn(test, "O")

conpig.wait_all()

Authors

• Matthew Mirman

• Eric Faust

License

See LICENSE file.