quora/asynq

Python library for asynchronous programming

asynq's Issues

Fails to build on mac, workaround

Python 3.9.0b1, official macOS build, fresh virtualenv, pip install asynq fails:

    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/mb/xx/T/pip-install-ki69lzou/asynq/setup.py", line 57, in <module>
        setup(
      File "/v/lib/python3.9/site-packages/setuptools/__init__.py", line 144, in setup
        _install_setup_requires(attrs)
      File "/v/lib/python3.9/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
        dist.fetch_build_eggs(dist.setup_requires)
      File "/v/lib/python3.9/site-packages/setuptools/dist.py", line 716, in fetch_build_eggs
        resolved_dists = pkg_resources.working_set.resolve(
      File "/v/lib/python3.9/site-packages/pkg_resources/__init__.py", line 780, in resolve
        dist = best[req.key] = env.best_match(
      File "/v/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1065, in best_match
        return self.obtain(req, installer)
      File "/v/lib/python3.9/site-packages/pkg_resources/__init__.py", line 1077, in obtain
        return installer(requirement)
      File "/v/lib/python3.9/site-packages/setuptools/dist.py", line 786, in fetch_build_egg
        return cmd.easy_install(req)
      File "/v/lib/python3.9/site-packages/setuptools/command/easy_install.py", line 679, in easy_install
        return self.install_item(spec, dist.location, tmpdir, deps)
      File "/v/lib/python3.9/site-packages/setuptools/command/easy_install.py", line 705, in install_item
        dists = self.install_eggs(spec, download, tmpdir)
      File "/v/lib/python3.9/site-packages/setuptools/command/easy_install.py", line 890, in install_eggs
        return self.build_and_install(setup_script, setup_base)
      File "/v/lib/python3.9/site-packages/setuptools/command/easy_install.py", line 1158, in build_and_install
        self.run_setup(setup_script, setup_base, args)
      File "/v/lib/python3.9/site-packages/setuptools/command/easy_install.py", line 1146, in run_setup
        raise DistutilsError("Setup script exited with %s" % (v.args[0],))
    distutils.errors.DistutilsError: Setup script exited with error: command '/usr/bin/gcc' failed with exit code 1

I guess it should be calling something other than gcc on mac.
Tested with pip==19.2.3 and pip==20.1.1.

However, manually installing dependencies and then asynq works:

(v) ~ > pip install cython
...
Successfully installed cython-0.29.19

(v) ~ > pip install qcore
Collecting qcore
...
Requirement already satisfied: Cython in... (from qcore) (0.29.19)
Collecting inspect2 (from qcore)
...
Installing collected packages: inspect2, six, qcore
  Running setup.py install for qcore ... done
Successfully installed inspect2-0.1.2 qcore-1.6.1 six-1.15.0

(v) ~ > pip install asynq
Collecting asynq
...
  Running setup.py install for asynq ... done
Successfully installed asynq-1.3.0 pygments-2.6.1

[Ask] Reason to not run batch parallel?

When I read the source code of scheduler.py

I saw that, after creating a list of _batches for TaskScheduler, you flush these batch sequentially?
I think flushing _batches could be run concurrently.

The proof of concept is how Google Datastore NDB batching request.

They use an EventLoop to execute Future and Batch tasks, batch and tasks could run paralleley

async and await will be keywords in Python 3.7

Python 3.5 introduced async and await as context-dependent keywords, and their usage as identifiers is deprecated. In 3.7, they will be full keywords, so usage of these names as e.g. a function name will be a SyntaxError (see https://www.python.org/dev/peps/pep-0492/#deprecation-plans).

asynq uses both async and await as function names. I propose that we rename the functions, but keep the old names as aliases in Python <3.7 for external code to use.

Here are my suggestions for renaming:

• @async() can be renamed to @coroutine(), by analogy with the @asyncio.coroutine decorator of Python 3.4 for asyncio coroutines.
• await() (which is not useful very often) can be renamed to await_tasks.

No batching when yielding dicts in a fanout

Hello,

While converting our code to asynq I have came across a bug that causes batching to not happen when yielding dicts in a fan out fashion.

I have repro code as well of this behavior in this gist: https://gist.github.com/LiraNuna/a90b6ac3cc820e07022787ab07204f9a

Notice when yielding a list only a single batch is flushed but when yielding a dict, each batch contains only a single item.

Providing pre-built wheels

asynq has been a fantastic library for us, but having to install Cython to compile it slows down our build and complicates artifact building. How feasible is it to provide pre-built wheels of asynq?

It looks like https://github.com/scrapinghub/python-crfsuite/pull/53/files is an example of a travis.yml that builds manylinux wheels.

asynq.tools.aretry typing doesn't allow for tuples

The docstring for asynq.tools.aretry suggests we can pass in a tuple of exceptions but the pyi file doesn't include this case.

Cannot deduplicate methods that use forced kwargs

When using python3's syntax for forced kwargs notation, it is impossible to deduplicate:

@asynq.tools.deduplicate()
def foo(pos, *, forcenamed):
    print(pos, forcenamed)

The error

File "asynq/tools.py", line 254, in decorator
    argspec = getargspec(original_fn)
File "qcore/inspection.py", line 133, in qcore.inspection.getargspec (qcore/inspection.c:3949)
    File "qcore/inspection.py", line 149, in qcore.inspection.getargspec (qcore/inspection.c:3696)
ValueError: keyword-only arguments are not supported by getargspec()

will be thrown.

I believe getfullargspec needs to be used instead for these. I'm not sure if python2 has getfullargspec.

How to use Asynq in correct way?

Hi Guys,

I just try to use Asynq with below code:

from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.dialects.mysql import \
    BIGINT, BINARY, BIT, BLOB, BOOLEAN, CHAR, DATE, \
    DATETIME, DECIMAL, DECIMAL, DOUBLE, ENUM, FLOAT, INTEGER, \
    LONGBLOB, LONGTEXT, MEDIUMBLOB, MEDIUMINT, MEDIUMTEXT, NCHAR, \
    NUMERIC, NVARCHAR, REAL, SET, SMALLINT, TEXT, TIME, TIMESTAMP, \
    TINYBLOB, TINYINT, TINYTEXT, VARBINARY, VARCHAR, YEAR
from sqlalchemy import exc, Column, Integer, String, Index, inspect
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
from asynq import async
import json

app = Flask(__name__)
# db = SQLAlchemy(app)

Base = declarative_base()


def object_as_dict(obj):
    return {c.key: getattr(obj, c.key)
            for c in inspect(obj).mapper.column_attrs}


SQLALCHEMY_DATABASE_URI_UNSHARDDB = "mysql+mysqlconnector://%s:%s@%s:%s/%s" % (
    "root", "123456", "localhost", "3306", "asynq")
engine = create_engine(SQLALCHEMY_DATABASE_URI_UNSHARDDB, pool_size=100, max_overflow=0,
                       pool_recycle=3600, pool_pre_ping=True)
db_session = scoped_session(sessionmaker(autocommit=False, autoflush=True, bind=engine))


class User(Base):
    __tablename__ = 'users'
    id = Column(BIGINT(20), primary_key=True)
    name = Column(VARCHAR(45), nullable=False)

    def __init__(self, name):
        self.name = name


class Post(Base):
    __tablename__ = 'posts'
    id = Column(BIGINT(20), primary_key=True)
    users_ID = Column(BIGINT(20), nullable=False)
    content = Column(VARCHAR(45), nullable=False)

    def __init__(self, users_ID, content):
        self.users_ID = users_ID
        self.content = content


class Comment(Base):
    __tablename__ = 'comments'
    id = Column(BIGINT(20), primary_key=True)
    posts_ID = Column(BIGINT(20), nullable=False)
    users_ID = Column(BIGINT(20), nullable=False)
    content = Column(VARCHAR(45), nullable=False)

    def __init__(self, posts_ID, users_ID, content):
        self.posts_ID = posts_ID
        self.users_ID = users_ID
        self.content = content

@async()
def get_user(id):
    query = {
        "id": id
    }
    item = db_session.query(User).filter_by(**query).first()
    if item is not None:
        return object_as_dict(item)
    else:
        return None

@async()
def get_post(id):
    query = {
        "id": id
    }
    item = db_session.query(Post).filter_by(**query).first()
    if item is not None:
        return object_as_dict(item)
    else:
        return None

@async()
def get_one_comment(id):
    query = {
        "id": id
    }
    item = db_session.query(Comment).filter_by(**query).first()
    if item is not None:
        comment = object_as_dict(item)
        return comment
    else:
        return None

@async()
def get_post_comment_ids_gen(post_id):
    query = {
        "posts_ID": post_id
    }
    comments = db_session.query(Comment).filter_by(**query).all()
    if comments is not None:
        if len(comments):
            return map(lambda x: x.id, comments)

    return []

@async()
def get_post_comment_ids(post_id):
    ids = yield get_post_comment_ids_gen(post_id)
    return

comment_result = None
post_result = None

@async()
def get_comment(id):
    comment = yield get_one_comment.async(id)
    user = yield get_user.async(comment["users_ID"])
    comment["user"] = user
    # print comment
    return


@async()
def get_post_info(pid):
    try:
        post = yield get_post.async(pid)
        comment_ids = yield get_post_comment_ids_gen.async(pid)
        futures = [get_comment(cid) for cid in comment_ids]
        comments = yield futures
        post["comments"] = comments
        # raise StopIterator((post, comments))
        print post
        return
    except Exception as e:
        print e

@app.route("/")
def main():
    post = get_post_info.async(1)()
    return json.dumps(post)


if __name__ == "__main__":
    app.run(host="0.0.0.0")

But I don't know my way is correct yet or still not? Could you help me explain the way to use Asynq for this case: I get a post with many comment, each comment also have to have user info of that comment.

Thank you very much

limit batch size for asynq.BatchBase queries?

Hi,

When subclassing asynq.BatchBase is it possible to specify batch size?

e.g. limit 1000 items per get_multi command, otherwise the response would be too big and will timeout sometimes.

Nested contexts don't work right if two contexts are __eq__ to each other

When we leave the context, we'll call the _context list's remove function, which will remove the first context that's eq to the given one. This can mess up the nesting order, e.g.:

x = [C(0), C(1), C(0)]
x.remove(x[2])

will leave [C(1), C(0)] when the intention is to leave [C(0), C(1)].

Build asynq fail on Mac OSX 10.11.6

When I build asynq on my Mac, I got this error

$ python setup.py build
running build
running build_py
creating build
creating build/lib.macosx-10.11-intel-2.7
creating build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/__init__.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/_debug.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/async_task.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/batching.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/contexts.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/debug.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/decorators.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/futures.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/generator.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/mock_.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/scheduler.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/scoped_value.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/tools.py -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/utils.py -> build/lib.macosx-10.11-intel-2.7/asynq
creating build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/__init__.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/caching.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/debug_cache.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/entities.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/helpers.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/model.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/sample.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/sample_model.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_async_task.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_base.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_batching.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_batching_no_yield.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_channels.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_contexts.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_debug.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_decorators.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_exceptions.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_generator.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_mock.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_model.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_multiple_inheritance.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_performance.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_recursion.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_recursive_task.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_scoped_value.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_tools.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_utils.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/tests/test_yield_result.py -> build/lib.macosx-10.11-intel-2.7/asynq/tests
copying asynq/async_task.pxd -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/batching.pxd -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/contexts.pxd -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/_debug.pxd -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/decorators.pxd -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/futures.pxd -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/scheduler.pxd -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/scoped_value.pxd -> build/lib.macosx-10.11-intel-2.7/asynq
copying asynq/utils.pxd -> build/lib.macosx-10.11-intel-2.7/asynq
running build_ext
building 'asynq/async_task' extension
creating build/temp.macosx-10.11-intel-2.7
creating build/temp.macosx-10.11-intel-2.7/asynq
cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c asynq/async_task.c -o build/temp.macosx-10.11-intel-2.7/asynq/async_task.o
asynq/async_task.c:17571:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/async_task.c:17568:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/async_task.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/async_task.c:17683:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/async_task.c:17745:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/async_task.c:16577:28: warning: unused function '__Pyx_CyFunction_InitDefaults' [-Wunused-function]
static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *func, size_t size, int pyobjects) {
                           ^
asynq/async_task.c:16586:27: warning: unused function '__Pyx_CyFunction_SetDefaultsTuple' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *func, PyObject *tuple) {
                          ^
asynq/async_task.c:16591:27: warning: unused function '__Pyx_CyFunction_SetDefaultsKwDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) {
                          ^
asynq/async_task.c:16596:27: warning: unused function '__Pyx_CyFunction_SetAnnotationsDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) {
                          ^
9 warnings generated.
asynq/async_task.c:10398:26: warning: implicit conversion loses integer precision: 'long' to 'int' [-Wshorten-64-to-32]
  __pyx_v_i = (__pyx_t_2 - 1);
            ~  ~~~~~~~~~~^~~
asynq/async_task.c:12623:22: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
    __pyx_v_length = __pyx_t_4;
                   ~ ^~~~~~~~~
asynq/async_task.c:17571:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/async_task.c:17568:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/async_task.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/async_task.c:17683:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/async_task.c:17745:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/async_task.c:16577:28: warning: unused function '__Pyx_CyFunction_InitDefaults' [-Wunused-function]
static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *func, size_t size, int pyobjects) {
                           ^
asynq/async_task.c:16586:27: warning: unused function '__Pyx_CyFunction_SetDefaultsTuple' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *func, PyObject *tuple) {
                          ^
asynq/async_task.c:16591:27: warning: unused function '__Pyx_CyFunction_SetDefaultsKwDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) {
                          ^
asynq/async_task.c:16596:27: warning: unused function '__Pyx_CyFunction_SetAnnotationsDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) {
                          ^
11 warnings generated.
cc -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -Wl,-F. build/temp.macosx-10.11-intel-2.7/asynq/async_task.o -o build/lib.macosx-10.11-intel-2.7/asynq/async_task.so
building 'asynq/batching' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c asynq/batching.c -o build/temp.macosx-10.11-intel-2.7/asynq/batching.o
asynq/batching.c:11297:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/batching.c:11294:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/batching.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/batching.c:11409:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/batching.c:11471:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/batching.c:10303:28: warning: unused function '__Pyx_CyFunction_InitDefaults' [-Wunused-function]
static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *func, size_t size, int pyobjects) {
                           ^
asynq/batching.c:10312:27: warning: unused function '__Pyx_CyFunction_SetDefaultsTuple' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *func, PyObject *tuple) {
                          ^
asynq/batching.c:10317:27: warning: unused function '__Pyx_CyFunction_SetDefaultsKwDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) {
                          ^
asynq/batching.c:10322:27: warning: unused function '__Pyx_CyFunction_SetAnnotationsDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) {
                          ^
9 warnings generated.
asynq/batching.c:11297:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/batching.c:11294:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/batching.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/batching.c:11409:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/batching.c:11471:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/batching.c:10303:28: warning: unused function '__Pyx_CyFunction_InitDefaults' [-Wunused-function]
static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *func, size_t size, int pyobjects) {
                           ^
asynq/batching.c:10312:27: warning: unused function '__Pyx_CyFunction_SetDefaultsTuple' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *func, PyObject *tuple) {
                          ^
asynq/batching.c:10317:27: warning: unused function '__Pyx_CyFunction_SetDefaultsKwDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) {
                          ^
asynq/batching.c:10322:27: warning: unused function '__Pyx_CyFunction_SetAnnotationsDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) {
                          ^
9 warnings generated.
cc -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -Wl,-F. build/temp.macosx-10.11-intel-2.7/asynq/batching.o -o build/lib.macosx-10.11-intel-2.7/asynq/batching.so
building 'asynq/contexts' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c asynq/contexts.c -o build/temp.macosx-10.11-intel-2.7/asynq/contexts.o
asynq/contexts.c:5755:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/contexts.c:5752:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/contexts.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/contexts.c:5867:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/contexts.c:5929:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/contexts.c:4657:27: warning: unused function '__Pyx_ErrFetchInState' [-Wunused-function]
static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) {
                          ^
asynq/contexts.c:5118:40: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
        static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                       ^
asynq/contexts.c:5167:35: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
        static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                                  ^
asynq/contexts.c:5352:34: warning: function '__Pyx_PyInt_As_int' is not needed and will not be emitted [-Wunneeded-internal-declaration]
        static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) {
                                 ^
9 warnings generated.
asynq/contexts.c:5755:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/contexts.c:5752:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/contexts.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/contexts.c:5867:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/contexts.c:5929:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/contexts.c:4657:27: warning: unused function '__Pyx_ErrFetchInState' [-Wunused-function]
static CYTHON_INLINE void __Pyx_ErrFetchInState(PyThreadState *tstate, PyObject **type, PyObject **value, PyObject **tb) {
                          ^
asynq/contexts.c:5118:40: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
        static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                       ^
asynq/contexts.c:5167:35: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
        static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                                  ^
asynq/contexts.c:5352:34: warning: function '__Pyx_PyInt_As_int' is not needed and will not be emitted [-Wunneeded-internal-declaration]
        static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) {
                                 ^
9 warnings generated.
cc -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -Wl,-F. build/temp.macosx-10.11-intel-2.7/asynq/contexts.o -o build/lib.macosx-10.11-intel-2.7/asynq/contexts.so
building 'asynq/_debug' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c asynq/_debug.c -o build/temp.macosx-10.11-intel-2.7/asynq/_debug.o
asynq/_debug.c:4938:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/_debug.c:4935:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/_debug.c:364:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/_debug.c:5050:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/_debug.c:5112:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/_debug.c:4646:36: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
    static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                   ^
asynq/_debug.c:4673:31: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
    static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                              ^
7 warnings generated.
asynq/_debug.c:4938:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/_debug.c:4935:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/_debug.c:364:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/_debug.c:5050:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/_debug.c:5112:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/_debug.c:4646:36: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
    static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                   ^
asynq/_debug.c:4673:31: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
    static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                              ^
7 warnings generated.
cc -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -Wl,-F. build/temp.macosx-10.11-intel-2.7/asynq/_debug.o -o build/lib.macosx-10.11-intel-2.7/asynq/_debug.so
building 'asynq/decorators' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c asynq/decorators.c -o build/temp.macosx-10.11-intel-2.7/asynq/decorators.o
asynq/decorators.c:11376:14: warning: incompatible pointer types passing 'PyTypeObject *' (aka 'struct _typeobject *') to parameter of type 'PyObject *' (aka 'struct _object *')
      [-Wincompatible-pointer-types]
    e = (*v)(p->task_cls, a); if (e) return e;
             ^~~~~~~~~~~
asynq/decorators.c:17993:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/decorators.c:17990:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/decorators.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/decorators.c:18105:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/decorators.c:18167:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/decorators.c:15361:27: warning: unused function '__Pyx_CyFunction_SetDefaultsKwDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) {
                          ^
asynq/decorators.c:15366:27: warning: unused function '__Pyx_CyFunction_SetAnnotationsDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) {
                          ^
asynq/decorators.c:15920:32: warning: unused function '__Pyx_GetItemInt_List_Fast' [-Wunused-function]
static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i,
                               ^
asynq/decorators.c:15950:32: warning: unused function '__Pyx_GetItemInt_Fast' [-Wunused-function]
static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list,
                               ^
asynq/decorators.c:16544:46: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
              static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                             ^
asynq/decorators.c:16593:41: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
              static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                                        ^
asynq/decorators.c:16778:40: warning: function '__Pyx_PyInt_As_int' is not needed and will not be emitted [-Wunneeded-internal-declaration]
              static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) {
                                       ^
13 warnings generated.
asynq/decorators.c:11376:14: warning: incompatible pointer types passing 'PyTypeObject *' (aka 'struct _typeobject *') to parameter of type 'PyObject *' (aka 'struct _object *')
      [-Wincompatible-pointer-types]
    e = (*v)(p->task_cls, a); if (e) return e;
             ^~~~~~~~~~~
asynq/decorators.c:17993:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/decorators.c:17990:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/decorators.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/decorators.c:18105:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/decorators.c:18167:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/decorators.c:15361:27: warning: unused function '__Pyx_CyFunction_SetDefaultsKwDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) {
                          ^
asynq/decorators.c:15366:27: warning: unused function '__Pyx_CyFunction_SetAnnotationsDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) {
                          ^
asynq/decorators.c:15920:32: warning: unused function '__Pyx_GetItemInt_List_Fast' [-Wunused-function]
static CYTHON_INLINE PyObject *__Pyx_GetItemInt_List_Fast(PyObject *o, Py_ssize_t i,
                               ^
asynq/decorators.c:15950:32: warning: unused function '__Pyx_GetItemInt_Fast' [-Wunused-function]
static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list,
                               ^
asynq/decorators.c:16544:46: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
              static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                             ^
asynq/decorators.c:16593:41: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
              static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                                        ^
asynq/decorators.c:16778:40: warning: function '__Pyx_PyInt_As_int' is not needed and will not be emitted [-Wunneeded-internal-declaration]
              static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) {
                                       ^
13 warnings generated.
cc -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -Wl,-F. build/temp.macosx-10.11-intel-2.7/asynq/decorators.o -o build/lib.macosx-10.11-intel-2.7/asynq/decorators.so
building 'asynq/futures' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c asynq/futures.c -o build/temp.macosx-10.11-intel-2.7/asynq/futures.o
asynq/futures.c:8097:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/futures.c:8094:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/futures.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/futures.c:8209:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/futures.c:8271:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/futures.c:7673:42: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
          static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                         ^
asynq/futures.c:7700:37: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
          static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                                    ^
7 warnings generated.
asynq/futures.c:8097:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/futures.c:8094:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/futures.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/futures.c:8209:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/futures.c:8271:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/futures.c:7673:42: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
          static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                         ^
asynq/futures.c:7700:37: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
          static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                                    ^
7 warnings generated.
cc -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -Wl,-F. build/temp.macosx-10.11-intel-2.7/asynq/futures.o -o build/lib.macosx-10.11-intel-2.7/asynq/futures.so
building 'asynq/scheduler' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c asynq/scheduler.c -o build/temp.macosx-10.11-intel-2.7/asynq/scheduler.o
asynq/scheduler.c:20184:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/scheduler.c:20181:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/scheduler.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/scheduler.c:20358:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/scheduler.c:18333:32: warning: unused function '__Pyx_GetItemInt_Tuple_Fast' [-Wunused-function]
static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i,
                               ^
asynq/scheduler.c:18348:32: warning: unused function '__Pyx_GetItemInt_Fast' [-Wunused-function]
static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list,
                               ^
asynq/scheduler.c:19302:28: warning: unused function '__Pyx_CyFunction_InitDefaults' [-Wunused-function]
static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *func, size_t size, int pyobjects) {
                           ^
asynq/scheduler.c:19311:27: warning: unused function '__Pyx_CyFunction_SetDefaultsTuple' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *func, PyObject *tuple) {
                          ^
asynq/scheduler.c:19316:27: warning: unused function '__Pyx_CyFunction_SetDefaultsKwDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) {
                          ^
asynq/scheduler.c:19321:27: warning: unused function '__Pyx_CyFunction_SetAnnotationsDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) {
                          ^
asynq/scheduler.c:19750:41: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
              static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                                        ^
11 warnings generated.
asynq/scheduler.c:9192:15: warning: implicit conversion loses integer precision: 'Py_ssize_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
  __pyx_v_l = __pyx_t_2;
            ~ ^~~~~~~~~
asynq/scheduler.c:20184:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/scheduler.c:20181:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/scheduler.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/scheduler.c:20358:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/scheduler.c:18333:32: warning: unused function '__Pyx_GetItemInt_Tuple_Fast' [-Wunused-function]
static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Tuple_Fast(PyObject *o, Py_ssize_t i,
                               ^
asynq/scheduler.c:18348:32: warning: unused function '__Pyx_GetItemInt_Fast' [-Wunused-function]
static CYTHON_INLINE PyObject *__Pyx_GetItemInt_Fast(PyObject *o, Py_ssize_t i, int is_list,
                               ^
asynq/scheduler.c:19302:28: warning: unused function '__Pyx_CyFunction_InitDefaults' [-Wunused-function]
static CYTHON_INLINE void *__Pyx_CyFunction_InitDefaults(PyObject *func, size_t size, int pyobjects) {
                           ^
asynq/scheduler.c:19311:27: warning: unused function '__Pyx_CyFunction_SetDefaultsTuple' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsTuple(PyObject *func, PyObject *tuple) {
                          ^
asynq/scheduler.c:19316:27: warning: unused function '__Pyx_CyFunction_SetDefaultsKwDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetDefaultsKwDict(PyObject *func, PyObject *dict) {
                          ^
asynq/scheduler.c:19321:27: warning: unused function '__Pyx_CyFunction_SetAnnotationsDict' [-Wunused-function]
static CYTHON_INLINE void __Pyx_CyFunction_SetAnnotationsDict(PyObject *func, PyObject *dict) {
                          ^
asynq/scheduler.c:19750:41: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
              static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                                        ^
12 warnings generated.
cc -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -Wl,-F. build/temp.macosx-10.11-intel-2.7/asynq/scheduler.o -o build/lib.macosx-10.11-intel-2.7/asynq/scheduler.so
building 'asynq/scoped_value' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c asynq/scoped_value.c -o build/temp.macosx-10.11-intel-2.7/asynq/scoped_value.o
asynq/scoped_value.c:5909:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/scoped_value.c:5906:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/scoped_value.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/scoped_value.c:5971:26: warning: unused function '__Pyx_PyObject_IsTrue' [-Wunused-function]
static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) {
                         ^
asynq/scoped_value.c:6021:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/scoped_value.c:6083:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/scoped_value.c:5278:38: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
      static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                     ^
asynq/scoped_value.c:5327:33: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
      static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                                ^
asynq/scoped_value.c:5512:32: warning: function '__Pyx_PyInt_As_int' is not needed and will not be emitted [-Wunneeded-internal-declaration]
      static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) {
                               ^
9 warnings generated.
asynq/scoped_value.c:5909:28: warning: unused function '__Pyx_PyObject_AsString' [-Wunused-function]
static CYTHON_INLINE char* __Pyx_PyObject_AsString(PyObject* o) {
                           ^
asynq/scoped_value.c:5906:32: warning: unused function '__Pyx_PyUnicode_FromString' [-Wunused-function]
static CYTHON_INLINE PyObject* __Pyx_PyUnicode_FromString(const char* c_str) {
                               ^
asynq/scoped_value.c:367:29: warning: unused function '__Pyx_Py_UNICODE_strlen' [-Wunused-function]
static CYTHON_INLINE size_t __Pyx_Py_UNICODE_strlen(const Py_UNICODE *u)
                            ^
asynq/scoped_value.c:5971:26: warning: unused function '__Pyx_PyObject_IsTrue' [-Wunused-function]
static CYTHON_INLINE int __Pyx_PyObject_IsTrue(PyObject* x) {
                         ^
asynq/scoped_value.c:6021:33: warning: unused function '__Pyx_PyIndex_AsSsize_t' [-Wunused-function]
static CYTHON_INLINE Py_ssize_t __Pyx_PyIndex_AsSsize_t(PyObject* b) {
                                ^
asynq/scoped_value.c:6083:33: warning: unused function '__Pyx_PyInt_FromSize_t' [-Wunused-function]
static CYTHON_INLINE PyObject * __Pyx_PyInt_FromSize_t(size_t ival) {
                                ^
asynq/scoped_value.c:5278:38: warning: unused function '__Pyx_PyInt_From_long' [-Wunused-function]
      static CYTHON_INLINE PyObject* __Pyx_PyInt_From_long(long value) {
                                     ^
asynq/scoped_value.c:5327:33: warning: function '__Pyx_PyInt_As_long' is not needed and will not be emitted [-Wunneeded-internal-declaration]
      static CYTHON_INLINE long __Pyx_PyInt_As_long(PyObject *x) {
                                ^
asynq/scoped_value.c:5512:32: warning: function '__Pyx_PyInt_As_int' is not needed and will not be emitted [-Wunneeded-internal-declaration]
      static CYTHON_INLINE int __Pyx_PyInt_As_int(PyObject *x) {
                               ^
9 warnings generated.
cc -bundle -undefined dynamic_lookup -arch i386 -arch x86_64 -Wl,-F. build/temp.macosx-10.11-intel-2.7/asynq/scoped_value.o -o build/lib.macosx-10.11-intel-2.7/asynq/scoped_value.so
building 'asynq/utils' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch i386 -arch x86_64 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c asynq/utils.c -o build/temp.macosx-10.11-intel-2.7/asynq/utils.o
asynq/utils.c:1901:8: error: 'inline' can only appear on functions
static CYTHON_INLINE PyObject *(*__pyx_f_5asynq_10async_task_unwrap)(PyObject *); /*proto*/
       ^
asynq/utils.c:225:27: note: expanded from macro 'CYTHON_INLINE'
    #define CYTHON_INLINE __inline__
                          ^
1 error generated.
error: command 'cc' failed with exit status 1

Possible optimization: Eager task execution

CPython just applied an optimization to asyncio which avoids creating a full-fledged Task object if possible: python/cpython#97696. This apparently gives up to a 50% speedup on some asyncio benchmarks.

In principle, the same optimization should apply to asynq: we always create an AsynqTask and put it on the scheduler for every executed asynq function, but many asynq function likely complete without ever needing to block. I don't have hard numbers for that, though.

This optimization could be implemented mostly in _call_pure (

asynq/asynq/decorators.py

Line 130 in c5dc852

def _call_pure(self, args, kwargs):

):

• If self.needs_wrapper is true, we have an @asynq function that never yields. We can just return a ConstFuture directly without going through the scheduler.
• Otherwise, we do need to execute the generator. We can start the first step of the generator. If it returns, we can end early too and return a ConstFuture. If it yields, we can check whether it's yielding a ConstFuture: if so, we can simply send the ConstFuture back into the generator and proceed. But if it yields something else, we have to fall back to using the real scheduler.

This change could break user-visible behavior in some ways: for example, there is code in the @deduplicate decorator that assumes .asynq() always returns an AsyncTask, not a ConstFuture. Adapting such code may need some additional work. Compare python/cpython#97696 (comment).

mock.patch should be NonAsyncContext

If you use with asynq.mock_.patch in an asynq function, the context may not get cleaned up properly. It should use context.NonAsyncContext to prevent this.

Future's _compute method excepts BaseException

Should probably just except Exception.

Support __self__ on asynq'ed methods

Regular PY bound method objects have a __self__ attribute, pointing to the object they are bound to.

Would be great to be able to have the same for asynq()ed methods.

In [36]: class A:
    ...:     def foo():
    ...:         pass
    ...:     @asynq()
    ...:     def bar():
    ...:         pass
    ...:

In [37]: A().foo.__self__.__class__ is A
Out[37]: True

In [38]: A().bar.__self__.__class__ is A

Traceback (most recent call last):
  File "/.../venv/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2862, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-38-a140d1d32042>", line 1, in <module>
    A().bar.__self__.__class__ is A
AttributeError: 'asynq.decorators.AsyncDecoratorBinder' object has no attribute '__self__'
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-38-a140d1d32042> in <module>()
----> 1 A().bar.__self__.__class__ is A

AttributeError: 'asynq.decorators.AsyncDecoratorBinder' object has no attribute '__self__'

asynq/qcore is incompatible with six > 1.10.0

Hi Guys,

I have installed asynq with two commands as below on python 2.7.14:
pip install qcore
pip install asynq

It has been installed successfully. But when I try importing the library asynq with line: from asynq import async
I encounter the error as following:
Traceback (most recent call last): File "app.py", line 13, in from asynq import async File "/usr/local/lib/python2.7/site-packages/asynq/init.py", line 24, in from . import debug File "/usr/local/lib/python2.7/site-packages/asynq/debug.py", line 18, in import qcore File "/usr/local/lib/python2.7/site-packages/qcore/init.py", line 21, in from . import events File "qcore/events.py", line 377, in init qcore.events File "/usr/local/lib/python2.7/site-packages/six.py", line 831, in prepare return meta.prepare(name, bases) AttributeError: type object 'EnumBasedEventHubType' has no attribute 'prepare'

Could you help me to check why? Thank you very much.

Debugging+Logging: find out when sync function run in asynq context

As we migrate our code to use asynq, we would love a way to detect usages of sync calls within asynq contexts.

For example, imagine:

@asynq.async()
def data_fetch_1():
    return 5

@asynq.async()
def data_fetch_times_2():
    data = data_fetch_1()
    return data + data

This will incur a sync fetch within an already asynq call which should be fixed since it incurs performance penalties especially when data_fetch_2 is batched.

I have not seen a way to log/debug/add a hook to figure out when this happens

Please give info on the environment confguration

I see that there are some dependencies like cython. Please provide a requirements.txt and also update the values in setup.py so that they also get installed while doing the pip install

Proposing a PR to fix a few small typos

Issue Type

[x] Bug (Typo)

Steps to Replicate and Expected Behaviour

• Examine asynq/tests/test_active_task.py and observe synchonous, however expect to see synchronous.
• Examine asynq/tests/test_tools.py and observe seperate, however expect to see separate.
• Examine asynq/async_task.py and observe curent, however expect to see current.
• Examine asynq/tools.py and observe accomodate, however expect to see accommodate.

Notes

Semi-automated issue generated by
https://github.com/timgates42/meticulous/blob/master/docs/NOTE.md

To avoid wasting CI processing resources a branch with the fix has been
prepared but a pull request has not yet been created. A pull request fixing
the issue can be prepared from the link below, feel free to create it or
request @timgates42 create the PR. Alternatively if the fix is undesired please
close the issue with a small comment about the reasoning.

https://github.com/timgates42/asynq/pull/new/bugfix_typos

Thanks.