perklet/aioify

Make every function async and await-able.

aioify (maintenance mode)

Authors of aioify and module-wrapper decided to discontinue support of these libraries since the idea: "let's convert sync libraries to async ones" works only for some cases. Existing releases of libraries won't be removed, but don't expect any changes since today. Feel free to fork these libraries, however, we don't recommend using the automatic sync-to-async library conversion approach, as unreliable. Instead, it's better to run synchronous functions asynchronously using https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor or https://anyio.readthedocs.io/en/stable/api.html#running-code-in-worker-threads.

Old documentation

Make every function async and await-able.

Usage

pip install aioify

For example, make os, shutil and user defined function await-able.

#!/usr/bin/env python
###########
# Warning #
###########
# This code should be executed only on POSIX OS with at least 1 GiB free space in /tmp/ directory and RAM!

from aioify import aioify
import os
import shutil


def generate_big_file(filename, file_size):
    with open(file=filename, mode='wb') as f:
        f.write(os.urandom(file_size))


aiogenerate_big_file = aioify(obj=generate_big_file)
aios = aioify(obj=os, name='aios')
aioshutil = aioify(obj=shutil, name='aishutil')


async def main():
    dir_path = '/tmp/big-files/'
    await aios.makedirs(name=dir_path, exist_ok=True)
    filename = os.path.join(dir_path, 'original')
    copy_filename = os.path.join(dir_path, 'copy')
    file_size = 1024 * 1024 * 1024
    await aiogenerate_big_file(filename=filename, file_size=file_size)
    await aioshutil.copy(src=filename, dst=copy_filename)
    await aioshutil.rmtree(path=dir_path)


if __name__ == '__main__':
    import asyncio as aio
    loop = aio.get_event_loop()
    loop.run_until_complete(main())

aioify's Issues

[QUESTION] Using asyncio.gather() with @aioify functions

I wonder how much of a shortcut Aioify really provides (on a way to async python).

One of the ways I would like to use Aiofy is to issue number of REST requests concurrently. Code looks something like this:

async def call_many_rest_async():
   result = await asyncio.gather(
        call_rest1_async(),
        call_rest2_async(),
        ...
   )
   # do something with result 

...
@aioify
def call_rest1_async():
   return call_rest1()
@aioify
def call_rest2_async():
   return call_rest2()

The question is, will the code above execute the REST calls sequentially or concurrently? I am afraid that the answer is - sequentially. Am I wrong?

aioify don't wrap imported modules and objects from other modules inside the requested module (for example, when wrapping os, os.path is not wrapped)

The old API “await aios.path.exists('/')” no longer works. Why? Is there any way to smoothly upgrade? If not, what is the latest version of the old API installable by pip?

General question regarding API usage/behaviour

Recently came across aioify and it's been great to use so far. In saying that, I do have one question about a the API that I was hoping someone could answer.

For context, I wanted to see if I could use aioify on CoinGeckoAPI() (from the pycoingecko library, wrapper for crypto web API) and then call its methods in an async fashion. The issue I'm seeing is; some methods don't return a coroutine when they should.

Methods:

• cg.get_coin_by_id(id="bitcoin")
• cg.get_supported_vs_currencies()

Issue (see comments for context):

import asyncio
from aioify import aioify
from pycoingecko import CoinGeckoAPI

cg = aioify(obj=CoinGeckoAPI(), name='cg')

async def main():

    # using await
    await cg.get_coin_by_id(id="bitcoin") # throws ‘TypeError: object dict can't be used in 'await' expression’
    await cg.get_supported_vs_currencies() # returns expected result

    # without await
    cg.get_coin_by_id(id="bitcoin") # returns expected result
    cg.get_supported_vs_currencies() # throws 'RuntimeWarning: coroutine 'CoinGeckoAPI.get_supported_vs_currencies' was never awaited'
    
    # what works
    cg.get_coin_by_id(id="bitcoin") # returns expected result
    await cg.get_supported_vs_currencies() # returns expected result

if __name__ == '__main__':
    asyncio.run(main())

Both methods simply query an endpoint and return the result. After comparing the two, the one difference is an internal wrapper @list_args_to_comma_separated on get_coin_by_id (this wraps many other methods in the library also).

# source: https://github.com/man-c/pycoingecko/blob/8b7a8451cc8911eff5a7f268c885da4bc1570c3f/pycoingecko/api.py

   def get_supported_vs_currencies(self, **kwargs):
        """Get list of supported_vs_currencies"""
   ...

   @list_args_to_comma_separated
   def get_coin_by_id(self, id, **kwargs):
       """Get current data (name, price, market, ... including exchange tickers) for a coin"""
    ...

Taking a stab in the dark, I suspect this issue occurs because @list_args_to_comma_separated is external to the CoinGeckoAPI object and it doesn't get "aioify-ed". When get_coin_by_id is executed, @list_args_to_comma_separated probably breaks aioify's async and returns the result as if it was a regular function rather than a coroutine.

Is there any way to handle this scenario? The expectation being both methods should be awaited.

NameError: name 'ModuleNotFoundError' is not defined

At first aioify was working very well I used it close to 4 months with out any issue
All of a sudden I get this following error
NameError: name 'ModuleNotFoundError' is not defined

import subprocess
import shlex 
import time
import asyncio
from aioify import aioify
try:
    import simplejson as json
except ImportError:
    import json

# Create an Asyncronous version subprocess.Popen
self.asyncfy_popen =  aioify(obj=subprocess.Popen)
self.SendWSSMessages =  aioify(obj=self.wss.websocket.sendMessage)

The above piece of code worked until it failed
witht he error discribed above.

aioify is a good library for sync function, Can I interrupt my sleep function with cancel?

import time
import asyncio
from aioify import aioify


@aioify
def long_time_task():
    print("I'm going to sleep 12 second")
    time.sleep(12)
    print("I have slept 12 second")


async def main():
    task = asyncio.create_task(long_time_task())
    await asyncio.sleep(2)
    task.cancel()
    try:
        await task
    except asyncio.CancelledError:
        print("The task has been canceled")


if __name__ == "__main__":
    asyncio.run(main())

That's my code, But I don't known why the task is still alive, and what can I do to make it work well.
I'd like to run a long time task, so I use time.sleep to make task run a long time.
When I make it async, it doesn't work as my expectd.
😭

Support Curio and Trio

How to join current thread by calling aioify?

I found my threadlocal value doesn't work in aioify since it create a new thread named 'ThreadPoolExecutor-2_0:140634794800200' to handle coroutine code.

del my_func():
    pass

async_call = aioify(obj=my_func, name='run_async_my_func')

await async_call()

2021-07-09-16:12:32  xxxxxxxxxxxxxxx "2021-07-09 16:12:32,464 [INFO] <ThreadPoolExecutor-2_0:140634794800200>

How to avoid creating new thread and just join to current thread?
Here is my workaround by using native asyncio now:

del my_func():
    pass

def async_call():
    import asyncio
    return asyncio.coroutine(my_func)()

await async_call()

Broken in Python v3.10

I found this issue in aioify's dependency, module-wrapper's repository:
rominf/module-wrapper#2

"""
It seems like importing the package in Python 3.10.1 will raise

FileNotFoundError: [Errno 2] No such file or directory: '/.../python3.10/site-packages/stdlib_list/lists/3.10.txt'

"""

Would it be possible to remove this dependency to fix the issue of not having py3.10 support?

Please upload 0.3.0 to pypi

I am trying to use aiojira, which depends on the latest version of this module, but it cannot be installed directly using pip.

Is there a reason to not update the version in pypi to 0.3.0? Could we get it updated?