multithread async deserialize

[andrew-ld]

multithread async deserialize

[iamDonkey]

I also want it🤯

[andrew-ld]

@iamDonkey I'm trying to find a solution but I'm not finding an easy way.

[andrew-ld]

a8fa5f2#diff-184fc5e51edee9e0cbe93c22165b2e4809c9279ea8d645de86630af29a75c751R378

if you look i made the way that the non deserialized body is read in the main thread, now it should be done that the second schema.read (the one under the todo) is done in another thread, the problem is that the deserializer is asynchronous so i should start another async loop on another thread and i don't like that my library does it internally, it could lead to leaks.

[andrew-ld]

@iamDonkey you can try the mt_body_deserialize branch, I have implemented a multithreaded deserialiser, you can let me know if it works well for you

[andrew-ld]

#5

[iamDonkey]

hi @andrew-ld i don't know how to test it but right now master branch seems to be broken.

ERROR:root:failure while read message from mtproto: Traceback (most recent call last):
  File "/root/LL-mtproto-master/ll_mtproto/client.py", line 258, in _mtproto_loop
    message = await mtproto_link.read()
  File "/root/LL-mtproto-master/ll_mtproto/network/mtproto.py", line 332, in read
    auth_key, auth_key_id = await self._get_auth_key()
  File "/root/LL-mtproto-master/ll_mtproto/network/mtproto.py", line 170, in _get_auth_key
    await self._create_auth_key()
  File "/root/LL-mtproto-master/ll_mtproto/network/mtproto.py", line 190, in _create_auth_key
    new_nonce, (p, q) = await asyncio.gather(
  File "/root/LL-mtproto-master/ll_mtproto/network/mtproto.py", line 134, in _in_thread
    return await self._loop.run_in_executor(self._executor, *args, **kwargs)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/root/LL-mtproto-master/ll_mtproto/math/primes.py", line 7, in factorize
    return cryptg.factorize_pq_pair(pq)
AttributeError: module 'cryptg' has no attribute 'factorize_pq_pair'

[andrew-ld]

@iamDonkey is not broken, you have to update the dependencies.

pip3 install -U -r requirements.txt

[iamDonkey]

Already installed and the other branch is working

[andrew-ld]

because the other branch does not use cryptg.factorize_pq_pair, make sure you have cryptg 0.3 installed.

[iamDonkey]

My fault i see cryptg version is set to 3

[iamDonkey]

hi @andrew-ld , i am trying to call a method concurrently/parallel request using asyncio.gather(*array of function) but if i check responses received time its like sequential.
is it the right way of calling method concurrently?

[andrew-ld]

@iamDonkey

LL-mtproto/test.py

Line 49 in b5ddb6c

multiple_requests_test = await session.rpc_call_multi([