peterhinch/micropython-async

Application of uasyncio to hardware interfaces. Tutorial and code.

Asynchronous programming in MicroPython

CPython supports asynchronous programming via the asyncio library. MicroPython provides asyncio which is a subset of this, optimised for small code size and high performance on bare metal targets. This repository provides documentation, tutorial material and code to aid in its effective use.

asyncio version 3

Damien has completely rewritten asyncio which was released as V3.0. This is incorporated in all recent firmware builds. The resources in this repo may be found in the v3 directory. These include a tutorial, synchronisation primitives, drivers, applications and demos.

Concurrency

Other documents provide hints on asynchronous programming techniques including threading and multi-core coding.

Go to V3 docs

uasyncio version 2

This is obsolete: code and docs have been removed.

micropython-async's Issues

asyncio_priority

In trying to use a StreamReader (w/ uart) and a lower priority coro. It appears StreamReader's readline() never yields (the way it is implemented from examples I found).

Here's an example of what I'm referring to:

import asyncio_priority as asyncio
from pyb import UART
uart = UART(1, 115200)

async def reader():
    sreader = asyncio.StreamReader(uart)
    while True:
        res = await sreader.readline()
        print(res)

async def heartbeat(led):
    while True:
        led.toggle()
        await asyncio.after_ms(500)  # Will hang while a coro loops on a zero delay

loop = asyncio.get_event_loop()
loop.create_task(heartbeat(pyb.LED(4)))
loop.create_task(reader())
loop.run_forever()

I think a workaround would be to use the following (but is there a workaround to use StreamReader with your low priority coro library?):

async def reader():
    while True:
        if uart.any()> THRESHOLD: #check enough bytes has been received to ensure a '\n' 
            res = uart.readline() 
        await asyncio.sleep_ms(20)

'IndexError: pop from empty list' Exception in Queue (V3) due to uncleared _evput state

This is using the temporary V3 Queue. Not sure if Im wasting my breath if there is already a new version somewhere else (at least I dont know of it...)? Anyway, I used this one and ran into a tad bit of an annoyance :)

Due to some "unfortunate" scheduling of routines, one can get greeted by a lovely "IndexError: pop from empty list" in the get() method of queue, where one would expect it to wait if the queue is empty - sadly doesn't always work...

I believe I found the reason (and fix) but first a minimal example and my explanation.

import uasyncio as asyncio
import Queue


async def putter(q, sleep_ms=50):
    # put some item, then sleep
    while True:
        await q.put(1)
        await asyncio.sleep_ms(sleep_ms)


async def getter(q):
   # checks for new items, and relies on the "blocking" of the get method
    while True:
        print(await q.get())


async def task():
    q = Queue()

    await asyncio.gather(
        putter(q),
        getter(q)
    )

asyncio.run(task())

The error appears only under some (scheduling) circumstances.
One instance is:

• if the putter routine is called first, meaning some item is placed in the queue, making it not empty and the _evput will be set (https://github.com/peterhinch/micropython-async/blob/master/v3/primitives/queue.py#L47).
• if now the getter method is executed, it will return the item placed in queue as the 'if self.empty():' check (https://github.com/peterhinch/micropython-async/blob/master/v3/primitives/queue.py#L34) will evaluate to False.
• if the getter methods now goes for the next iteration, BEFORE another item is placed (for example due to the putter method sleeping or some scheduling happenstance), the L34 check will now be True as the queue is empty, however, the L36 call 'await self._evput.wait()' will immediately return as the _evput is still set, since we never cleared it. Clearing only happens after an empty queue was awaited and a new item was put.

So, a proposal solution is:

    async def get(self):  #  Usage: item = await queue.get()
        if self.empty():
            # Queue is empty, put the calling Task on the waiting queue
            await self._evput.wait()
        self._evput.clear()
        return self._get()

Not sure if causes other problems, that I didnt consider due to 1.30 am tiredness, but ye. But I guessed, regardles of whether self.empty() was True or False, if the 'self._evput.clear()' is reached, some item was placed by now (and it will be set, so I guess we dont need any check about it...)
I can also make a PR tomorrow after work if you agree that is a bug after all.

PS: In https://github.com/peterhinch/micropython-async/blob/master/v3/primitives/queue.py#L51 and https://github.com/peterhinch/micropython-async/blob/master/v3/primitives/queue.py#L59
it should be preferable to have

if self.maxsize and self.qsize() >= self.maxsize:

instead of the current

if self.qsize() >= self.maxsize and self.maxsize:

?

I believe Python evaluates boolean conditions lazily, meaning 'if self.maxsize' evaluates to false, we save ourself the second evaluation. Albeit we should probably check for 'if self.maxsize > 0', as we do not check or otherwise validate negative inputs. In the cpython/official version, negative numbers are treated like 0 that way and imply infinite size.

v3/as_drivers/client_server: socket close/server close

I use two ESP323 to test server and client.
MicroPython v1.13 on 2020-09-02

In
https://github.com/peterhinch/micropython-async/blob/master/v3/as_drivers/client_server/userver.py

...
    def close(self):
        print('Closing server')
        self.server.close()
        await self.server.wait_closed()
        print('Server closed')

server = Server()
try:
    asyncio.run(server.run())
except KeyboardInterrupt:
    print('Interrupted')  # This mechanism doesn't work on Unix build.
finally:
    print('finally:')
    server.close()
    _ = asyncio.new_event_loop()

There is server.close() in finaly,
but I don't see 'Closing server' and 'Server closed' in output.

The output is:

...
Received ['value', 17] from client 3
Received ['value', 18] from client 3
Received ['value', 19] from client 3
Received ['value', 20] from client 3
Interrupted
finally:

Are socket and server still opened?

I'm trying to use aremote.py togother with picoweb, and thing works werid

I'm doing something like this:

ROUTES = [
    # You can specify exact URI string matches...
    ("/", lambda req, resp: (yield from app.sendfile(resp, "index.html"))),
    ("/favicon.ico", lambda req, resp: (yield from app.sendfile(resp, "favicon.ico"))),
    ("/tinycolorpicker.css", lambda req, resp: (yield from app.sendfile(resp, "tinycolorpicker.css"))),
    ("/tinycolorpicker.js", lambda req, resp: (yield from app.sendfile(resp, "tinycolorpicker.js"))),
    ("/text-color.png", lambda req, resp: (yield from app.sendfile(resp, "text-color.png"))),
    ("/rgb/", rgb),
]
            
def cb(data, addr):
    if data == REPEAT:
        print('Repeat')
    elif data >= 0:
        print(hex(data), hex(addr))
    else:
        print('{} Address: {}'.format(data, hex(addr)))

print('Test for IR receiver. Assumes NEC protocol.')
machine.freq(160000000)
p = machine.Pin(14, machine.Pin.IN)
ir = NEC_IR(p, cb, True)  # Assume r/c uses extended addressing
    
app = picoweb.WebApp(__name__, ROUTES)
app.run(debug=False, host="0.0.0.0")

and aremote seem to be working fine, but every http request is blocked until pressing a remote key.
couldn't figure out what fishy here...

Calling `aclose()` on StreamReader ?

When closing a StreamReader e.g. as in:

 sr, sw = await asyncio.open_connection(host, port, ssl=use_ssl)
 # ....
 await sr.aclose()

the program will crash. My code runs also on cpython and interestingly the corresponding StreamReader on that platform doesn't even have a .close() method.

So my the questions is: am I even supposed to call sr.aclose() (and if not why does it exist)?

AttributeError: 'module' object has no attribute 'Event'

Hi Peter,
I am newbie. I am trying to use queue.py from primitives folder but I dont understand where to find "Event" class/function in asyncio.
In query.py appears this line and triggers the error <<AttributeError: 'module' object has no attribute 'Event'>>
self._evput = asyncio.Event() # Triggered by put, tested by get

I use Pycom device and I downloaded https://github.com/micropython/micropython-lib to use uasyncio but I didn´t find it in the whole lib. Thank you!

Suggestion: write hardware independent docs

Hi Peter!

Very nice work, but I'm a bit sad when start reading/testing uasyncio v3 code on my esp32. :(

Many docs are referencing and works just with pyboard, and I and many people do not have it and do not have any intention to have one pyboard.

My suggestion is to write the ALL docs where works in any board/processor. Mostly Micro controllers has that peripherals (GPIO, UART, I2C, AD, etc). So, if someone are using any micro controller where is capable to run MicroPython, the demos, examples and so on, will works. For now, the mostly demos, drivers, docs, etc are running just with Pyboard - that is a problem for all people that do not use pyboard. I understand that there are something that is supported just for the Pyboard, and the same for other processors, like as ESP32, like as WIFI/BLE/RMT. But this suggestions is for what is common between all micro controllers.

Anyway, specific features from a board is welcome, I'm just saying to not be the mostly them. ESP32 has, like as I told you, specific hardware features. That is welcome in your tutorial too :). But I know if you prefer to use effort to write docs where more people can use, and this way, is, in my opinion, writing hardware independent docs.

Thank you.

Show asyn initialisations in section 3: Synchronisation

The import of the separate async module as well as Event, Barrier and Semaphore initialisations are not explicitly shown in the examples of section 3.
It is mentioned in the introduction to section 3, but this may easily be overlooked when one jumps from one subsection to the other using the bookmarks in the table of contents.

At least, that is what happened to me yesterday and it took me a bit before it dawned on me that I needed to import an additional module.

An explicit link to the async GitHub repository would equally be a handy addition.

Cheers!

as_GPS: issues when gps module sampling > 1Hz

Hi Peter,

The as_GPS driver worked great when the gps module was at 1Hz standard sampling rate, however after I had set the module to sample at 5Hz, the example code shown in "3 Basic Usage" (the one using callback) behaved like it got stuck - it might print one or two lines of satellite data, but most of the time, it printed nothing, only 'waiting for GPS data' till it returned.

I connected the gps module by USB-TTL to PC, and it did sample at 5Hz.

What could be the issue?

Thanks.
Kaiyuan

running a aremote.py on esp32

I was running aremote.py few months back on esp8266, I have the new esp32 and tried using it.
got it to run, but it wasn't working, i.e. not of my of my remote click weren't cought and didn't recived on the _pin_cb function.

I tird wiring in every single combo, but it wasn't working. is there some limitation with esp32 input pin ? or something else i'm missing here ?

Suggestion: rename CancelError

This is more a suggestion than an issue, but since I saw that the commits about NamedCoro etc. are very recent, I thought it might be a good time to change it before it's too late :).

I think that CancelError is very confusing:

• it's not really an error, it's a part of the flow control: in the same way, we use StopIteration instead of StopIterationError

• at a first look, it's unclear whether it means that the task was canceled, or whether there was an error while canceling it

Some proposals for a better name:

• Canceled
• NamedCoro.Stop or NamedCoro.Canceled
• AsyncStop
• StopCoroutine, which is nice because it's similar to StopIteration

asyncio with multiple methods in microPython

Dear @peterhinch
When i am executing following code, "say" methods and led toggle method, are executed asynchronously so well. However, when i add time.sleep(10) in "async def say ()" method, led stops toggling, until completion of while True in "async def say()" method. Is it possible in micropython, to execute both functions asynchronously before waiting the completion of while True in one of each? Because led toggle should never been interrupted in according to the execution of another method.

Thanks for your support,

import uasyncio as asyncio
from machine import Pin
import time

LED_PIN = 13
led = Pin(LED_PIN, Pin.OUT, value=1)

async def toggle():
    while True:
        await asyncio.sleep_ms(500)
        led_toggle()

def led_toggle():
    led.value(not led.value())

async def say(what, when):
    # time.sleep(10)  ## !! ((if time sleep added for delay, led stops toggling)) !!
    while True:
        await asyncio.sleep(when)
        print(what)

loop = asyncio.get_event_loop()
loop.create_task(toggle())
loop.create_task(say('hello', 0.5)) # Schedule ASAP
loop.create_task(say('world', 1)) # Schedule ASAP
loop.run_forever()

DGRAM support?

Im trying to use socket.recvfrom for DNS server please add DGRAM support.

v3 tutorial: Event example not CPython compatible

I'm using this tutorial to get up-to-speed with the changes in uasyncio V3 in MicroPython.

I discovered one thing today that may or may not be of interest. I think one of the tutorial goals is to stick close to CPython behaviour in the examples. If that's right, you might be interested in this minor issue that I observed.

Running the Event example in CPython produces this result:

$ python original_event.py
Task exception was never retrieved
future: <Task finished name='Task-2' coro=<waiter() done, defined at original_event.py:5> exception=RuntimeError("Task <Task pending name='Task-2' coro=<waiter() running at original_event.py:7>> got Future <Future pending> attached to a different loop")>
Traceback (most recent call last):
  File "original_event.py", line 7, in waiter
    await event.wait()  # Pause here until event is set
  File "C:\Python38\lib\asyncio\locks.py", line 309, in wait
    await fut
RuntimeError: Task <Task pending name='Task-2' coro=<waiter() running at original_event.py:7>> got Future <Future pending> attached to a different loop
Waiting for event
Setting event
Event is set

Googling the error I found a seemingly related post:
asyncio.Semaphore RuntimeError: Task got Future attached to a different loop

I modified the Event example, applying the solution from the above post:

import asyncio
from asyncio import Event

async def waiter(event):
    print('Waiting for event')
    await event.wait()  # Pause here until event is set
    print('Waiter got event.')
    event.clear()  # Flag caller and enable re-use of the event

async def main():
    event = Event()
    asyncio.create_task(waiter(event))
    await asyncio.sleep(2)
    print('Setting event')
    event.set()
    await asyncio.sleep(1)
    # Caller can check if event has been cleared
    print('Event is {}'.format('set' if event.is_set() else 'clear'))

asyncio.run(main())

After modification, the example runs in CPython.

$ python test_event.py
Waiting for event
Setting event
Waiter got event.
Event is clear

The modified example also runs cleanly in MicroPython V1.14

uasyncio open_connection seems broken after freezing it

I just started a issue there:

micropython/micropython-lib#177

for me after freezing uasyncio it seems that yield from open_connection() never returns.

I'm investigating...

Double click not working

Using your latest aswitch/pushbutton functions press/release/double/long.
Using constructor pb = Pushbutton(pbpin,suppress=True)
The press, release and long functions are working, however the double function not regardless how fast i press the second time...
The async loop is busy, might that be a reason it doesn't seem to notice the second click? The double click is noticed as a single click.

You do know that the pushbutton function is checking the state of the pin every 50ms and if the pin has changed from 0 to 1 to 0 in that 50ms, you will not notice it?

Instead of poll, is Pin.irq an option? That will also release the pressure on the loop, since it won't check the pins anymore until the pins are actually pressed...

ImportError: no module named 'primitives'

Hello,

I am having an issue using the primitives modules. I get the 'ImportError: no module named 'primitives'' message any time I try to import a primitive. I can import uasyncio and print it's version though, 3,0,0. Any thoughts on what I maybe doing wrong? Keep in mind that I am Very new to programming so its most likely very basic .

A more ample introduction is required for the complete beginner.

With all due respect, Peter, but as a beginner I found it hard to jump head first into your otherwise excellent text.

Reading the web article Asynchronous Python for the Complete Beginner first, helped with rendering your text more accessible.

So I would suggest linking or adding some of these elements to the introduction of your text.

Thanks!
Serge

How to get ISR from c to trigger asyncio event

Hi,

I'm developing an embedded system. I want to get C side ISRs to trigger asyncio
activity. By doing so, I hope to avoid polling which drains batteries.

I've understood how to get C side to call a python callback, but
can't get the python function to initiate an asyncio action.

My code is:
import uasyncio as asyncio
from uasyncio.queues import Queue

msgSentQueue = Queue()
sendMsgQueue = Queue()
msgRecvQueue = Queue()
cbQueue = Queue()
loop = asyncio.get_event_loop()

def callback(cbQueue):
print('in Callback')
cbQueue.put(1)

loop.call_soon(eventHandler)

async def eventHandler(cbQueue, msgSentQueue):
#msg = ws.recvMsg()
while True:
result = await(cbQueue.get()) # may pause if a size limited queue fills
msg = "test"
print('in eventHandler')
await msgSentQueue.put(msg) # may pause if a size limited queue fills

async def cmdProc(msgRecvQueue):
while True:
result = await(msgRecvQueue.get()) # Will pause if q is empty
print('Result was {}'.format(result))

async def bar(cbQueue):
count = 0
while True:
count += 1
print(count)
callback(cbQueue)
#await cbQueue.put(1)
await asyncio.sleep(1) # Pause 1s

I would like to call the function callback, from
c when an event occurs, but in my test I can't
get callback to initiate execution in eventHandler.

I've tried using both:
cbQueue.put(1)
loop.call_soon(eventHandler)
neither work.

However if I put
#await cbQueue.put(1)
in async function bar, all work
as expected!

Can you advice?!?!
BR
Paul

Question: async write file to SD card

Hi Peter,

Thanks for the tutorial.

In my case, I have a displacement sensor which is an encoder triggered by pulling the cable. I need to write the data to SD card when certain conditions are met (e.g. some movement or displacement which are in a random matter). Since writing to SD card can be slow, I'm afraid the writing process will block the sensor from getting new data. How can I make it non-blocking with uasyncio V3? Would you pls give me an example? Thanks.

Kaiyuan

uasyncio Task Monitor #1

Hi Peter.

Great work about the uasyncio Task Monitor (https://github.com/peterhinch/micropython-async/tree/master/v3/as_demos/monitor), congratulations!

The question is: is necessary on the scope one channel for each task that will be monitored?

I have a scope, but it has just two channels.

Thank you.

Possible reliability problem with aswitch debounce mechanism

When using your aswitch module as underlying basecode I realized, that you are not using the debounce in the normal way. Maybe your intention was to improve response time, I don't know.

Expected behaviour:
If the pin state changes, wait the debounce time, then check the state again. If it is still the same, trigger the callback.

Actual behaviour:
If the pin state changes, immediately trigger the callback, then wait the debounce time, which in this case actually is just a delay until the state is checked again.

Expected problems:
Mainly a reliability problem. The pin state change could be a very short burst caused by fluctuations on the power line.
Example: The bell system in my house is very old. If someone rings the bell of any of my neighbours, I get a short burst on my bell line too. Not enough to trigger the real bell of course but with my esp8266 without a long enough debounce time I can see that burst and it would trigger the callbacks.
Therefore I'd have a reliability problem using your debounce implementation with my bell system.

Suggestion:
Check pin state after the debounce time again to ensure that it was not only a fluctuation/very short spike.

Edit: I realize that by not using interrupts like I did, the possibility for even recognizing that short voltage burst is very low. So I guess the possibility for actually experiencing unreliability can be considered low? Could it be more reliable to use the debounce normally?
You probably tested that module very carefully.

aswitch with both short and long press

Just started using your asyncio libs a few hours ago and are trying to implement the Pushbutton press and long functions.

    pb = Pushbutton(gpio.but)
    pb.press_func(self.butshort,())
    pb.long_func(self.butlong,())

Problem is, when it is a long press, first the press fires and then the long fires.

How can I configure this so that only long fires during a long press ?

distribute on pypi?

Thank you for this project!

Have you thought about hosting this as a micropython- package on pypi? It'd be a lot more seamless than having users manually copying primitives. (But perhaps there's a reason for this that I'm missing?)

Thanks again!

Handing loop object to function/class

Not a real issue:
Is there a difference when handing the loop object to the function or class, similar to Test 5&6?
I always thought there is only one loop instance. But now I am unsure. And since asyncio.Task.all_tasks() is not available I can't find it out.

micropython-async/cantest.py

Line 422 in 8239a3d

loop.run_until_complete(run_cancel_test6(loop))

How to make uasyncio.core a frozen module

I'm probably being incredibly dim, but I'm having great difficulty including uasyncio.core a frozen module on the esp8266. It needs to be frozen, as it's too big to upip or import from the file system. I'm symbolically linking from the micropython-lib directory.
First, the dot in the directory name causes compile errors on the build thusly:
build/frozen_mpy.c:4121:41: error: expected '=', ',', ';', 'asm' or 'attribute' before '.' token
STATIC const byte bytecode_data_uasyncio.core_example_call_soon__lt_module_gt__cb[52] = {

Second, the uasyncio.core module has no init.py file, so I don't see how it can be used as a module as is.

I'm asking this here, as other aspirant pythonistas may be similarly stuck and this seemed a good place. Apologies if this is not a good place. Could you give me any pointers on the correct way to do this?
Thanks in advance.

FAQ ?

Hi;

Posted on the micropython forum but no replies, so am adding here.

Is it possible to create a stand-alone asyncio class that can be called from code that does not import uasyncio and does not use uasyncio libs ?

Feature request: support for multiple event loops

A project I'm working on has two main contexts code can run in. Previously I had modeled those two contexts as two different event loops, with an explicit hand-off between them.

One loop was the "motion control" event loop. It was a higher priority and there were only specific spots it would be able to stop at safely. I could turn it into a blocking function (with the help of interrupts to handle some edge cases) but there are reasons why it's better for it to be co-operative multi-tasking based. The idea was that it would call motionLoop.stop() at regular intervals to allow the general loop to run.

The other loop was the general purpose loop which would run things like user plugins, a web interface, etc.

StreamReader sreader.read() not works on asyncio v3.

Hi Piter.

I have this code already perfect working in my application running on MicroPython 1.11.0, using asyncio v2 of course:

async def uart_receive():
    sreader = asyncio.StreamReader(uart)
    while True:
        await asyncio.sleep(1)
        res = await sreader.read()
        print('Received from UART: {}'.format(res))

Now I trying to port it to v3 (using MicroPython 1.12) , but this code not works anymore on uasyncio v3. I have this error:

Task exception wasn't retrieved
future: <Task> coro= <generator object 'uart_receive' at 3f8200e0>
Traceback (most recent call last):
  File "uasyncio/core.py", line 1, in run_until_complete
  File "umain.py", line 177, in uart_receive
TypeError: function takes 2 positional arguments but 1 were given

Ps: if I change from sreader.read() to sreader.readline() works on uasyncio v3, but I want to continue to use sreader.read().

Thank you so much.

OPENMV 4 AttributeError: 'module' object has no attribute 'deque'

Hi, had coording to the guide doc copy the related file to asyncio folder . When I run the auart.py on OPENMV4,but any error as show: 'module' object has no attribute 'deque',how to solve this issue.The OPENMV4 firmware is : MicroPython v1.11-omv OpenMV v3.5.1 2019-12-06; OPENMV4-STM32H743
Thanks!

check_async_code.py

https://github.com/peterhinch/micropython-async/blob/master/check_async_code.py - page not found.

asyncio.get_event_loop() is not attribute time()

File "aremote.py", line 80, in _cb_pin
AttributeError: type object 'Loop' has no attribute 'time'

79 loop = asyncio.get_event_loop()
80 self._ev_start.set(loop.time()) # asyncio latency compensation

import uasyncio as asyncio
obj = asyncio.get_event_loop()
dir(obj)
['class', 'module', 'name', 'qualname', 'close', 'stop', 'bases', 'dict', 'create_task', 'run_until_complete', 'call_exception_handler', '_exc_handler', 'run_forever', 'set_exception_handler', 'get_exception_handler', 'default_exception_handler']

no time

Async Telnet ?

I thought I'd try out the new v3 on a nightly master v1.14 to implement a simple non-blocking telnet server. To allow the terminal to function while the server is running, I'm running it in its own thread. I derive from IOBase, but have to bridge between the non-blocking, sync desires of uos.dupterm and the blocking (select-polling?) async reader/writer provided by uascynio.start_server. Below is what I have so far. It does connect (and reconnect!), and can see strings sent from the client, but it inevitably leads to a _thread.Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled., even, after some time, without any traffic (though perhaps the telnet client may send some commands at intervals).

Naturally it's hard to debug since you can't print things to the terminal without entering infinite recursion. Any advice/thoughts would be much appreciated!

import uasyncio as asyncio
from uio import IOBase 
import uos

class TelnetServer(IOBase):
    TELNET_PORT = 23
    async def run(self):
        print("Starting Telnet...",end='')
        self.server = await asyncio.start_server(self.accept, '0.0.0.0', self.TELNET_PORT)
        print("done")
        while True:
            await asyncio.sleep(100)
        
    async def accept(self, reader, writer):
        peer = reader.get_extra_info('peername')
        print("New telnet connection: {}".format(peer))
        self.writer = writer
        self.buf=b''
        #uos.dupterm(None) # Clear any old connection
        uos.dupterm(self)
        try:
            while True:
                try:
                    res = await reader.readline()
                    self.handle(res)
                except asyncio.TimeoutError:
                    res = b''
                if res == b'':
                    raise OSError
        except OSError as e:
            pass
        finally: 
            await reader.wait_closed()
            print('Telnet client disconnected: {}'.format(peer))

    def handle(self,res):
        while True:             # strip telnet commands
            f=res.find(b'\xff');
            if f is -1: break
            res=res[0:f] + res[f+3:]
        res = res.replace(b'\r\n',b'\n')
        self.buf += res
        
    def readinto(self,b):
        read_len = min(len(self.buf),len(b))
        if read_len:
            b[0:read_len] = self.buf[0:read_len]
            self.buf = self.buf[read_len:]
            return read_len
        else:
            return None

    def write(self,data):
        asyncio.create_task(self._write(data)) # fire and forget
        return(len(data)+2) # assume the best
        
    async def _write(self,data):
        self.writer.write(data+b'\r\n')
        await self.writer.drain()

import _thread
tns = TelnetServer()
def loop_in_thread(loop):
    loop.run_until_complete(tns.run())

loop = asyncio.get_event_loop()
_thread.start_new_thread(loop_in_thread,(loop,))

AttributeError: 'Server' object has no attribute 'task'

I am trying to run a asynchronous webserver using microdot libraries microdot.py and microdot_asyncio.py and came across the following error:

Traceback (most recent call last): File "<stdin>", line 40, in <module> File "uasyncio/core.py", line 1, in run File "uasyncio/core.py", line 1, in run_until_complete File "uasyncio/core.py", line 1, in run_until_complete File "<stdin>", line 37, in main File "microdot_asyncio.py", line 165, in start_server File "uasyncio/stream.py", line 1, in wait_closed AttributeError: 'Server' object has no attribute 'task'

Does it have to do with my asyncio version? I am running MicroPython v1.15 with the microdot files frozen into a custom build.
Thanks!

client_server example rewrite to uasyncio v3.

Hi Piter!

Two questions:

1. Is the StreamWriter/StreamReader still the best idea/strategy to works with uasyncio v3, I mean, in async mode (no blocking) TCP/IP socket Client and Server, like as this https://github.com/peterhinch/micropython-async/tree/master/client_server used in uasyncio v2?
2. If yes, could you please, to rewrite it to use the uasyncio v3, like as you did with UART example (ported to uasyncio v3) that are using as well the StreamWriter/StreamReader idea https://github.com/peterhinch/micropython-async/blob/master/v3/as_demos/auart.py ?

Thank you so much :)

First Asyncio example does not work as described.

In the tutorial
2.1 Program structure the event loop.
in following code

import uasyncio as asyncio
loop = asyncio.get_event_loop()
async def bar():
    count = 0
    while True:
        count += 1
        print(count)
        await asyncio.sleep(1)  # Pause 1s

loop.create_task(bar()) # Schedule ASAP
loop.run_forever()

I note that loop is a PollEventLoop object, not an EventLoop object and thus has no create_task or run_forever methods. I assume I'm doing something wrong, but what?

BTW. I'm using this build https://github.com/loboris/MicroPython_ESP32_psRAM_LoBo

IndexError While using the "Chain" Method on a function with @asyn.cancellable

Now I have a real issue:
If you use the normal await style from the chain example e.g:
result = await compute(x, y)
on a function with an @asyn.cancellable decorator, it will throw an:
IndexError: tuple index out of range Error.
(File "asyn.py", line 327, in new_gen)

micropython-async/chain.py

Line 15 in 8239a3d

result = await compute(x, y)

micropython-async/asyn.py

Line 327 in 8239a3d

if isinstance(args[0], TaskId): # Not a bound method

A useful usecase: REPL on REPL

I have an interesting suggestion for a use case for this documentation I hope: REPL on slave.

A second micropython (slave) board is connected to one of UARTs of the primary board (master). Using REPL on master board, we should be able to control REPL on slave, for example:

>>> import pyb
>>> remote import pyb

where remote is a software written using uasyncio which sends data to slave and receives answers.

I suppose such a use case would be really valuable to understand micropython-async and useful in real life.

Make the cancellable condition recursive through await functions

My understanding is that you have one subset of coros in the pool which have been assigned the cancellable property, and these all get stopped when we issue the cancel_all().

But I'd really like an emulation of the behavior of threads where, once execution has entered via the entry point, functions call other functions as they normally do, but the entire stack can be killed back to this entry point via a thread.stop.

Imagine if we had a single Cancellable declaration on a function that was made using a create_task() function (so we can think of the create_task() as a bit like a thread.start() at that function), and any call from within that function using the await keyword inherited the cancellable condition and belonged to the same sub-pool). Then I could think of this whole set as its own thread, which I could control and kill off all at once at my convenience.

My use-case is here:
https://github.com/goatchurchprime/jupyter_micropython_developer_notebooks/blob/master/scriptlet_technology/MQTT_scriptlets.ipynb

I am asyncronously waiting for an MQTT message (broken into pieces in this case and stitched together into a file called scriptlet.py) containing a piece of python code, which I execute asyncronously, and then continue waiting on the MQTT channel for more sets of python code while it continues to execute. This is where I had reason to want to kill off all the previously running functions received earlier.

I'd like not to have to insert @cancellable onto every single function in the scriptlet; it's enough to need to put in async and await everywhere (and maybe I could do that with some preprocessing, so I could pretend the same code can run natively as within this async environment).

ESP32 IRQ Latency

Hi @peterhinch .
I am working on an ir receiver / sender . Here is my simple demo.
This code work for me . I was able to receive and send back with absolute success.
Maybe you can combine this with your "aremote.py" to enable receiving NEC signal. !

The weird thing is that if I run this in combination with other code or running with _thread or uasyncio , the receiver flickering .

#version = 2.0
from machine import *
from time import *
import sys
"""
	Platform Note :;
		ESP32 is a sensitive platform , limited memory , no allocation memory when in ISR
		Therefore , there is a pre_buffer bytearray to optimize runtime speed
		Also , don't use this with _thread , not sure why .
"""
class Remote:
	def __init__(self):
		# Initialize GPIO
		self.recv = Pin(26 , Pin.IN , Pin.PULL_UP)
		self.recv.irq(trigger = Pin.IRQ_RISING|Pin.IRQ_FALLING , handler = self._handler)
		self.pwm = PWM(Pin(25) , duty = 0 , freq = 38000)
		
		# Initialize Buffer , see ISR note !
		self.buffer = [0 for x in range(1000)]
		self.bin = 0
		self.length = 0
		
		# Initialize Timing Property
		self.prev_irq = 0
		
		# Initialize User Interface
		self.learning = None 
		self.event_list = {}
		
	def _handler(self , source):
		self.time = ticks_us()
		if self.prev_irq == 0:
			self.prev_irq = self.time
			self.length = 0
			return
		self.buffer[self.length] = ticks_diff(self.time , self.prev_irq)
		self.prev_irq = self.time
		self.length += 1
		
	def _routine(self):
		while True :
			sleep_ms(200)
			if ticks_diff(ticks_us(),self.prev_irq) > 200000 and self.length > 0 :
				self._debug()
				print('DECODED = [{},{}] '.format(self.decode()[1],self.length))
				
				#self._send ( self.buffer , self.length , 40 )
				
				self.length = 0
				self.prev_irq = 0
				for x in range(len(self.buffer)):
					self.buffer[x] = 0
	def _debug(self):
		print('__________________RECV  {}  ______ {} _______'.format(self.length , self.bin))
		for x in range(self.length//8) :
			for i in range(8):
				print(self.buffer[x*8+i] , end = '\t' if  self.buffer[x*8+i]*50 > 300 else '#\t')
			print()
		for x in range(self.length%8):
			print( self.buffer[self.length//8 + x] , end = '\t' if  self.buffer[self.length//8 + x]*50 > 300 else '#\t')
			
		print('------------------------------------------')
		
		
	"""
		
	"""
	# Learn the next message and then written to the file . This is non-blocking
	def learn (self , name ):
		self.learning = name
		
	def event(self , name):
		self.event_list[name] = self._load(name)
	
	def _load(self,name):
		pass
	def inside	( self,t , r ):
		# avoid string allocation !
		if t[0] == r[0] and t[1] == r[1] :
			return True
		if t[1] > r[1] :
			return False 
		for x in range(r[1]-t[1]+1):
			if 	(((((2**(r[1]-x-t[1]\
				)-1)|((2**x -1)<<(t[\
				1]+(r[1]-x-t[1]))))^\
				(2**r[1]-1))&r[0])>>\
				(r[1]-x-t[1]))==t[0]:
				return True
		return False	
		
	def send(self,packet,length = None , mul = 40):
		length = length or len(packet)
		self.recv.irq(trigger = 0 , handler = None) # Disable IRQ Receiving Channel
		p = packet
		# Heavily optimize needed
		sleep = sleep_us
		duty = self.pwm.duty
		n = 0 # prev
		b = ticks_diff
		c = ticks_us
		d = self.buffer
		for x in range(length):
			if x%2 == 0 :
				duty(512)
			else :
				duty(0)
			sleep( p[x]) 
		duty(0)
		sleep_ms(3) # magic
		self.recv.irq(trigger = Pin.IRQ_RISING|Pin.IRQ_FALLING , handler = self._handler) # Re-enable IRQ
	
	def decode(self):
		self.bin = 0
		m = 50000
		for x in range(self.length):
			m = min(self.buffer[x],m)
		for x in range(0,self.length,2):
			if self.buffer[x+1] > m*3 and self.buffer[x] > m*3:
				continue
			if self.buffer[x+1] > self.buffer[x]*3//2 :
				self.bin += 2**(x//2)
			else :
				pass
		return hex(self.bin) , bin(self.bin)
		
remote = Remote()	
remote._routine()
		
		

NameError with Cancellable coroutines

I'm using Cancellable coroutines to display the duration of a process via serial connection. the following code works fine when called for the first time.

async def measure_dose_time(*args: Any, **kwargs: Any) -> None:
    """Measure the current dose time and send to host."""
    t0 = utime.ticks_ms()

    while True:
        await asyncio.sleep(1)
        t1 = utime.ticks_ms()
        dose_time_s = t1 - t0
        await ser.send_msg_async(
            json.dumps({"type": "dose_time", "seconds": dose_time_s}) + "\n"
        )


async def dose() -> None:
    """Handle automatic trigger signal."""
    loop.create_task(asyn.Cancellable(measure_dose_time)())
    asyncio.sleep(5)
    await asyn.Cancellable.cancel_all()

When calling the code a second time I get the following error message.

Traceback (most recent call last):
  File "main.py", line 16, in <module>
  File "meter.py", line 236, in <module>
  File "uasyncio/core.py", line 146, in run_forever
  File "uasyncio/core.py", line 101, in run_forever
  File "meter.py", line 168, in dose
  File "asyn.py", line 268, in cancel_all
  File "asyn.py", line 260, in _cancel
  File "uasyncio/core.py", line 239, in cancel
NameError: local variable referenced before assignment
MicroPython v1.9.3-240-ga275cb0f on 2018-01-26; PYBv1.1 with STM32F405RG
Type "help()" for more information.

I looked up the code in core.py but can not explain why this error occurs. Do you have any suggestions?

AttributeError: cancel (ESP32)

I'm trying to cancel a task created using a coro. The task runs and the start_progress() function runs, but the cancel fails as the board reports that the cancel attribute doesn't exist.

I'm running the latest 3.x firmware (esp32-idf3-20200327-v1.12-308-gdbba6b05d.bin) on an ESP32 (Adafruit ESP32 Feather).

Any hints? I can't figure out what I'm doing wrong.

This is essentially my code, with the irrelevant parts removed.

import uasyncio as asyncio

async def start_progress():
    print('test')
    await asyncio.sleep(1)

async def main(loop):
    #get data from websocket...
    if data.get('state') == 'play':
        progress = start_progress()
        loop.create_task(progress)
    else:
        asyncio.cancel(progress)

loop = asyncio.get_event_loop()
loop.create_task(main(loop))
loop.run_forever()

If you want the full code, let me know and I'll point you to it.

Thanks!

RTC clock sync breaks async loop timer

Hey guys - love the work on this, been transitioning to Async code for the last 6 months and loving the stability vs. multithreading on Pycom G01.

I'm having a really strange issue though with pretty simple async code that has only recently popped up. Following is my test function - start a async loop and run a function forever.

import uasyncio as asyncio
from _connections import Connections
connect = Connections(None)
loop = asyncio.get_event_loop()
loop.create_task(connect.pybytes_sender(1000)) #data sending, interval in seconds 
loop.run_forever()  

In this function I connect to LTE or Wifi (either has same behaviour). Once connected I call a function to sync with ntp:

  async def time_sync(self,conmode):
      timer = time.ticks_ms()
      while not self.rtc.synced():
          self.rtc.ntp_sync('time.google.com')
          print("Syncing clock")
          if time.ticks_ms() - timer > timeout: break
          await asyncio.sleep(3)
      if self.rtc.synced(): 
          self.isconnected = True
          log.info('RTC Sync: ' + str(self.rtc.now()))
      return

The issue is that as soon as the clock syncs from rtc.ntp_sync the asyncio.sleep line hangs forever and all other loops also stop.

I've worked through this many different ways - using ntptime.py (micropython) has the same effect, as soon as I update the internal clock the async.sleep calls either hang forever or are completely ignored. I've updated from V2 to V3 with no difference.

I'm guessing there may be a time reference change when the clock is synced - i.e. we jump far into the past or future and the timer doesn't adjust with this time change? Its the only thing I can think would explain the behaviour.

All my code runs as expected without syncing the internal clock.

Any ideas? Help would be greatly appreciated.

Memory Allocation Failed

I'm beginner and my limited knowledge of english doesn't help me. However, I've been trying to decoder the remote control of my devices using your code.

I'm using a NODEMCU and a YS-IRYM (v2.2) sensor but the issue is this: After uploaded the files to the root of the microcontroller (files together in the root: aremote.py, art.py, asyn.py) when I try to run art.py (uploaded to the root as main.py) it is displayed this error:

File "main.py", line 18, in
File "aremote.py", line 9, in
MemoryError: memory allocation failed, allocating %u bytes

I'd would really appreciate that you give me a hand,

Thanks,

Cancellable for Function with no argument .

Hi ,
def cancellable(f):
def new_gen(*args, **kwargs):
if isinstance(args[0], TaskId): # Not a bound method
task_id = args[0]
g = f(*args[1:], **kwargs)
else: # Task ID is args[1] if a bound method
task_id = args[1]
args = (args[0],) + args[2:]
g = f(*args, **kwargs)
try:
res = await g
return res
finally:
NamedTask._stopped(task_id)
return new_gen

Apparently this will raise TypeError when the function doesnt require any arg .
Example .
num = 1 @asyn.cancellable async def print_nums(): while True: global num;print(num) led.value(not led.value()) num += 1 await asyn.sleep(0.1)

Sorry for bad copy , no idea how this work , the problem is on line 327 of asyn.py .

Tutorial - gather() - observing different behaviour

Following your tutorial, you have mentioned that gather can return exceptions with the correct flag set.

However, using the code you provide in section 3.3 gather on MicroPython 1.17, I am entering the asyncio.CancelledError catch block even with the flag set. I believe this is contradictory to the behaviour you described with the return_exceptions=True argument?

>>> %Run -c $EDITOR_CONTENT
Start cancellable bar()
Start barking
Start timeout coro foo()
About to cancel bar
Cancelled
Result:  None

If I simply set bar() to return without being cancelled (code below for completeness sake, but I'm sure you get the idea), I do get the intended behaviour where the timeout error is included in the return collection.

I'm a total microcontroller (and Python) rookie, so I haven't done a whole lot of digging to understand the root cause myself, sorry.

try:
    import uasyncio as asyncio
except ImportError:
    import asyncio

async def barking(n):
    print('Start barking')
    for _ in range(6):
        await asyncio.sleep(1)
    print('Done barking.')
    return 2 * n

async def foo(n):
    print('Start timeout coro foo()')
    while True:
        await asyncio.sleep(1)
        n += 1
    return n

async def bar(n):
    print('Start cancellable bar()')
    return n

async def do_cancel(task):
    await asyncio.sleep(5)
    print('About to cancel bar')
    task.cancel()

async def main():
    tasks = [asyncio.create_task(bar(70))]
    tasks.append(barking(21))
    tasks.append(asyncio.wait_for(foo(10), 7))
    asyncio.create_task(do_cancel(tasks[0]))
    res = None
    try:
        res = await asyncio.gather(*tasks, return_exceptions=True)
    except asyncio.TimeoutError:  # These only happen if return_exceptions is False
        print('Timeout')  # With the default times, cancellation occurs first
    except asyncio.CancelledError:
        print('Cancelled')
    print('Result: ', res)

asyncio.run(main())

>>> %Run -c $EDITOR_CONTENT
Start cancellable bar()
Start barking
Start timeout coro foo()
About to cancel bar
Done barking.
Result:  [70, 42, TimeoutError()]

usyncio tutorial

hi all here / peter, Im looking for a tutorial for this new lib at 1.13 usyncio. I know is not the one Peter created here but I think after some days trying to do something I think any starting point is welcome. I was lookin these ones but still dont got it.

https://docs.micropython.org/en/latest/library/uasyncio.html?highlight=sched
https://gpiocc.github.io/learn/micropython/esp/2020/06/13/martin-ku-asynchronous-programming-with-uasyncio-in-micropython.html

for the records I need to run in after wake up a routine issuen 3 task with different frecuencies each, (i.e. #1 each 3 secs, #2 each 60 secs, #3 each 10 min) in a forever loop each one needs a few seconds to complete.
something like:

import uasyncio

async def AlarmedStatus(period):
print('sendSMS(status_msg)')
await uasyncio.sleep(period)

async def AlarmedLocation(period):
print('sendLocationMsg()')
await uasyncio.sleep(period)

async def wakeUpBySensor():
print('SetWifiAP() # prendo el AP para desactivar si owner se conecta ')
await uasyncio.sleep(1)
print('2G_Serial(1)')
await uasyncio.sleep(1)
print('2G_EN(1) # prendo el radio')
await uasyncio.sleep(1)
print('GPS_EN(1) # prendo el GPS')
await uasyncio.sleep(1)
print('2G_DataEN(1) # conecto los datos')
await uasyncio.sleep(1)

event_loop = uasyncio.get_event_loop() # creo un loop en asyncio
print('lanzo las tareas')
periodStatus = 3
periodLocation = 1
event_loop.create_task(AlarmedStatus(periodStatus))
event_loop.create_task(AlarmedLocation(periodLocation))
event_loop.run_forever()

wakeUpBySensor()

ESP32 exception BREAK instr when running with async v3

Hi,

I've faced issue with async code. I've tested 2 cases: loop with synchronous code, and loop with async code V3 as follow on ESP32 without PSRAM. I use the latest MicroPython version 1.15.

1. Synchronous code:

When running this function, it could run well 24/7 without any exceptions. Sleep in 1ms.

def _ctask_fn(arg):
   try:
      while True:
         _ccall(arg['_cfn'], arg)
         time.sleep_ms(arg['sleep'])
   except Exception as e:
      print('[_ctask_fn] %s' % str(e))
   finally:
      return

2. Async code

Also sleep in 1ms. But run in tens of seconds, or some minutes (2, 3), it occurs exceptions. It's rather random. Most of them were
Guru Meditation Error: Core 1 panic'ed (Unhandled debug exception)
Debug exception reason: BREAK instr

arg['_cfn'] is just a callback function, do nothing for test.

If sleep in 50 or 100ms, it's running rather long without exceptions. I've not tested with several hours/days.

async def _ctask_afn(arg):
   try:
      while True:
         _ccall(arg['_cfn'], arg)

         if 'sleep' in arg:
            await asyncio.sleep_ms(arg['sleep'])
         elif 'event' in arg:
           await arg['event'].wait()
           arg['event'].clear()
         elif 'tsf' in arg:
            await arg['tsf'].wait()

   except Exception as e:
      print('[_ctask_afn] %s' % str(e))
   finally:
      return

I really dont know why. It's just a normal/minimal async code. Pls guide me how to debug it. I think that problem is at async scheduler side.

Thanks.

Link/url broken #1

Hi Peter!

In the link https://github.com/peterhinch/micropython-async/blob/master/v3/docs/I2C.md has a "This can occur transparently to the application and is covered in detail in section 5.3." where "in section 5.3" is a hyperlink to https://github.com/peterhinch/micropython-async/blob/master/v3/docs/README.md#53-responder-crash-detection where this link do not exists.

Thank you.

DRIVERS.md document correction

First a thank you. Thanks for this fanatic code and documentation!

I was followinghttps://github.com/peterhinch/micropython-async/blob/master/v3/docs/DRIVERS.md and it states to install the primitives directory on your hardware. I have done that then moved on to "2. Installation and usage" section. Doing so, the first 3 from primatives.[...] imports work fine but the last does not.

import os

print(os.listdir('.'))
['boot.py', 'net_connect.py', 'primitives', 'pushover.py', 'sync_main.py', 'wifi_creds.py']

print(os.listdir('primitives'))
['__init__.py', 'aadc.py', 'barrier.py', 'condition.py', 'delay_ms.py', 'message.py', 'pushbutton.py', 'queue.py', 'semaphore.py', 'switch.py', 'tests']

print(os.listdir('.'))
['boot.py', 'net_connect.py', 'primitives', 'pushover.py', 'sync_main.py', 'wifi_creds.py']

print(os.listdir('primitives'))
['__init__.py', 'aadc.py', 'barrier.py', 'condition.py', 'delay_ms.py', 'message.py', 'pushbutton.py', 'queue.py', 'semaphore.py', 'switch.py', 'tests']

from primitives.switch import Switch
from primitives.pushbutton import Pushbutton
from primitives.aadc import AADC
from primitives.irq_event import IRQ_EVENT
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: no module named 'primitives.irq_event'

I searched wthin the primitives directory/tree and could not find a file that provides irq_event.

Am I over looking something?