Asynchronous reading from and writing to the network

[johanstokking]

Asynchronous reading from and writing to the network

[icraggs]

Hello Johan. Thanks for opening this issue. Last year I started adding threading into the client for the FreeRTOS port. I think that continuing and optimizing this approach for Linux and FreeRTOS is worth exploring, which I will try to do this week.

On the callback option, can you explain what you mean by "I would not require this to be supported by the network interface"? I'm not sure I understand.

[johanstokking]

Thanks, I'm looking forward. If you need testing or anything, let me know.

I mean that we shouldn't require socket callback support in the network interface (like NetworkInit()), because it may not be supported. I think there's better support for threading than for socket callbacks, but I'm not an embedded operating system's expert.

[johanstokking]

BTW, in my particular use case, I can work with FreeRTOS.

What can I do to make this working using threading?

[icraggs]

Hi Johan. Sorry I've been distracted by other work, including the Paho C and embedded MQTT-SN clients. I'll try to get to this issue this week. I can work with Linux in the first place - that would be easier for me anyway. I intend to make it work the same way on Linux and FreeRTOS. You did say that you use another OS - was it OpenWRT? Are you using the Linux interface at the moment?

[johanstokking]

@icraggs would be great. I'm very much looking forward. We're using a PIC32 with Microchip Harmony but turns out that it runs FreeRTOS. I'm currently using Linux indeed; it should there as well in my use case.

[icraggs]

Can you tell me the version of FreeRTOS used? And what TCP stack?

[johanstokking]

We're using FreeRTOS v8.x, and a TCP/IP library from Microchip Harmony. The latter is kind of exotic but we've got it working. It's doable to implement the network abstraction in there and I'll contribute it to this project.

[ammaree]

@icraggs any progress on the async client? I agree with @johanstokking that the thread solution (as opposed to callbacks) is better supported in the RTOS space. We use FreeRTOS v9.0.0 with 2 different TCP/IP stacks, SimpleLink on CC3200 and Cyclone/TCP on BCM43362. Threads can work in both environments, callbacks not.

In a current CoAP based implementation we have used slightly different roles for the 2 threads.

Receive/read thread: receives all incoming requests (GET/PUT/POST/DELETE) and responses (ACK etc) and completely act on these up to the point (if required) of producing a response packet. At this stage an entry is put into a message queue for the transmit thread.

Transmit/write thread: takes as input entries from it message queue and transmits these. Handles all retransmission (if required) and correctly frees up (if any) dynamically allocated memory buffers.

In this scenario the configuration can be tuned based on resources (available memory), performance (thread priorities for receiving and transmitting volumes) and various other considerations.

We would like to start a migration from CoAP to MQTT (with paho mqt) but having asynchronous client communications is a must.

Thanks

[johanstokking]

@ammaree I got it working, see https://github.com/thethingsnetwork/paho.mqtt.embedded-c

I'm planning a pull request. We changed the code formatting though, which was very inconsistent, so loads of diffs, but you'll get the point.

[icraggs]

@johanstokking As we discussed, please make a pull request. I'm happy to take some time to figure out how to merge.