crossbario/autobahn-c

autobahn-c's Issues

Wifi configuration and first boot

When a user button connected to the ESP32 is pressed while the device is powered up, the device is entering "configuration mode".

When the device starts in "configuration mode", it will start the Wifi as an AP. The user will then connect to the ESP32 Wifi (which is acting as a Wifi AP accepting the Wifi connection coming in from the users smartphone) from an app. The app is a configuration tool we may provide, and which will upload the target Wifi configuration, as well as download the device public key.

The app then disconnects from the ESP32 Wifi, reconnects the users smartphone to the Internet, logs into a service and pairs the ESP32 device public key with the respective router and other configuration.

Before disconnecting, the smartphone app will send a "reboot" command to the ESP32, and the next boot cycle the ESP32 comes up, it will configure the Wifi for client mode, and connect to the Wifi the user has previsouly configured.

The different states and results that happen during above Wifi and device pairing process is display using different colors and intensity patterns ofa user RGB LED connected to the ESP32.

Hence, above scheme requires the following hardware connected to the ESP32:

1. a button
2. a RGB LED

Alpha example

A concrete first alpha goal for the library is to support the following example program.

The program is a simple C user program exposing a hardware button and a hardware LED as a WAMP component in a Crossbar.io based IoT system.

The user program interfaces with the hardware, registers a C procedure as a WAMP procedure to control the LED, and publishes WAMP events whenever the button is pressed.

For this, the user program creates a WAMP session object and uses that for registering, subscribing, calling and publishing. The session object is attached to a transport object that is a concrete implementation of an abstract stream transport.

The concrete stream transport is implemented for two platforms: Espressif ESP32 and ST STM32 F7, and runs on top of FreeRTOS, lwIP and mbedTLS.

The session object on the other hand is implemented in pure C and completely TLS, networking, OS, platform, SoC and CPU agnostic.

The library could be structured into:

1. platform runtime support (FreeRTOS, lwIP, mbedTLS, BLE)
2. byte stream transport abstraction (bidirectional reliable ordered stream of bytes), for example TCP-TLS
3. WAMP message channel transport abstraction (bidirectional reliable ordered stream of native language objects for WAMP messages), for example WAMP-RawSocket-CBOR
4. WAMP session state machine: this is driven from and uses the byte channel or RawSocket channel abstraction above is interfaces with user code - the rest of the library above is not public API

WAMP cryptosign / cryptobox support

Not a short term goal, but it would be awesome to have WAMP-cryptosign (and possibly WAMP-cryptobox) in AutobahnC later.

Besides NaCl/libsodium, look at these:

• https://tweetnacl.cr.yp.to/
• http://munacl.cryptojedi.org/ (-> FAQ)

Autobahn C Coding Standards

@oberstat So after googling it, there is no real globally used standard for C code. Every company, college, etc. seem to have invented their own. That being said, here are the two that come to mind

• Linux Kernel (https://www.kernel.org/doc/Documentation/CodingStyle)
• GNU (https://www.gnu.org/prep/standards/html_node/Formatting.html#Formatting)

The GNU one is short and sweet while the Linux one seems to be more concise AND give reasons for why things are a bad idea. I am leaning towards Linux but open for suggestions.

Futures and tasks

FreeRTOS provides a relatively sopisticated task abstraction to schedule and run user code on a shared, limited resources system, and with real-time reaction constraints in place.

The question would be if we could create a Future abstraction for use in C that is implemented on top of FreeRTOS tasks. Each Future created is a new FreeRTOS task, and the callback and errback code attached run in that task. When there is no more code to run, the task auto-destroys itself.

Here are a few resources:

• https://gist.github.com/Geal/8f85e02561d101decf9a
• http://www.freertos.org/a00125.html
• https://tls.mbed.org/kb/how-to/mbedtls-tutorial
• http://www.freertos.org/implementing-a-FreeRTOS-task.html
• http://www.freertos.org/Documentation/161204_Mastering_the_FreeRTOS_Real_Time_Kernel-A_Hands-On_Tutorial_Guide.pdf

CBOR library and API

Due to the nature of C, I think it doesn't make sense to try to aim for complete independence of the user API from the underlying serializer.

We do that for languages that are

• dynamically typed (JS, Python, ..)
• statically typed, but provide run-time reflection (Java/Jackson, ..)

---

• https://github.com/01org/tinycbor
• https://01org.github.io/tinycbor/current/index.html
• https://git.iotivity.org/iotivity/plain/resource/csdk/security/src/amaclresource.c

LTS API: write an idiomatic "Hello WAMP

Device pairing

When the device boots, it will check for the presence of a private Ed25519 key in the persistent memory (see https://ed25519.cr.yp.to/).

If there is no key, it will generate a new private key and store that in persistent memory. When a key has been freshly generated, it will also store a flag "new-key" in persistent memory.

The device then checks for Wifi and Crossbar.io configuration in persistent memory.

If the configuration is incomplete, it will start a BLE Gatt Server (see below), enter "BLE configuration mode" and start blinking the LED.

Subsequently during boot, the flag "unpaired" is checked for.

If the device is unpaired, the device will check if the Wifi and Crossbar.io URL are configured.

If the Wifi is configured, it will try connecting to the configured Crossbar.io router using WAMP-RawSocket-CBOR over TCP-TLS as transport.

It will then authenticate using WAMP-cryptosign with the device public-private key pair.

When connected, the device will start blinking green, and when the device and all app components are finally ready, it will show green permanently, only shortly flashing green-white-green every N seconds for heartbeating.

Maybe use a code generator

Could get annoying in C in particular to write all the WAMP message parsing and serialization code. Hence:

• crossbario/autobahn-java#162 (comment)
• https://github.com/darkl/meta-wamp

Remodel / PoC using the CoAP/GNRC example

I've read more about RIOT, in particular networking.

This from the release announcement of RIOT 2015/09:

RIOT 5th release is fresh out of the oven! featuring improvements and new functionalities such as:

• The new generic ("gnrc") network stack, a highly modular and configurable IPv6/6LoWPAN network stack. It implements a large number of IETF RFCs, such as RFC 2473, RFC 4861, RFC 4944, RFC 6550, or RFC 6775. It provides a unified API between the different layers and a generic network device interface.
• A new timer subsystem is introduced by xtimer, replacing hwtimer and vtimer modules. xtimer offers very precise timer operations as well as support for long-term timers running over days and weeks. Along with well-known timer operations in RIOT, it also provides a function for accurate periodic timers.

See: http://zolertia.io/blog/riot-5th-official-release-features-re-mote#sthash.FBlksRzR.dpuf

---

The GNRC network stack is described here.

There is an example (CoAP) that is using GNRC here.

The example "uses the GNRC network stack through RIOT's conn socket API.", in particular the GNRC UDP API, eg:

• conn_udp_create
• conn_udp_recvfrom
• conn_udp_sendto

This example is interesting, as it could serve as a blueprint for what we could do with WAMP.

For example, conn_udp_recvfrom seems to require a

static uint8_t _udp_buf[512]; /* udp read buffer (max udp payload size) */

buffer that will be filled with the complete UDP datagram payload, and app layer (in this case CoAP, but in ours WAMP), then parses this raw payload (into a CoAP request) and constructs a response in that very same buffer to send via conn_udp_sendto.

The example is also interesting as it uses RIOT's "threads" and message queues to wire up the different layers.

---

Now, as far as I get that, 6LoWPAN is an IPv6 (only) adaption layer that should allow IPv6 UDP with full, maximum payload (64k).

Autobahn shared core API

It seems totally possible to mimick the new shared core API in AutobahnPython and AutobahnJS also in an object-oriented style of C.

Having a core API shared across Python, JavaScript and C would be of tremendous value. It allows to apply WAMP and Autobahn API knowledge from tiny microcontrollers to the data-center to browsers and mobile.

Target HW Platforms

AutobahnC initially has two target SoC platforms:

• ST STM32F7 (for Ethernet based projects)
• Espressif ESP32 (for Wifi based projects)

---

• STM32 Nucleo-144 evalboard
• NUCLEO-F767ZI
• STM32F767ZIT6

---

• Espressif ESP32
• Espressif ESP-WROOM-32
• FreeRTOS
• lwIP
• mbedTLS

Dump from research:

FreeRTOS

lwIP

http://lwip.wikia.com/wiki/LwIP_Wiki

http://savannah.nongnu.org/projects/lwip/
http://www.nongnu.org/lwip/2_0_x/index.html

mbedTLS

https://tls.mbed.org/
https://github.com/ARMmbed/mbedtls

formerly PolarSSL! they bought it:

https://community.arm.com/iot/b/blog/posts/polarssl-is-dead-long-live-mbed-tls

they relicensed under Apache 2.0 (formerly GPL)

support TLS 1.2 with ECDHE-RSA, AES-GCM, SHA-384!
supports EC crypto with both NIST and Brainpool curves!

https://tls.mbed.org/core-features
https://tls.mbed.org/supported-ssl-ciphersuites

Perfect: TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384

it can be easily integrated into an event/callback based network stack (like lwIP)

https://tls.mbed.org/kb/how-to/how-do-i-port-mbed-tls-to-a-new-environment-OS

in particular:

mbedtls_ssl_set_bio()
https://tls.mbed.org/api/ssl_8h.html#a8b7442420aef7f1a76fa8c5336362f9e




Espressif ESP-WROOM-32


https://www.espressif.com/sites/default/files/documentation/esp32_datasheet_en.pdf

http://esp32.net/

https://github.com/espressif/esp-idf

http://esp-idf.readthedocs.io/en/latest/

https://www.soselectronic.de/products/espressif/esp-wroom-32-234033


The operating system chosen for ESP32 is freeRTOS with LWIP; TLS 1.2 with hardware acceleration is built in as
well.

https://www.pycom.io/product-category/shop/oem-products/

https://www.pycom.io/product/w01/
https://www.pycom.io/product/g01/

https://www.pycom.io/wp-content/uploads/2016/12/g01Specsheet.pdf

https://pypi.python.org/pypi/micropython-uasyncio.core
https://github.com/micropython/micropython-lib/tree/master/uasyncio

http://www.gemalto.com/m2m/solutions/modules-terminals/industrial/ems31

LTE Category M1 ("LTE Cat M1") and M2 ("NB-IoT")

https://www.qualcomm.com/media/documents/files/paving-the-path-to-narrowband-5g-with-lte-iot.pdf

https://www.pycom.io/
https://www.pycom.io/product/w01/
https://www.pycom.io/wp-content/uploads/2016/12/w01Specsheet.pdf
https://docs.pycom.io/pycom_esp32/_downloads/wipy2SpecsheetGraffiti.pdf

Cellular IoT Solutions

Altair Semi. ALT1250/ALT1910

https://altair-semi.com/press-releases/altair-introduces-alt1250-advanced-integrated-narrowband-cat-m1-nb1-cellular-iot-chip/
http://altair-semi.com/products/alt1250/
http://altair-semi.com/wp-content/uploads/2017/02/Altair-ALT1250-Product-Brochure.pdf

Altair ist eine israelische Fabless Chipdesign Schmiede die im Beseitz von Sony ist.

https://www.kickstarter.com/projects/1795343078/fipy-the-worlds-first-5-network-iot-dev-board

Autobahn C Transport Layer Discussion

Hey @oberstet. I thought allot about this and wanted to describe the issues I see (just to make sure we are on the same page) and make a design suggestion.

Writing a library in C that is reusable is very interesting because you have so many different environments that it needs to support and there is no standard API with how the language interacts with the system. That is all environment dependent. Lets give a few examples for clarity

• C running in Linux - Include the "pthread" and you have threading and mutex capabilities
• C running on device with RTOS (such as free RTOS) - threads, semaphores, and queues are available to synchronize between threads, interrupts, etc
• C running from "int main()" - Only main thread and interrupts are available. Need to use a timer interrupt to create events (and at that point you are starting to make your own scheduler)

As one can see, this poses many issues when trying to write a library that is reusable across the 3 environments because there is no standard way to interface the system.

The current implementations of WAMP use event loops of some sort but that requirement is really only introduced because the transport layer requires it, so it almost gets pushed up the entire stack as a requirement. Architecturally the current libraries are defined as follows

    +------------------------------+
    |          Connection          |
    +------------------------------+
    +----------------++------------+
    |   Transport    ||  Session   |
    +----------------++------------+

The connection basically ties together the transport and the session. Once the transport is established, the user code interfaces with the session directly. One interesting thing about this design is that technically, the session itself does NOT need the event loop in any way. If the user makes a call, it will ask the transport to send a packet. If a packet is received, it will process the packet (internally) and that may equate to some method callback or sending another packet. Regardless, the session does not need to schedule events, only the transport does. The session only needs callbacks when the state of the transport changes.

That all being said, I have created systems like this before where I actually had the application layer (the session essentially) send and receive over multiple transports at the same time (would have loved WAMP here. I probably half wrote it then ;)). For example, I was running the exact same application code in the following setups

• ARM Cortex <--> UART <--> Host PC
• ARM Cortex <--> Audio Port <--> iOS
• ARM Cortex <--> BT <--> iOS
• ARM Cortex <--> Socket (external WIFI) <--> iOS
• ARM Cortex <--> RF <--> ARM Cortex
• ARM Cortex <--> UART <--> ARM Cortex
• etc.

You get the gist. One interesting thing is that I actually had the EXACT same library code running on the ARM Cortex, iOS, and the Host PC since they all support C/C++ (it was a fun project). Allot of these setups were also one-to-many connections. So I basically had the same code running on top of all sorts of different flavors of transport in all sorts of different environments.

Anyways, the way I was able to get it so they worked like Lego blocks was to not try and make it so the blocks just blindly connected together. I instead clearly specified all of the interfaces and then left it up to the integration code to tie the pieces together. The integration code for this system running on ARM Cortex with FreeRTOS (interrupts, semaphores, queues, waits, etc) looked allot different than it running in iOS (callbacks) BUT the libraries themselves did not need to be modified in any fashion. The application layer was well tested and once the connection and transport logic was written, it "just worked".

Another issue I was having was that the system threads in an Embedded project are defined at the system level and not at the library level so my library could not make any assumptions about threads or mutexes since it needed to be environment agnostic. I ended up abstracting those into callbacks where the integration code could provide a "lock/unlock" mechanism if desired so I could have a TX and RX thread running simultaneously by sharing the same mutex.

The other and probably larger issue is that even if it is the same protocol (regardless what it is), the implementation of it is going to look different between a Freescale, TI, Silicon Labs, etc SoC even if they are the same processor. And allot of times, the TX/RX device can be off chip and there is some serial interface controlling it.

All of that being said, from my experience, my suggestion would be to not focus on the transport or connection inside the Autobahn C library (for now) but rather define a very clear interface and documentation for the session and then leave it up to the implementor to connect the dots. I would totally agree we can make examples of using it over different interfaces in different designs but I would not try to include that logic in the library itself. There are just too many environments in the C language since you are "at the metal" so to speak and no way to really account for all of them.

C Deferred

we need this to define the core API

Proposed Autobahn OO C Style

Given the complexities we will face with creating a WAMP stack in C, I would like to propose an OO style of coding to use that gives us as much OO capabilities as possible so we can cleverly use inheritance throughout the stack. The goal is to support as many OO features as we can without making the code difficult to develop/maintain. Here I will use an example of a "base" class and then create a "shape" and a "circle" using inheritance (ignore minor syntax issues and coding style)

base.h

#ifndef __BASE_H__
#define __BASE_H__

#include <stdlib.h>

typedef int bool;
#define true 1
#define false 0

// ** "void *" is used to eliminate struct conversion errors when using polymorphism of structure **
// ** methods are pointers in the structure so the pointer can be modified when subclassing **
// ** "super" included so subclassing instance can have access to super methods **
#define BASE_CLASS \
    char *_type; \
    void *_super; \
    void (*init)(void *); \
    void (*destroy)(void *); \
    char *(*type)(void *); \
    void *(*super)(void *); \
    bool (*is_kind_of)(void *, char *); \
    bool (*is_instance_of)(void *, char *);

typedef struct base_s {
    BASE_CLASS
} base_t;

// Constructor(s)
base_t *base_();

#endif

base.c

#include "base.h"
#include <string.h>

// ** methods static to disallow linking from outside this file **
static void base_init(void *);
static void base_destroy(void *);
static char *base_type(void *);
static void *base_super(void *);
static bool base_is_kind_of(void *, char *);
static bool base_is_instance_of(void *, char *);

base_t *base_() {
    base_t *this = (base_t *)malloc(sizeof(base_t));
    base_init(this);
    return this;
}

static void base_init(void *self) {
    // ** cast back to correct type **
    base_t *this = self;

    // ** Init variables **
    this->_super = NULL;
    this->_type = "base";

    // ** init methods **
    this->init = &base_init;
    this->destroy = &base_destroy;
    this->type = &base_type;
    this->super = &base_super;
    this->is_kind_of = &base_is_kind_of;
    this->is_instance_of = &base_is_instance_of;
}

static char *base_type(void *self) {
    base_t *this = self;
    return this->_type;
}

static void *base_super(void *self) {
    base_t *this = self;
    return this->_super;
}

static void base_destroy(void *self) {
    // ** void type is fine since free only needs address to free, doesn't care about type **
    free(self);
}

static bool base_is_kind_of(void *self, char *type) {
    base_t *this = self;
    if (strcmp(this->_type, type) == 0) {
        return true;
    }
    else if (this->_super != NULL) {
        // ** Example of calling the super of the instance **
        return (((base_t *)this->_super)->is_kind_of)(this->_super, type);
    }
    return false;
}

static bool base_is_instance_of(void *self, char *type) {
    base_t *this = self;
    return (strcmp(this->_type, type)) == 0?true:false;
}

shape.h

#ifndef __SHAPE_H__
#define __SHAPE_H__

#include "base.h"

typedef struct point_s {
    double x;
    double y;
} point_t;

// ** BASE_CLASS will inherit base attributes and methods **
#define SHAPE_CLASS \
    BASE_CLASS \
    int _id; \
    int (*id)(void *); \
    double (*area)(void *); \
    double (*circumference)(void *);

typedef struct shape_s {
    SHAPE_CLASS
} shape_t;

// Constructor(s)
shape_t *shape_(int);

#endif

shape.c

#include "shape.h"

// ** dummy class to allow access to super methods **
static base_t *super = NULL;

// ** methods static to disallow linking from outside this file **
static void shape_init(void *);
static int shape_id(void *);

shape_t *shape_(int id) {
    shape_t *this = (shape_t *)malloc(sizeof(shape_t));
    shape_init(this);
    this->_id = id;
    return this;
}

static void shape_init(void *self) {
    // ** cast back to correct type **
    shape_t *this = self;

    // Call the super initialization
    if (super == NULL) {
        super = base_();
    }
    (*super->init)(this);

    // ** Init variables **
    this->_super = super;
    this->_type = "shape";
    this->_id = -1;

    // ** init methods **
    this->init = &shape_init;  // ** overload super "init" method **
    this->id = &shape_id;
    this->area = NULL;  // ** virtual **
    this->circumference = NULL;  // ** virtual **
}

static int shape_id(void *self) {
    shape_t *this = self;
    return this->_id;
}

circle.h

#ifndef __CIRCLE_H__
#define __CIRCLE_H__

#include "shape.h"

// ** Inherits from SHAPE_CLASS **
#define CIRCLE_CLASS \
    SHAPE_CLASS \
    point_t _center; \
    double _radius;

typedef struct circle_s {
    CIRCLE_CLASS
} circle_t;

// Constructor(s)
circle_t *circle_(int, point_t, double);

#endif

circle.c

#include "circle.h"
#include "math.h"

// ** dummy class to allow access to super methods **
static shape_t *super = NULL;

static void circle_init(void *);
static double circle_area(void *);
static double circle_circumference(void *);

circle_t *circle_(int id, point_t point, double radius) {
    circle_t *this = (circle_t *)malloc(sizeof(circle_t));
    circle_init(this);

    this->_id = id;
    this->_center.x = point.x;
    this->_center.y = point.y;
    this->_radius = radius;

    return this;
}

static void circle_init(void *self) {
    circle_t *this = self;

    // ** Call the super initialization **
    if (super == NULL) {
        super = shape_(-1);
    }
    (*super->init)(this);

    // Init variables
    this->_super = super;
    this->_type = "circle";
    this->_center.x = -1;
    this->_center.y = -1;
    this->_radius = -1;

    // ** overload methods **
    this->init = &circle_init;
    this->area = &circle_area;
    this->circumference = &circle_circumference;
}

static double circle_area(void *self) {
    circle_t *this = self;
    return M_PI*this->_radius*this->_radius;
}

static double circle_circumference(void *self) {
    circle_t *this = self;
    return 2*M_PI*this->_radius;
}

main.c

#include "circle.h"
#include <stdio.h>

int main() {
    circle_t *circle;
    point_t center = {1,2};

    // ...

    circle = circle_(1234, center, 2.5);
    printf("circle id: %d\n", (*circle->id)(circle));  // ** calls shape "id" method **
    printf("circle area: %f\n", (*circle->area)(circle));  // ** calls circle overloaded "area" method **
    printf("circle circumference: %f\n", (*circle->circumference)(circle));  // ** calls circle overloaded "circumference" method **

    // ...

    printf("circle type: %s\n", (*circle->type)(circle));
    printf("circle is kind of base: %d\n", (*circle->is_kind_of)(circle, "base"));  // ** true **
    printf("circle is kind of circle: %d\n",(*circle->is_kind_of)(circle, "circle"));  // ** true **
    printf("circle is instance of base: %d\n",(*circle->is_instance_of)(circle, "base"));  // ** false **
    printf("circle is instance of circle: %d\n",(*circle->is_instance_of)(circle, "circle"));  // ** true **

    // ...

    (*circle->destroy)(circle);  // ** calls base "destroy" method **
    circle = NULL;

    return 0;
}

output

circle id: 1234
circle area: 19.634954
circle circumference: 15.707963
circle type: circle
circle is kind of base: 1
circle is kind of circle: 1
circle is instance of base: 0
circle is instance of circle: 1

Using this technique will provide inheritance, polymorphism, overloading, and access to "super" methods. It does not provide encapsulation (which is not possible in C). It also provides ways to add "is_kind_of" and "is_instance_of" functionality to allow more advanced coding patterns (as shown above).

The only downsides are

• Method calls are a little ugly ("(circle->area)(circle);") but this can be cleaned up using macros if desired ("#define area(a) ((a)->area)((a))" -> "area(circle);") but this may introduce naming conflicts
• Each class has a static instance of it's "super" which could be viewed as a waste of memory but this can be ommited if desired.