AREG SDK

Project Status

Introduction

AREG SDK is an interface-centric real-time asynchronous communication engine to enable distributed- and mist-computing, where connected Things interact and provide services as if they act like thin distributed servers. The communication engine of AREG SDK is based on Object Remote Procedure Call (ORPC) protocol.

Motivation
More than embedded
Composition
Roadmap
Software build
Software integration
Use cases and benefits
Examples
Licensing
Call for action

Motivation

Traditionally, devices act as connected clients to stream data to the cloud or fog servers for further processing.

Since data is generated and collected at the edge of the network (mist network), it makes sense to change the role of connected Things and provide network-accessible (Public) services directly on devices. This extends Cloud to the extreme edge and it is a good foothold for robust solutions such as:

Increase data privacy, which is an important factor for sensitive data.
Decrease data streaming, which is a fundamental condition to optimize network communication.
Autonomous, intelligent and self-aware devices with services directly in the environment of data origin.

[ ↑ to top ↑ ]

More than embedded

When we were designing AREG SDK, the guiding principle was to provide a homogeneous solution for Multithreading, Multiprocessing and Internet communication wrapped in services appropriately having Local, Public and Internet categories. These services are neither processes nor tasks managed by the operating system, they are software components with a predefined interface, in which methods are invoked remotely.

💡 In current version, the AREG communication engine supports Local (multithreading) and Public (multiprocessing) service categories. The Internet (web) service category is in development pipeline.

The AREG engine forms a fault-tolerant system and does not require process startup priority. It automatically discovers services, automates communications, enables distributed service programming, and helps developers to focus on application business logic as if they would program a single process application with one thread where methods of objects are event-driven. The engine guarantees that:

The crash of one application does not cause the crash of the system.
The service clients automatically receive service availability status notifications.
The requests, response and notifications are automatically invoked and run in thread-safe mode.

[ ↑ to top ↑ ]

Composition

Currently AREG SDK consists of:

Multicast router (mcrouter) for IPC that runs either as a service managed by the OS or as a console application.
AREG framework (or engine) is a (shared or static) library to link with every application.
Code generator tool to create client and server base objects from a service prototype document.

The network communication model of AREG framework is based on multicast router and service architecture is interface-centric, which in combination of both enable to comprise all features of action-centric architecture (Client-Server / Request-Reply model) and data-centric architecture (Publisher-Subscriber / PubSub model).

💡 The examples of AREG SDK contain multiple projects to demonstrate the features and fault tolerant behavior of AREG communication engine.

[ ↑ to top ↑ ]

Roadmap

The aim of AREG SDK is a lightweight self-sufficient system, which consist of an intelligent framework and multiple tools that help developers to create complex Desktop, Embedded and/or IoT edge applications in shorter time (reduce development and testing time 50-30%).

Planned framework features:
- Multi-channel and multi-protocol communication.
- Logging service (separate process) to collect logs in the network.
- Internet (web) category service.
Planned tools:
- Service interface designer.
- Interactive log viewer.
- Service testing and Data simulation tool.

[ ↑ to top ↑ ]

Software build

💡 Chek the Wiki page of AREG SDK for mode details. We change the content and add more details.

The source codes of AREG framework and examples support following platform, CPU and compilers:

OS	Linux (list of POSIX API), Windows 8 and higher.
CPU	x86, x86_64, arm and aarch64.
Compilers	GCC, g++, clang, MSVC and cygwin CC.

The tools to use to compile sources:

Solution	Platforms	API	Quick actions to compile
`CMakeLists.txt`	Linux, Windows	POSIX, Win32	Make one of these actions: - Call `cmake` in command line. - Configure and build in Visual Studio Code; - Configure and build in Microsoft Visual Studio.
`Makefile`	Linux	POSIX	Call `make` in command line.
`areg-sdk.sln`	Windows	Win32	Open and build in Microsoft Visual Studio.
`.project`	Linux, Windows	POSIX	Import and build projects in Eclipse.

💡 The other POSIX-compliant OS and compilers are not tested yet.
💡 Make user specific changes (like switch compiler or output folder) only in appropriate user files:

For cmake, make changes in conf/cmake/user.cmake file.

For make, make changes in conf/make/user.mk file.

For MSBuild, make changes in conf/msvc/user.props file.

After compilation, normally binaries are located in <areg-sdk>/product/build/<compiler-platform-path>/bin folder. Details on how to change compiler, load and compile sources for various targets are described in HOWTO document. The next are quick overviews.

Build with `cmake`

To build with cmake, open Terminal in your projects folder and take the steps:

# Step 1: Get sources from GitHub
$ git clone https://github.com/aregtech/areg-sdk.git
$ cd areg-sdk

# Step 2: Initialize cache and build configuration in folder './build' folder.
#         The switch 'BUILD_EXAMPLES' enable or disables examples. By default, it is enabled.
#         Example: cmake -B ./build -DBUILD_EXAMPLES=OFF
$ cmake -B ./build

# Step 3: Compile sources by calling: cmake --build ./build [-j [concurrent jobs]]
$ cmake --build ./build -j8

Build with `make`

To build with make, open Terminal in your projects folder and take the steps:

# Step 1: Get sources from GitHub
$ git clone https://github.com/aregtech/areg-sdk.git
$ cd areg-sdk

# Step 2: Compile sources by calling: make [all] [framework] [examples]
$ make

Build with IDE

Open Terminal or preferred IDE to clone source codes like this:

$ git clone https://github.com/aregtech/areg-sdk.git
$ cd areg-sdk

Depending on IDE or preferences make one of following:

Open areg-sdk.sln file in MSVS (2019 or higher) and compile solution.
Open areg-sdk folder in MSVC (2019 or higher), select CMakeLists.txt in areg-sdk root, configure and build.
Open areg-sdk folder in VS Code, select CMakeLists.txt in areg-sdk root, configure and build.
Open Eclipse, import all projects from areg-sdk root, select projects and build.

💡 Compilation with Eclipse under Windows might require to change the Toolchain.

[ ↑ to top ↑ ]

Software integration

Multicast router

Configure router.init file to set the IP-address and the port of multicast router (mcrouter):

connection.address.tcpip    = 127.0.0.1	# the address of mcrouter host
connection.port.tcpip       = 8181      # the connection port of mcrouter

The multicast router is a process that can run as a system service or application to form the network, and it can run on any device like PC, gateway or even a small router. In case of multithreading application, there is no need to configure and start mcrouter.

Logging service

Configure log.init to set scopes, priorities and file name for logging:

log.file        = %home%/logs/%appname%_%time%.log # create logs in 'log' subfolder of user home 
scope.mcrouter.*= NOTSET ;                         # disable logs for mcrouter.

scope.my_app.*                   = DEBUG | SCOPE ; # enable all logs of my_app
scope.my_app.ignore_this_scope   = NOTSET ;        # disable logs of certain scopes in my_app
scope.my_app.ignore_this_group_* = NOTSET ;        # disable logs of certain scope group in my_app

💡 By default, the router.init and log.init files are located in the config subfolder of binaries.
💡 To enable all logs of all applications, use scope.* = DEBUG | SCOPE ; .
💡 Currently logging is possible only in file.

Development

The development guidance and step-by-step example to create a simple service-enabled application are described in DEVELOP. See Hello Service! as an example to create service. More details are in the Wiki page of AREG SDK.

[ ↑ to top ↑ ]

Use cases and benefits

AREG SDK can be used in a very large scope of multithreading and multiprocessing application development.

💡 Expand each section to see the details.

Distributed solution

Click to show / hide distributed solution.

AREG SDK is a solution of distributed services that can run on any node of the network and because of automations and transparent communication interact as if the components are located in one process. To distribute services in the processes and define relationships, developers create model to load during runtime.

The following is a demonstration of a static model to start and stop services:

// Defines static model with 2 services
BEGIN_MODEL("MyModel")

  BEGIN_REGISTER_THREAD( "Thread1" )
    BEGIN_REGISTER_COMPONENT( "SystemShutdown", SystemShutdownService )
      REGISTER_IMPLEMENT_SERVICE( NESystemShutdown::ServiceName, NESystemShutdown::InterfaceVersion )
    END_REGISTER_COMPONENT( "SystemShutdown" )
  END_REGISTER_THREAD( "Thread1" )

  BEGIN_REGISTER_THREAD( "Thread2" )
    BEGIN_REGISTER_COMPONENT( "RemoteRegistry", RemoteRegistryService )
      REGISTER_IMPLEMENT_SERVICE( NERemoteRegistry::ServiceName, NERemoteRegistry::InterfaceVersion )
      REGISTER_DEPENDENCY("SystemShutdown")
    END_REGISTER_COMPONENT( "RemoteRegistry" )
  END_REGISTER_THREAD( "Thread2" )

END_MODEL("MyModel")

int main()
{
    // Initialize application, enable logging, servicing, routing, timer and watchdog.
    // Use default settings.
    Application::initApplication( );
    // load model to start service components
    Application::loadModel("MyModel");
    // wait until Application quit signal is set.
    Application::waitAppQuit(NECommon::WAIT_INFINITE);
    // stop and unload service components
    Application::unloadModel("MyModel");
    // release and cleanup resources of application.
    Application::releaseApplication();

    return 0;
}

This example uses MACRO to create a model "MyModel" with two services:

Service with the role "SystemShutdown" is registered in the thread "Thread1" and provides an interface with name NESystemShutdown::ServiceName.
Service with the role "RemoteRegistry" is registered in the thread "Thread2", provides an interface with name NERemoteRegistry::ServiceName and has dependency (i.e. is a client) of service with role "SystemShutdown".

The services are started when load model by calling function Application::loadModel("MyModel"), and stopped when call function Application::unloadModel("MyModel"). When define a model, these two services can be registered in the same thread or distributed in 2 processes. In all cases, the physical location of service components remain transparent, so that the architectures have more flexibility to distribute computing power. An example of developing a service and a client in one and multiple processes is in Hello Service! project described in the development guide.

Driverless devices

Click to show / hide driverless devices.

Normally, the devices are supplied with the drivers to install in the system and with the header files to integrate in the application(s). The drivers often run in Kernel mode and the crash of the driver crashes the entire system. Driver development requires a special technique, which is different for each operating system, and it is hard to debug.

Our proposal is to deliver driverless service-enabled devices, where device-specific services are described in the interface prototype documents.

In contrast to drivers, the service development does not differ from user mode application development, it is faster to develop, easily serves multiple applications (service clients), contains fewer risks and the code generator helps to generate client object from service interface document.

Real-time solutions

Click to show / hide real-time solutions.

When a remote method of the service interface is called, the engine of AREG SDK automatically generates and delivers messages to the target and automatically invokes the exact methods of the exact target objects. This makes communication real-time with ultra-low networking latency. Such solutions are highly required to develop time-sensitive applications for automotive, flock of drones, medtech, real-time manufacturing, real-time monitoring and other projects.

Digital twin

Click to show / hide digital twin.

Often, the digital twin applications use client-server architecture, where the middleware server collects the data of external devices and the UI application virtualizes them. In such solutions, devices interact either through server or UI client applications. The event-driven and the service-oriented architecture, and the real-time communication of AREG framework is a good solution to develop digital twin applications to virtualize, monitor and control external devices, and immediately react to environment or device state change in real-time mode. External devices may also communicate without additional layers, which is an important factor for emergency, security and safety cases.

Simulation and tests

Click to show / hide simulation and tests.

When hardware provisioning to all employees is impossible, testing and checking unexpected phenomena of rapidly changing software in a simulated environment is the most rational solution. If unit tests are for testing a small portion of code and the tests may contain bugs, the simulation is used by developers and testers to check functionality and stability of the system. Simulations are portable and accessible to everyone, help to optimize solutions and avoid unnecessary risks. Projects using simulations are better prepared for remote work and easier to outsource.

The software components in applications normally are split into Data, Controller, Business and the optional Presentation layers. Distributed and service-oriented solution of AREG framework can ease system testing in a simulated environment, where the Simulation application provides an implementation of Data layer services, so that the rest of the application can be tested without any change.

The same technique of simulating data can be used to create API-driven test automations.

[ ↑ to top ↑ ]

Examples

There are various examples to demonstrate features of the AREG SDK. The examples are listed in the examples/README.md document.

[ ↑ to top ↑ ]

Licensing

AREG SDK is under free open source Apache License Version 2.0. However, AREG SDK can be commercially licensed, which includes the commercial support, full rights to create and distribute software without open source license obligations. For commercial license, support or additional information, please visit Aregtech website or contact info[at]aregtech[dot]com.

[ ↑ to top ↑ ]

Call for action

We look for help and welcome to join the project:

See the list of open issues where we look for help.
If you need new features, please open new issue or start new discussion.
Please submit your pull requests in the candidate branch, which is our branch with latest sources.
When create a pull request, please understand that reviewing and testing takes time, and we as well pay attention on coding style.
If you look for invoiced commercial support or trainings, or if your project has possibility commercially support AREG SDK, please contact info[at]aregtech[dot]com.

Did we help your project? Have you learned something new? Have we inspired you for new great ideas? Then we ask not to be indifferent and . This small thank inspires contributors and help us to expand our community. Please also share the project with your network at and other social media platforms.