Giter Site home page Giter Site logo

lexus2k / tinyproto Goto Github PK

View Code? Open in Web Editor NEW
218.0 13.0 46.0 5.52 MB

Tiny Software Protocol for communication over UART, SPI, etc

License: GNU General Public License v3.0

Makefile 0.50% C 43.85% C++ 51.47% Shell 0.79% Batchfile 0.17% CMake 0.39% Python 2.83%
protocol communication protocol-library hdlc-like hdlc arduino microcontroller atmega avr esp32

tinyproto's Introduction

Tiny Protocol

Github actions Coverage Status Documentation License License

Introduction

Tiny Protocol is LAYER 2 protocol. It is intended to be used for the systems with low resources. It is also can be compiled for desktop Linux system, and it can be built for Windows. Using this library you can easy implement data transfer between 2 microcontrollers or between microcontroller and pc via UART, SPI, I2C or any other communication channels. You don't need to think about data synchronization between points. The library use no dynamic allocation of memory.

TinyProto is based on RFC 1662. Tiny Protocol supports 2 HDLC modes: ABM (for peer to peer connections) and NRM (for multi-drop connections).

Tiny Protocol is NOT an application layer protocol, although, for example, it can be used with Protocol Buffers. The protocol supports ABM and NRM modes.

Key Features

Main features:

  • Hot plug/unplug support for ABM (peer to peer).
  • Connection autorecover for Full duplex (both for ABM and NRM modes) and Light protocols (with enabled crc)
  • Error detection: Simple 8-bit checksum (sum of bytes), FCS16 (CCITT-16), FCS32 (CCITT-32)
  • Platform independent hdlc framing implementation (hdlc low level API: hdlc_ll_xxxx)
  • Easy to use Light protcol - analogue of a SLIP protcol (tiny_light_xxxx API, see examples)
  • Full-duplex protocol (tiny_fd_xxxx true RFC 1662 implementation, supports confirmation, frames retransmissions: ABM and NRM modes )
  • one to one and one to many modes
  • Frames of maximum 32K or 2G size (payload limit depends on platfrom).
  • Low SRAM consumption (starts at 60 bytes).
  • Low Flash consumption (starts at 1.2KiB, features can be disabled and enabled at compilation time)
  • No dynamic memory allocation!
  • Special serial loopback tool for debug purposes and performance testing (ABM mode only)

Supported platforms

  • Any platform, where C/C++ compiler is available (C99, C++11)
Platform Examples
ESP32 IDF
Cortex M0 Zero
Linux Linux
Windows Win32
Other Any platform with implemented HAL

What if my platform is not yet supported?

That's not a problem. Just implement abstraction layer for your platform (timing and mutex functions). Please go through the steps below:

  • add TINY_CUSTOM_PLATFORM define to your compilation flags.
  • Implement HAL functions and call tiny_hal_init() to pass your platform functions to the library
  • add CONFIG_TINYHAL_THREAD_SUPPORT define to your compilation flags if your platform supports standard c++ thread library

Refer to tiny_hal_init() function. To understand HAL implementation refer to Linux and ESP32 examples in HAL abstraction layer. You may use template code platform_hal.c

Easy to use

Cpp

Usage of light Tiny Protocol in C++ can look like this:

#include "tinyproto.h"

tinyproto::Light  proto;
tinyproto::Packet<256> packet;

void setup() {
    ...
    proto.beginToSerial();
}

void loop() {
    if (Serial.available()) {
        int len = proto.read( packet );
        if (len > 0) {
            /* Send message back */
            proto.write( packet );
        }
    }
}

Example of using full duplex Tiny Protocol in C++ is a little bit bigger, but it is still simple:

#include "tinyproto.h"

tinyproto::Fd<FD_MIN_BUF_SIZE(64,4)>  proto;

void onReceive(void *udata, tinyproto::IPacket &pkt) {
    // Process message here, you can do with the message, what you need
    // Let's send it back to the sender ;)
    if ( proto.write(pkt) == TINY_ERR_TIMEOUT ) {
        // Do what you need to do if looping back failed on timeout.
        // But never use blocking operations inside callback
    }
}

void setup() {
    ...
    // Here we say FD protocol object, which callback to call once new msg is received
    proto.setReceiveCallback( onReceive );
    proto.begin();
}

void loop() {
    if (Serial.available()) {
        uint8_t byte = Serial.read();
        proto.run_rx( &byte, 1 ); // run FD protocol parser to process data received from the channel
    }
    uint8_t byte;
    if ( proto.run_tx( &byte, 1 ) == 1 ) // FD protocol fills buffer with data, we need to send to the channel
    {
        while ( Serial.write( byte ) == 0 ); // Just send the data
    }
}

Python

import tinyproto

p = tinyproto.Hdlc()
def on_read(a):
    print("Received bytes: " + ','.join( [ "{:#x}".format(x) for x in a ] ) )

# setup protocol
p.on_read = on_read
p.begin()

# provide rx bytes to the protocol, obtained from hardware rx channel
p.rx( bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E  ]) )

How to build

Linux

make
# === OR ===
mkdir build
cd build
cmake -DEXAMPLES=ON ..
make

Windows

mkdir build
cd build
cmake -G "Visual Studio 16 2019" -DEXAMPLES=ON ..

ESP32

Just place the library to your project components folder.

Setting up

  • Arduino Option 1 (with docs and tools)

    • Download source from https://github.com/lexus2k/tinyproto
    • Put the downloaded library content to Arduino/libraries/tinyproto folder
    • Restart the Arduino IDE
    • You will find the examples in the Arduino IDE under File->Examples->tinyproto
  • Arduino Option 2 (only library without docs)

    • Go to Arduino Library manager
    • Find and install tinyproto library
    • Restart the Arduino IDE
    • You will find the examples in the Arduino IDE under File->Examples->tinyproto
  • ESP32 IDF

  • Linux

  • Plain AVR

  • Python

Using tiny_loopback tool

  • Connect your Arduino board to PC

  • Run your sketch or tinylight_loopback

  • Compile tiny_loopback tool

  • Run tiny_loopback tool: ./bld/tiny_loopback -p /dev/ttyUSB0 -t light -g -c 8 -a -r

  • Connect your Arduino board to PC

  • Run your sketch or tinyfd_loopback

  • Compile tiny_loopback tool

  • Run tiny_loopback tool: ./bld/tiny_loopback -p /dev/ttyUSB0 -t fd -c 8 -w 3 -g -a -r

For more information about this library, please, visit https://github.com/lexus2k/tinyproto. Doxygen documentation can be found at Codedocs xyz site. If you found any problem or have any idea, please, report to Issues section. If you find the library useful and want to support future development, you may contact me.

Paypal Bitcoin Etherium
Not available BTC
3CtUY6Ag2zsvm1JyqeeKeK8kjdG7Tnjr5W
ETH
0x20608A71470Bc84a3232621819f578Fb9C02A460

License

The project is released under dual license: GPLv3, or Commercial license.

Copyright 2016-2022 (C) Alexey Dynda

This file is part of Tiny Protocol Library.

GNU General Public License Usage

Protocol Library is free software: you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

Protocol Library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

You should have received a copy of the GNU Lesser General Public License along with Protocol Library. If not, see http://www.gnu.org/licenses/.

Commercial License Usage

Licensees holding valid commercial Tiny Protocol licenses may use this file in accordance with the commercial license agreement provided in accordance with the terms contained in a written agreement between you and Alexey Dynda. For further information contact via email on github account.

tinyproto's People

Contributors

chenlijun99 avatar lekoook avatar lexus2k avatar nmaas87 avatar r2r0 avatar tpopiel avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tinyproto's Issues

What flags needed in build when using tinyproto as library

Hi,

I could not figure out why tinyproto was behaving much differently in my application that in the loopback example and I came to conclusion that this is about my project build flags. How should I build my project to have tinyproto working correctly inside? Currently when I run tinyproto functions I get e.g. memory related exceptions, or just application crashes. It does not happen why I do not use tinyproto, also it does not happen in loopback example build with your makefiles. I copied loopback code directly to mine project and it's crashing not specifying the reason.

Regards

Connection is silently re-established

If a peer disconnects (e.g. due to reset) and then quickly re-connects (e.g. right after boot) TinyProto should notify via on_connect_event_cb that a new connection has been established, instead of proceeding as if no disconnection has happened.

riscv processor based esp32 series support(and more)

When trying to build for esp32c3(or any riscv processor based esp32) tiny_types.c and tiny_types.h don't include the esp32/esp32_hal.inl(or .h, I don't remember).
This can be fixed by adding to the line "#elif defined(__XTENSA__)" " || defined(__riscv)".

Should I create a new branch with this simple fix and do a pull request?

Simple build fails on Teensy 4.1

Hi, I am trying to build a simple tinyproto example on Teensy 4.1 using PlatformIO. I replaced tinyproto::Light to Tiny::ProtoLight and tinyproto::StaticPacket<256> to Tiny::Packet<256> because tinyproto namespace didn't exist. The error is undefined reference to millis.

My main.cpp code looks as follows:

#include <Arduino.h>
#include <TinyProtocol.h>

/* Creating protocol object is simple */
Tiny::ProtoLight  proto;

void setup()
{
    /* No timeout, since we want non-blocking UART operations. */
    Serial.setTimeout(10);
    /* Initialize serial protocol for test purposes */
    Serial.begin(115200);
    /* Lets use 8-bit checksum, available on all platforms */
    proto.enableCheckSum();
    /* Redirect all protocol communication to Serial0 UART */
    proto.beginToSerial();
}

/* Specify buffer for packets to send and receive */
Tiny::Packet<256> packet;

void loop()
{
    if (Serial.available())
    {
        int len = proto.read( packet );
        if (len > 0)
        {
            /* Send message back */
            proto.write( packet );
        }
    }
}

My platformio.ini file is simple:

[env:teensy41]
platform = teensy
board = teensy41
framework = arduino
lib_deps = 
	lexus2k/tinyproto@^0.9.3

Here is the full error:

> Executing task: C:\Users\Robotic\.platformio\penv\Scripts\platformio.exe run --environment teensy41 <

Processing teensy41 (platform: teensy; board: teensy41; framework: arduino)
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------Verbose mode can be enabled via `-v, --verbose` option
CONFIGURATION: https://docs.platformio.org/page/boards/teensy/teensy41.html
PLATFORM: Teensy (4.15.0) > Teensy 4.1
HARDWARE: IMXRT1062 600MHz, 512KB RAM, 7.75MB Flash
DEBUG: Current (jlink) External (jlink)
PACKAGES:
 - framework-arduinoteensy 1.156.0 (1.56) 
 - tool-teensy 1.155.0 (1.55)
 - toolchain-gccarmnoneeabi 1.50401.190816 (5.4.1)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 93 compatible libraries
Scanning dependencies...
Dependency Graph
|-- <tinyproto> 0.9.3
Building in release mode
Compiling .pio\build\teensy41\src\main.cpp.o
In file included from .pio\libdeps\teensy41\tinyproto\src/TinyProtocol.h:32:0,
                 from src\main.cpp:2:
.pio\libdeps\teensy41\tinyproto\src/TinyProtocolHd.h: In lambda function:
.pio\libdeps\teensy41\tinyproto\src/TinyProtocolHd.h:97:90: warning: invalid conversion from 'uint8_t* {aka unsigned char*}' to 'char*' [-fpermissive]
                [](void *p, void *b, int s)->int { return Serial.readBytes((uint8_t *)b, s); });
                                                                                          ^
In file included from C:\Users\Robotic\.platformio\packages\framework-arduinoteensy\cores\teensy4/WProgram.h:51:0,
                 from C:\Users\Robotic\.platformio\packages\framework-arduinoteensy\cores\teensy4/Arduino.h:6,
                 from src\main.cpp:1:
C:\Users\Robotic\.platformio\packages\framework-arduinoteensy\cores\teensy4/usb_serial.h:113:9: note:   initializing argument 1 of 'size_t usb_serial_class::readBytes(char*, size_t)'
  size_t readBytes(char *buffer, size_t length) {
         ^
In file included from .pio\libdeps\teensy41\tinyproto\src/TinyProtocol.h:33:0,
                 from src\main.cpp:2:
.pio\libdeps\teensy41\tinyproto\src/TinyProtocolFd.h: In lambda function:
.pio\libdeps\teensy41\tinyproto\src/TinyProtocolFd.h:91:90: warning: invalid conversion from 'uint8_t* {aka unsigned char*}' to 'char*' [-fpermissive]
                [](void *p, void *b, int s)->int { return Serial.readBytes((uint8_t *)b, s); });
                                                                                          ^
In file included from C:\Users\Robotic\.platformio\packages\framework-arduinoteensy\cores\teensy4/WProgram.h:51:0,
                 from C:\Users\Robotic\.platformio\packages\framework-arduinoteensy\cores\teensy4/Arduino.h:6,
                 from src\main.cpp:1:
C:\Users\Robotic\.platformio\packages\framework-arduinoteensy\cores\teensy4/usb_serial.h:113:9: note:   initializing argument 1 of 'size_t usb_serial_class::readBytes(char*, size_t)'
  size_t readBytes(char *buffer, size_t length) {
         ^
Linking .pio\build\teensy41\firmware.elf
.pio\build\teensy41\libbbb\libtinyproto.a(tiny_types.c.o): In function `tiny_events_wait':
tiny_types.c:(.text.tiny_events_wait+0x10): undefined reference to `millis'
tiny_types.c:(.text.tiny_events_wait+0x54): undefined reference to `millis'
.pio\build\teensy41\libbbb\libtinyproto.a(tiny_types.c.o): In function `tiny_millis':
tiny_types.c:(.text.tiny_millis+0x0): undefined reference to `millis'
collect2.exe: error: ld returned 1 exit status
*** [.pio\build\teensy41\firmware.elf] Error 1
================================================================================================================= [FAILED] Took 1.03 seconds =================================================================================================================
The terminal process "C:\Users\Robotic\.platformio\penv\Scripts\platformio.exe 'run', '--environment', 'teensy41'" terminated with exit code: 1.

Terminal will be reused by tasks, press any key to close it.

I'd appreciate any help in fixing this error. Thanks in advance.

IFd class does not allow station address intiailization.

We use IFd for both side of connection and IFd initializes station address with 0 (HDLC_E_BIT).
In the ABM mode with 2 stations, it works somehow - both stations have address 0 and they send frames to peer (also) with address 0.
But if we will try to add another station then we will need to assign addresses explicitly which is not possible with current IFd implementation.

Is the above reasoning valid?
If yes, ten would it be possible to add to IFd appropriate update for station address initialization?

Questions on connection management

Hi @lexus2k & contributors.
First, thank you for this excellent library!

I'm trying to implement a socket-like wrapper of tinyproto FD for my project, both in C/C++ and in Python. What I'm having a hard time doing is the connection establishment. In particular I could not find these things:

  • on_connect/on_disconnect callback. I'm aware that there is the tiny_fd_get_status function, but I would like to avoid polling.
  • Even if I used tiny_fd_get_status AFAIK this function is missing in the Python wrapper.

Have these things ever been considered? Or are there valid reasons not to have them? Would you accept a PR?

Multidrop / Master+Slaves on RS485 or similar

Is there any possibility of using this between 2 or more nodes in a small network, with simple addressing? Or any suggestions of a protocol that is light and can support more than 2 nodes?

Add higher level of Tiny Protocol to support request/responses functionality

Add higher level of Tiny Protocol to support request/responses functionality.
At present tiny_layer2 supports only sending packets without acknowledges.
The main idea is to add next level of Tiny Protocol stack to add such communication.
It is assumed that uid support of low level will be used to implement next level of communication.

Examples vs Tinyproto documentation

Hi,

I'm a bit noob with C++ and have some difficulties to understand how to use Tinyproto.

In the Tinyproto documentation there is mentioned that higher level protocols needs 4 user defined callbacks.

  • int write_func_cb(void *user_data, const void *data, int len);
  • int read_func_cb(void *user_data, void *data, int len);
  • int on_frame_read(void *user_data, void *data, int len);
  • int on_frame_sent(void *user_data, const void *data, int len);

Is the documentation (in https://codedocs.xyz/lexus2k/tinyproto/index.html) out of sync with the main branch because in the examples I find references to the on_frame_read and on_frame_sentfunctions, but not to the write_func_cb or read_func_cbfunctions?

Examples where on_frame_read or on_frame_sent are used:
./examples/linux/hdlc_demo_multithread/hdlc_demo_multithread.cpp
./examples/esp32_idf/spi/master/main/app_main.c
./examples/esp32_idf/spi/slave/main/app_main.c

Or have I missed something essential?

-timo-

Pyserial and tinyproto example

Hi @lexus2k ,
I am trying to run the sketch from python3 and pyserial.
Here is my code.

import tinyproto
import time
import serial

p = tinyproto.Hdlc()
ser = serial.Serial('/dev/ttyS0', 57600, parity = serial.PARITY_NONE, timeout= 0)

def on_read(a):
    print("Received bytes: " + ','.join( [ "{:#x}".format(x) for x in a ] ) )

def on_send(b):
    print("Callback")
    print("Wrote bytes: " + ','.join( [ "{:#x}".format(x) for x in b ] ) )

# setup protocol
p.on_read = on_read
p.on_send = on_send
p.begin()

# provide rx bytes to the protocol, obtained from hardware rx channel
p.rx( bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E ]) )

# buf = tinyproto.
# bytearray buf = [ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E  ]

while True:
    buf = bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E  ])
    print("Writing byte array to serial")
    print(p.tx(buf))
    ser.write(p.tx(buf))

I get 0 from the p.tx function assumed packet cannot be formed.
There is no more explicit python doc except the wrapper class.
Please let me know if I am doing anything wrong or an proper
example to achieve full duplex communcation ( Threaded ).

Thank you,
Sasank Panda

Integrate 1-Wire protocol to tinyproto library

Maybe, it will be good option to integrate software 1-Wire protocol to tinyproto library to allow communication between system modules over single GPIO line.
1-Wire protocol is well-known and well-documented.

Pi Pico serial1 problem

Hi @lexus2k & contributors,
Firstly thank you this qualified work. I would like to use this protocol with Pi Pico. But I have some issues about Serial1 When used on Pi Pico. My hardware setup is like this with an Arduino Nano board.
Tiny
I ran loopback code with hd parameter on OrangePi SBC, RX and TX leds blinked both modules(USB to RS485 and TTL to RS485). It works without problem. But only When I changed Pi Pico instead of Nano. It isn't work. TX led didn't blink although RX led blinked, on TTL to RS485. But When I try Serial1.print("Hello") TX led is open. My Pi Pico code is like this.

#include <Arduino.h>
#include <TinyProtocol.h>

void onFrameIn(uint8_t *buf, int len);

uint8_t buffer[64];

Tiny::ProtoHd  proto(buffer, sizeof(buffer), onFrameIn);

void onFrameIn(uint8_t *buf, int len)
{
    uint8_t tx_buffer[64];
    memcpy( tx_buffer, buf, len );
    proto.write( (char *)tx_buffer, len );
}

void setup() {
    Serial1.setTimeout(10);
    Serial1.begin(115200);
    proto.enableCheckSum();
    proto.beginToSerial();
}

void loop()
{
  proto.run();
  //Serial1.print("Hello"); //Open TX led
}

Also I tried USB communication between Pi Pico and Orange Pi with Tiny FD. It works. I defined Serial.begin(115200) for USB communication on Pi Pico.

Arduino Core basic documentation for Nano RP2040 same as Pi Pico:
https://docs.arduino.cc/tutorials/nano-rp2040-connect/rp2040-01-technical-reference

Best regards,
Ahmet

Will Not Compile on Cortex M0

src/TinyProtocol.h:266:73: error: call of overloaded 'put(int)' is ambiguous

I'm trying to implement reliable serial communications between two Cortex MO's using Serial 1.

I'm assuming I'm getting this error because of the 32 bit platform. I'm not sure where the ambiguity is between the various forms of put though.

Any suggestions?

License?

I stumbled upon your project and was intrigued. I have both hobby and work projects where I need a simple, robust protocol, to communicate messages over a serial/SPI interface. It appears you put a lot of effort into tinyproto and I suspect it would be useful for many of my projects. My enthusiasm, however, significantly diminished when I noticed the license: LGPLv3. Although fine for home/hobby projects, it's not something I could integrate in work/commercial projects due to the copy-left requirements. For deeply embedded projects it's impossible to separate tinyproto code from a proprietary application. For this reason MIT/BSD/Apache licenses are more desirable and easier to work with.

If you don't have strong feelings about LGPLv3 and copy-left issues, I wonder if you would consider changing the license to one that is more permissible? I tend to avoid expending time and energy learning and integrating a library if I have to worry about where I can eventually utilize it. With that said, since you wrote the code, you get to chose where/how people use it. I just wanted to make you aware, in case you weren't, how your license choice impacts the adoption by a broader audience of developers. Perhaps this is as intended . . . but I hope you might reconsider?

Regardles, thanks for making it available!

Link management re-sync timeout

Hello,

I'm using the library working in FD mode at both sides: device and PC application connected over serial interface. Everything works fine when the application on PC is up and running and HDLC link between node is alive, but if I close the link (e.g.: by closing the application) I need to wait 5 seconds till the device is able to establish new HDLC link with the PC application. I wonder if it is possible with the tinyproto following:

  1. Get current HDLC link status.
  2. Close HDLC link (the node in such case should send U-Frame Disconnect DISC command).
  3. Reset HDLC link (the node in such case should send U-Frame Reset RSET command).

Best regards,
Tomek

Unreliable communication with tinyproto

I am unable to get reliable data transmission
at a bit higher baud rate: 115200, 57600.
Enabled crccheck16() on both sides.

Send :
Rpi python

ser = serial.Serial('/dev/ttyS0', 115200, parity = serial.PARITY_NONE, timeout= 0)

while True:
    buf = bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x40, 0x7E  ])
    buf = "SAM\0"
    buf = "HELLO-WORLD\0"
    print("Writing byte array to serial")
    print(p.put(buf)) #bytearray([ 0x7E, 0xFF, 0x3F, 0xF3, 0x39, 0x7E  ])
    # print(p.tx())
    ser.write(p.tx())

Receive:
Arduino

#include <TinyProtocol.h>
#include <SoftwareSerial.h>

int buff[256];
tinyproto::Hdlc proto_hdlc(buff, 256);
SoftwareSerial mySerial(A3, A2);


void hdlcRecieveCallback(tinyproto::IPacket &pkt)
{
    //Recive and process packets
    Serial.println("HDLC Recieve callback");
    Serial.println(pkt.getString());
    Serial.println("Recieved data size : " + String(pkt.size()));
    Serial.println("Max buffer size : " + String(pkt.maxSize()));
    Serial.println("=========================");
}
void setup() {
    /* No timeout, since we want non-blocking UART operations. */
    Serial.setTimeout(0);
    mySerial.setTimeout(0);
    /* Initialize serial protocol for test purposes */
    Serial.begin(115200);
    mySerial.begin(9600);
    proto_hdlc.setReceiveCallback(hdlcRecieveCallback);
    proto_hdlc.begin();
    proto_hdlc.enableCrc16();
}


void loop()
{

    size_t len =   mySerial.available();  
    // char bytes[128];
    if (len) {
        uint8_t bytes = mySerial.read();
        proto_hdlc.run_rx( &bytes, 1 );
        // mySerial.readBytes(bytes,len > 128 ? 128 : len);
        // proto_hdlc.run_rx( bytes, len > 128 ? 128 : len ); // run FD protocol parser to process data received from the channel
    }
}

Baud: 115200
Zero reliability

2021-07-30_02-28-54

More reliable than above.
But still unacceptable to the requirement.
Baud : 57600
2021-07-30_02-31-21

Most reliable
Still too many error packets
Baud : 9600
2021-07-30_02-37-51

Am I doing something wrong.
or the observation is correct.

Thank you in advance.
Sasank Panda

Question to usage of tinyproto

Dear Aleksei,

I have multiple question regarding the usage of tinyproto (and probably some thinking errors on my side :)), which I wanted to clarify.

I plan on building the here pictured system:

SerialMux

Basically a Command PC with a Python Frontend, delivering data to an tinyproto instance to be sent over an serial link via radio. The commands shall be received via the main serial port of an AT SAM D21E18. Due to the fact that I need to have acknowledgments of the sucessful recveie of the command as well as error correction, I think I will need to use the TinyProtocol Half Duplex version. The received commands contain the number of the target subsystem, which will be then be evaluated by the AT SAM, switching the MUX to the correct subystem and passing on the received and "unpacked" command to one of several, serial-linked subsystems via the second serial interface. In answer to this command, the subystem can then create a new message, which will be passed via MUX and Serial2 to the AT SAM, then in the ideal case be packaged by tinyproto and sent back via Serial1 to the Command PC.

Questions:
1.) Is the TinyProtocol HD the correct choice for what I want to do? I bascially would need to both send and receive tinyproto frames via the same Serial Interface - or is it only possible to i.e. sent from the Command PC to the Embedded System as a secured tinyproto message and then do the answer out-of-scope of tinyproto?

2.) I tried to use the current master branch of tinyproto (directly checked out from Github) and could successfully test the sketch_sender and sketch_listener on Arduino UNOs, but not the sketch_hd_sender and sketch_hd_listener. Compilation works, but the UNOs do not seem to do anything.

3.) Trying to compile the sketch_hd_sender sketch for ESP32 with the latest ESP32 for Arduino package (1.0.2) gives the error call of overloaded 'millis()' is ambiguous


In file included from X:\sketchbook\libraries\tinyproto\src/proto/hal/tiny_types.h:36:0,

                 from X:\sketchbook\libraries\tinyproto\src/proto/hdlc/tiny_hdlc.h:3,

                 from X:\sketchbook\libraries\tinyproto\src/proto/half_duplex/tiny_hd.h:35,

                 from X:\sketchbook\libraries\tinyproto\src/TinyProtocolHd.h:32,

                 from X:\sketchbook\libraries\tinyproto\examples\sketch_hd_sender\sketch_hd_sender.ino:5:

X:\sketchbook\libraries\tinyproto\src/proto/hal/esp32/tiny_defines.h: In function 'uint32_t millis()':

X:\sketchbook\libraries\tinyproto\src/proto/hal/esp32/tiny_defines.h:59:31: error: conflicting declaration of C function 'uint32_t millis()'

 static inline uint32_t millis() {   return (uint32_t)(esp_timer_get_time()/1000); }

                               ^

In file included from X:\arduino\portable\packages\esp32\hardware\esp32\1.0.2\cores\esp32/Arduino.h:35:0,

                 from X:\arduino\build\sketch\sketch_hd_sender.ino.cpp:1:

X:\arduino\portable\packages\esp32\hardware\esp32\1.0.2\cores\esp32/esp32-hal.h:92:15: note: previous declaration 'long unsigned int millis()'

 unsigned long millis();

               ^

X:\sketchbook\libraries\tinyproto\examples\sketch_hd_sender\sketch_hd_sender.ino: In function 'void loop()':

sketch_hd_sender:39:34: error: call of overloaded 'millis()' is ambiguous

     unsigned long start = millis();

                                  ^

In file included from X:\arduino\portable\packages\esp32\hardware\esp32\1.0.2\cores\esp32/Arduino.h:35:0,

                 from X:\arduino\build\sketch\sketch_hd_sender.ino.cpp:1:

X:\arduino\portable\packages\esp32\hardware\esp32\1.0.2\cores\esp32/esp32-hal.h:92:15: note: candidate: long unsigned int millis()

 unsigned long millis();

               ^

In file included from X:\sketchbook\libraries\tinyproto\src/proto/hal/tiny_types.h:36:0,

                 from X:\sketchbook\libraries\tinyproto\src/proto/hdlc/tiny_hdlc.h:3,

                 from X:\sketchbook\libraries\tinyproto\src/proto/half_duplex/tiny_hd.h:35,

                 from X:\sketchbook\libraries\tinyproto\src/TinyProtocolHd.h:32,

                 from X:\sketchbook\libraries\tinyproto\examples\sketch_hd_sender\sketch_hd_sender.ino:5:

X:\sketchbook\libraries\tinyproto\src/proto/hal/esp32/tiny_defines.h:59:24: note: candidate: uint32_t millis()

 static inline uint32_t millis() {   return (uint32_t)(esp_timer_get_time()/1000); }

                        ^

sketch_hd_sender:40:19: error: call of overloaded 'millis()' is ambiguous

     while (millis() - start < 1000)

                   ^

In file included from X:\arduino\portable\packages\esp32\hardware\esp32\1.0.2\cores\esp32/Arduino.h:35:0,

                 from X:\arduino\build\sketch\sketch_hd_sender.ino.cpp:1:

X:\arduino\portable\packages\esp32\hardware\esp32\1.0.2\cores\esp32/esp32-hal.h:92:15: note: candidate: long unsigned int millis()

 unsigned long millis();

               ^

In file included from X:\sketchbook\libraries\tinyproto\src/proto/hal/tiny_types.h:36:0,

                 from X:\sketchbook\libraries\tinyproto\src/proto/hdlc/tiny_hdlc.h:3,

                 from X:\sketchbook\libraries\tinyproto\src/proto/half_duplex/tiny_hd.h:35,

                 from X:\sketchbook\libraries\tinyproto\src/TinyProtocolHd.h:32,

                 from X:\sketchbook\libraries\tinyproto\examples\sketch_hd_sender\sketch_hd_sender.ino:5:

X:\sketchbook\libraries\tinyproto\src/proto/hal/esp32/tiny_defines.h:59:24: note: candidate: uint32_t millis()

 static inline uint32_t millis() {   return (uint32_t)(esp_timer_get_time()/1000); }

                        ^

Bibliothek tinyproto in Version 0.7.0 im Ordner: X:\sketchbook\libraries\tinyproto  wird verwendet
exit status 1
call of overloaded 'millis()' is ambiguous

Thank you very much for your help and have an awesome weekend :)

Nico

proto.run_rx hangs indefinitely in Arduino

Hi @lexus2k,
The communication with Rpi, and Arduino
could happen over the TinyLight protocol.

However, I want to use HDLC in Arduino
as I am using from Pi side.
Now, I have spent some hours now, debugging
but it seems like there is some issue with the
method

tinyproto::Hldc->run_rx()

I have no debugging environment set up so it was
hard for me to debug further inside the library.
But I have done some preliminary analysis.
See sample code.

FD Example

#include "TinyProtocol.h"
tinyproto::Fd<64>  proto;

void onReceive(void *udata, tinyproto::IPacket &pkt) {
    // Process message here, you can do with the message, what you need
    // Let's send it back to the sender ;)
    Serial.println("Callback");
    if ( proto.write(pkt) == TINY_ERR_TIMEOUT ) {
        // Do what you need to do if looping back failed on timeout.
        // But never use blocking operations inside callback
    }
}

void setup() {
    Serial.begin(9600);
    Serial.println("Full duplex example");
    // ...
    // Here we say FD protocol object, which callback to call once new msg is received
    proto.setReceiveCallback( onReceive );
    proto.begin();
}

void loop() {
    if (Serial.available()) {
      Serial.println(Serial.available());
        uint8_t byte = Serial.read();
        // proto.run_rx( &byte, 1 ); // run FD protocol parser to process data received from the channel 

**The run_rx doesn't return and hangs indefinitely. I can see the buffer available() when this line is commented out. 
Also, the callback method onRecieve is never called even if valid packets are sent.**

    }
    // uint8_t byte;
    // if ( proto.run_tx( &byte, 1 ) == 1 ) // FD protocol fills buffer with data, we need to send to the channel
    // {
    //     while ( Serial.write( byte ) == 0 ); // Just send the data
    // }
}

HDLC Example :

#include <TinyProtocol.h>
#include <SoftwareSerial.h>
/* Creating protocol object is simple */
//tinyproto::Light  proto;
//tinyproto::Packet<16> hdlc_packet;

int buff[16];
tinyproto::Hdlc proto_hdlc(buff, 16);

SoftwareSerial mySerial(A3, A2);


void hdlcRecieveCallback(tinyproto::IPacket &pkt)
{
    //Recive and process packets
    Serial.println("HDLC Recieve callback");
    // Serial.println(pkt.data());
    Serial.println("=========================");
}
void setup() {
    /* No timeout, since we want non-blocking UART operations. */
    Serial.setTimeout(0);
    mySerial.setTimeout(0);
    /* Initialize serial protocol for test purposes */
    Serial.begin(9600);
    mySerial.begin(57600);
    proto_hdlc.setReceiveCallback(hdlcRecieveCallback);
    proto_hdlc.begin();
    // proto_hdlc.begin([](void *p, const void *b, int s) -> int { return mySerial.write((const uint8_t *)b, s); },
    //           [](void *p, void *b, int s) -> int { return mySerial.readBytes((uint8_t *)b, s); });
    proto_hdlc.disableCrc();
}



void loop()
{

    if (mySerial.available()) {
        Serial.println(String(mySerial.available()));
        uint8_t byte = mySerial.read();
        //Serial.println(proto_hdlc.run_rx( &byte, 1 )); // run FD protocol parser to process data received from the channel

      **[ HERE ] : Blocks indefinitely and the call back is never called.** 
    }
    // uint8_t byte;
    // if ( proto_hdlc.run_tx( &byte, 1 ) == 1 ) //FD protocol fills buffer with data, we need to send to the channel
    // {
    //     while ( mySerial.write( byte ) == 0 ); // Just send the data
    // }
}

Please correct me I am doing something wrong.
Or is there something that needs correction.
Thank you.
Sasank Panda

Add support for ESP32 "BluetoothSerial.h"?

In case it would help, I can post a simple serial string loopback test using PyBluez on Python and BluetoothSerial.h on esp32.

I'd like to do the rest of the work, too, but the code in this library is a bit over my head and would take more work than I think I can afford right now.

TinyProtocolStream - encapsulate tinyproto into Arduino Stream for ease of use

I need a reliable Serial connection between two Arduinos, and found your project. I created an Arduino Stream class that inherits from IFd and it's getting close to reliable at high throughput as tested through a loopback test running between two Teensy 4.1s.

https://github.com/embedded-creations/tinyproto/tree/stream

Take a look at this example to see how simple it is to use (just a few lines reference tinyproto in the sketch, and the API is nearly identical to a Stream object like Serial which makes it easy for a new user):

https://github.com/embedded-creations/tinyproto/blob/stream/examples/arduino_generic/TinyProtocolStreamTest/TinyProtocolStreamTest.ino

Some questions:

  • Would you like to include FdStream in the library once it's finished?
  • It's difficult to see the status of the library, have you thought about how to improve the API in this regard? I added a getStatus() method to access tiny_fd_get_status(), but beyond that I'd like to know if I lost any data so I can reset the protocol running on top of tinyproto.
  • Have you run any loopback tests that validate data? I didn't see any in the repo and found packets were getting dropped unless I left a couple of I slots free when running at high throughput. This may go overlooked if you're just counting packets received and not validating.
  • Do you have any Teensy devices? I'm using the Teensy 4 as it has multiple hardware serial ports, and I can enable multiple USB serial ports so I can get the tinyproto debug output on one, and sketch data on another. I had to make some HAL tweaks to get this to work, which I can contribute if it helps you.

Arduino due doesn't seem to work

I am trying to see how the library works and I am trying to see what happens onReceive by trying to see the output through another serial port (arduino Due), this would be the code. Is there something I'm doing wrong or don't know? Thank you

#include <TinyProtocol.h>
// We need this hack for very small controllers.
#include <proto/fd/tiny_fd_int.h>

/* Creating protocol object is simple. Lets define 48 bytes as maximum. *
   size for the packet and use 4 packets in outgoing queue.             */
tinyproto::Fd<FD_MIN_BUF_SIZE(48, 4)>  proto;

void onReceive(void *udata, tinyproto::IPacket &pkt)
{
  Serial3.println("-----------------");
//  for (int i = 0; i < pkt.size(); i++)
//  {
//    Serial3.print(pkt.getByte());
//    Serial3.print(" ");
//  }
//  Serial3.println();
  
  if (proto.write(pkt) == TINY_ERR_TIMEOUT )
  {
    // Do what you need to do if there is no place to put new frame to.
    // But never use blocking operations inside callback
  }
}

void setup()
{
  /* No timeout, since we want non-blocking UART operations. */
  Serial.setTimeout(0);
  /* Initialize serial protocol for test purposes */
  Serial.begin(115200);
  /* Lets use 8-bit checksum, available on all platforms */
  proto.enableCheckSum();
  /* Lets process all incoming frames */
  proto.setReceiveCallback( onReceive );
  /* Redirect all protocol communication to Serial0 UART */
  proto.begin();

  Serial3.begin(115200);
  Serial3.println("START");
}

void loop()
{
  if (Serial.available())
  {
    Serial3.write(".");
    Serial3.flush();
    proto.run_rx([](void *p, void *b, int s)->int { return Serial.readBytes((uint8_t *)b, s); });
  }
  proto.run_tx([](void *p, const void *b, int s)->int { return Serial.write((const uint8_t *)b, s); });
}

Software serial support

Hi @lexus2k & contributors,
I have a situation where I want to reliably send high-speed data from RPi and Arduino.
The problem is I am using an Arduino Nano which has one HW Serial.
The problem is I cannot connect the RPi to the HW Serial because it creates issues in uploading
sketches with the Serial USB, so I am using Software Serial to communicate with the Pi.
Now exploring very few solutions I stumped across this beautiful library as it will serve the purpose
of the Pi communicating with TinyProto in python as well.
But I see now the library uses only Hardware Serial library and provides hardware redirects
beginToSerialX() wehre X-> No of serial ports.
Is there any issue with adding something like
beginToSoftwareSerial(Rx, Tx). and begin with SoftwareSerial.
Please let me know if is possible through some code hack in the stable version.
I would really like to see this as an official feature.
Let me know what you think about it.

Thank you,
Sasank Panda

Null dereferencing, examples to small and not working.

Hi,

At hdlc_run_rx_until_read() there is no check for ( readcb != nullptr ) before calling it, so program is unexpectedly crashing when it's not initialized. It could be at least checked with assertion to help debugging.
And examples are too short, I think it's required to call at least proto.begin() to begin using proto( to avoid null dereferencing ), so why it's not mentioned in examples? They would be more helpful then.

Anyway, I'm starting to work with your library and it looks like good job.

Regards,
Zataras

Empty payload frame cannot be parsed as a Tiny::Packet

When I tried to parse a frame with no payload just the UID using the regular buffer (technique no.1) it seems working normally. While when I tried using Tiny's Packet class (technique no.2) it didn't seems to work, and it just ignores the whole frame. Am I missing any information?

Hint: I am using the following functions in both techniques to initialize hdlc instance

hdlc.beginToSerial(); /* using Arduino's Serial Instance to parse and send data */
hdlc.enableCrc16(); /* Enabling CRC16 to frames */
hdlc.enableUid(); / * Enabling Uid for Tiny::Packet */

Technique no. 1

char buff[100]={0};
Tiny::Proto hdlc;
int len = 0;
len = hdlc.read(buff,sizeof(buff),TINY_FLAG_NO_WAIT);
if (len > 0)
{
    /* Process the Data */
}

Technique no. 2

char buff[100]={0};
Tiny::Proto hdlc;
Tiny::Packet myPacket(buff,sizeof(buff));
int len = 0;
len = hdlc.read(myPacket, TINY_FLAG_NO_WAIT);
if (len > 0)
{
    /* Process the Data */
}

Build with visual studio works, but unable to do a proto begin --> m_handle=null

Hi Team,

I have an issue when I m using the example project tiny_loopback build with Microsoft Visual Studio. Every time that I tried to start proto.begin(), it 's not possible to create the m_handle. It seems that program failed at :

/* Next we allocate some space for peer-related data */
ptr = TINY_ALIGN_BUFFER(ptr);
protocol->peers_count = peers_count;
protocol->peers = (tiny_fd_peer_info_t *)ptr;
protocol->next_peer = 0;
ptr += sizeof(tiny_fd_peer_info_t) * peers_count;

if ( ptr > (uint8_t *)init->buffer + init->buffer_size )
{
LOG(TINY_LOG_CRIT, "Out of provided memory: provided %i bytes, used %i bytes\n", init->buffer_size,
(int)(ptr - (uint8_t *)init->buffer));
return TINY_ERR_INVALID_DATA;
}

In tiny_fd.

The strange things, it's I have not the same behavior if I'm using clang ++ and ninja to build the project.

Is someone found this issue too?

Regards,
Julien R .

Unable to connect to tinyproto after second connection.

I found this issue when I ported the library to my target board, then I tried to reproduce this issue with your example code.

My Setup

I created virtual serial port and run 2 tiny_loopbacks example to communicate with each other. With your example I can close one program and restart it without any issue. But after I added couple of delay to generator side, I found the issue.

Virtual Serial port
socat -d -d pty,raw,echo=0 pty,raw,echo=0

Generator side
$TINYPROTO_EXAMPLE_PATH/linux/loopback/tiny_loopback -p /dev/pts/30 -g

Loopback side
$TINYPROTO_EXAMPLE_PATH/linux/loopback/tiny_loopback -p /dev/pts/31

Build

rm -rf build &&
mkdir build &&
cd build &&
cmake -DENABLE_FD_LOGS=ON -DEXAMPLES=ON ..
make

My edited code (tiny_loopback.cpp)
image

image

diff --git a/examples/linux/loopback/tiny_loopback.cpp b/examples/linux/loopback/tiny_loopback.cpp
index cf327fa..5176661 100644
--- a/examples/linux/loopback/tiny_loopback.cpp
+++ b/examples/linux/loopback/tiny_loopback.cpp
@@ -193,6 +193,8 @@ static int run_fd(tiny_serial_handle_t port)
     proto.setSendTimeout(s_generatorEnabled ? 1000 : 0);
     proto.setReceiveCallback(onReceiveFrameFd);
     proto.setSendCallback(onSendFrameFd);
+    proto.setConnectEventCallback([](void *userData, uint8_t addr, bool connected) -> void
+                                  { printf("peer %x, connected %d\n", addr, connected); });
     s_protoFd = &proto;
 
     proto.begin();
@@ -227,6 +229,8 @@ static int run_fd(tiny_serial_handle_t port)
             {
                 fprintf(stderr, "Failed to send packet\n");
             }
+
+            std::this_thread::sleep_for(std::chrono::milliseconds(1000));
         }
         else
         {

Result of first connection. (Left side is generator / Right side is loopback)
image

Result of second/third connection. (I close the generator side and restart it)
image
Generator side try to connect and then fail and the this loop happens repeatedly.

Do you have any clue or workaround for this issue?
Thanks.

Wrong alignement for function pointers (callbacks) causes CPU fault on Coretex-M4

I got usage fault beacuse of unaligned memory on STM32WB55 (Cortex-M4) when accessing structure fields holding function pointers (callbacks) i.e. function hdlc_ll_reset is failing at line: handle->rx.state = hdlc_ll_read_start;.

For confirmation I added (dirty hack) alignment to memory pointers used to initialize buf field in hdlc_ll_init_t and such change allows code to execute properly:
TinyProtocolFd.h:301

uint8_t m_data[S] __attribute__ ((aligned (8))) {};

TinyProtocolFd.h:317

    : IFd(new uint8_t[size+7], size+7)

tiny_fd.c:584:

uint32_t ptrValue = (uint32_t)ptr;
ptrValue = ((ptrValue + 7U) / 8U) * 8U;
_init.buf = (void*)ptrValue;

Compiler GCC 10.2.0

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.