clusterduck-protocol/clusterduck-protocol

Firmware for an ad-hoc mesh network of Internet-of-Things devices based on LoRa (Long Range radio) that can be deployed quickly and at low cost.

What is the CDP?

The ClusterDuck Protocol (CDP), an innovative open-source project under the Linux Foundation, represents a significant leap in IoT communications, harnessing the power of low bandwidth and low power such as LoRa technology. Distinct from LoRaWAN, this protocol utilizes LoRa's point-to-multipoint capabilities to facilitate robust communication in diverse environments.

The ClusterDuck Protocol was created by OWL Integrations (formerly Project OWL) to provide developers with an advanced library for crafting distributed communication networks, tailored for a variety of IoT applications. Since its creation, the vision for the ClusterDuck Protocol has grown beyond servicing natural disasters to large events (cellular congestion), sensor networks, and more.

How does it work?

The network is made up of multiple nodes called Ducks. There are 3 basic types of Ducks in a ClusterDuck Network: DuckLink, MamaDuck, and PapaDuck. DuckLinks serve as edge nodes that only transmit data. MamaDucks inherit the same functionality of the DuckLinks, but can also receive messages. This allows the MamaDucks to relay messages from DuckLinks and/or other MamaDucks along the network (towards the gateway). PapaDucks are the endpoint of the network where all of the data is collected and can be stored or pushed up into the cloud. (free tier cloud DMS platform in BETA! Reach out to us on Discord for more info)

Installation

Check out the Wiki to learn how to build your own ClusterDuck Network. Feel free to visit the ClusterDuck Protocol website for more information on projects that the community is building. You can reach out directly on our Discord Community for any questions and/or to work with the community.

To use the ClusterDuck Protocol follow the Installation Manual.

Testing

Starting with release 4.0.0 we have unit tests available with the PlatformIO test framework unity

Tests are located in the ClusterDuckProtocol/test folder. These tests are unit tests as they validate the CDP publicly accessible APIs. However they must be run on a device. This means you have to connect a device to your development machine and build the tests to run on the device. Platform IO test command will build, deploy and run the tests and report back the results on your terminal console.

Unit tests are great in detecting problems before they make it to the release, so it is important no only to run them to validate your changes, but to continuously update them.

PlatformIO test environments

The project platformio.ini defines environment configurations for supported boards. For example below is the configuration for tests on the Heltec LoRa v3 Arduino board

[env:test_heltec_wifi_lora_32_V3]
  test_build_src = yes
  test_filter = test_* 
  test_framework = unity
  platform = ${env:heltec_wifi_lora_32_V3.platform}
  board = ${env:heltec_wifi_lora_32_V3.board}
  framework = ${env:heltec_wifi_lora_32_V3.framework}
  monitor_speed = ${env.monitor_speed}
  build_src_filter  = +<./> +<./include> +<./include/boards>
  lib_deps =
      ${env.lib_deps} 
      ${env:heltec_wifi_lora_32_V3.lib_deps}
      ; why do I need to add the following libraries is a mystery to me
      SPI
      WIRE
      FS
      WIFI
      
  build_flags = 
      ${env.build_flags}
      -std=gnu++11
      -DUNIT_TEST

How to run the tests

Here are the steps to run the tests (on Linux or Mac OS). This assumes you have platformIO installed on your system.

For more details on setting up your development environment, please refer to the developer's guide.

PlatformIO CLI test command

# Open a terminal 
# goto the project root folder (where the platformio.ini is located)
$ cd ClusterDuckProtocol
$ platformio test -e test_heltec_wifi_lora_32_V3

Example output

Processing test_DuckUtils in test_heltec_wifi_lora_32_V3 environment
-------------------------------------------------------------------------------------------------------------------------------------
Building & Uploading...
Testing...
If you don't see any output for the first 10 secs, please reset board (press reset button)

test/test_DuckUtils/test_DuckUtils.cpp:181: test_DuckUtils_getCdpVersion                        [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:182: test_DuckUtils_toUpperCase                          [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:183: test_DuckUtils_stringToByteVector                   [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:184: test_DuckUtils_getRandomBytes                       [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:185: test_DuckUtils_createUuid_with_given_length         [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:186: test_DuckUtils_createUuid_with_default_length       [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:187: test_DuckUtils_convertToHex                         [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:188: test_DuckUtils_toString_printable_characters        [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:189: test_DuckUtils_toString_non_printable_characters    [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:190: test_DuckUtils_isEqual_true                         [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:191: test_DuckUtils_isEqual_false                        [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:192: test_DuckUtils_isEqual_false_with_different_sizes   [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:193: test_DuckUtils_toUint32                             [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:194: test_DuckUtils_saveWifiCredentials                  [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:195: test_DuckUtils_saveWifiCredentials_zero_length      [PASSED]
test/test_DuckUtils/test_DuckUtils.cpp:196: test_DuckUtils_loadWifiSsid                         [PASSED]
-- test_heltec_wifi_lora_32_V3:test_DuckUtils [PASSED] Took 23.34 seconds -------------------------------------------------------------

How to Contribute

We host a bi-weekly CDP Town Hall on Mondays at 2pm EST. The town hall is the place to get updates on protocol, get your questions about CDP answered, and discuss on-going projects. All the current projects is documented in a public roadmap in Github Projects.

Please read CONTRIBUTING.md for details on our Code of Conduct, the process for submitting ClusterDuck Protocol improvements, and how to join our town halls and livestreams.

This project is governed by its Technical Charter and led by its Technical Steering Committee.

Contributors

License

This project is licensed under the Apache 2 License - see the LICENSE file for details.

clusterduck-protocol's Issues

Support for ARDUINO MKR WAN 1310

There was a request to support this device that does not use an ESP32. https://store.arduino.cc/usa/mkr-wan-1310

Adopt an object-oriented approach for ducks

Based on the approach started by @Salzi in #37

The goal is to reduce the size of the codebase, eliminate duplication, and reduce the surface area for potential bugs. This can also help reduce the number of duck types (DuckLink vs Mama).

Over the Air (OTA) Updates

As the CDP evolves, it will be necessary to update already deployed devices with the new firmware. Currently, the only way to update devices is to physically plug the device into a computer and upload the new firmware. This is a very arduous process as existing devices are placed far apart and are in difficult to reach areas.

OTA would solve a lot of these issues by allowing devices to be updated without needing to physically interact with each individual device. This would save a lot of time and even allow for networks to be updated remotely.

OTA updating methods:

• WiFi

• Bluetooth

• Captive Portal

The most challenging:

• LoRa
  **One idea is to do partial updates. Binary patches vs transmitting whole binary

OTA updating over LoRa would allow for the whole network to be upgraded at the same time. This is especially useful for large scale deployments and devices in very hard to reach areas.

QA Process and Duck Telemetry Data

Currently, there is no standard testing routine or health checks for first-time boot and remote testing. QA testing has been carried out by sending a message using the default captive portal and watching for data on a duck hooked up to a computer serial monitor. While this process is somewhat effective, it is time-consuming when having to prepare multiple devices and does not return much telemetry data. The only automated health checks are the iAmAlive() “Health Quack” that runs every 30 minutes by default and a reboot() every 12 hours. However, these are the only checks that are run after a Duck has been deployed.

There are a couple of ways that might make an easier QA process. Some ideas were: using LEDs with light status patterns (some boards have onboard LEDs), automatic LoRa message on boot, use a phone app to connect to duck-wifi, and run a test routine or get device telemetry data.

Battery status, packet loss, and signal to noise ratio are also helpful data points.

PaPa Iridium Smart Backhaul V.2

Is your feature request related to a problem? Please describe.
I wouldn't say that is a problem it just is a revision to the current version that is out for PaPa Iridium. Currently, PaPa Iridium only uses the RockBlock Satellite Modem the Wifi upload part of the original PaPa was taken out. But as discussed with the community and Owl it can get quite expensive sending data over the Iridium Network. So a better and smarter solution needs to be put in place of V.1.

Describe the solution you'd like
So In order to resolve this issue, I am going to make a V.2 of PaPa Iridium so that when WiFi is available it will use it. But for whatever reason a storm hits, earthquake, grid/network infrastructure goes down. The PaPa Iridium would switch back to using the Iridium Sattelite Modem until WiFi comes back online.

• Add WiFi Data Upload Backin

• Add Smart Switch Feature

• Update Docuementation for V.2

Update Build-a-Duck Markdown to match current date and version

Describe the bug
The Markdown references March and a back-level version:

• Current: March 2020, v1.0.2
• Should be: July 2020, v1.1.6 (this is on the bottom of README.md)

We're over the LTFS bandwidth quota. Need to revert back to regular storage, at least for the README images

Describe the bug
We're over the LTFS bandwidth quota. Need to revert back to regular storage, at least for the README images

To Reproduce
Load the README

Expected behavior
Images to show up, but they are broken references to LTFS links

Additional context
Seems like this is a paid add on. Will need to explore alternatives.

Standardize Payload Data

Currently, it is difficult to differentiate between different sources of data through the payload. Each source has a different way of recording data and makes it difficult to run analytics on the returned data. Sources include:

• GPS

• Captive Portal

• Sensor

• Health

• DPS

One way to make this easier is by transmitting the payload in JSON. However, this can be an issue as JSONifying the payload can use up a lot of precious bytes needed to transmit data (especially in a disaster situation). To mitigate this, coded JSON templates could be used and be translated on the server-side. This is also a bit of an issue as it is a bit complex.

Another way to do this is by giving each source its own byte code. This could then be used on the PapaDuck to send data over different topics. This would also make it easier to store data coming from different sources.

Make CDP Board Independent

Currently, the CDP only supports ESP32 and is hardcoded to work with the SX1276 LoRa radio. While this system works well so far, we envision the CDP integrating on a large family of devices. We see other technologies offering CDP compatibility and potentially as a standard used in the future. This flexibility allows for more use cases and a wider range of operation.

The RadioLib library currently supports many different radios. However, the current CDP doesn’t make it that easy to switch between different architectures/modules. It was suggested that it might be easier to have a separate configuration file. If going down this route, it would be ideal to keep the configuration file out of the library folder if possible to make editing easier. Another idea to make configuration easier is by adding #defines for other development board profiles.

Hardcoded radio setting areas in ClusterDuck.cpp: Line 8, 79, 84, 89, 94, and 99.

Next steps:

• Supporting more of the radio modules supported by the RadioLib

• Creating a configuration file

If there is a radio that is not supported by RadioLib, please create a new issue and link it to this one.

Template for printing stickers for case?

Is your feature request related to a problem? Please describe.
Is there a doc/pdf for printing the stickers on the outside of the plastic cases?

Describe the solution you'd like
Share location to the file(s) used to print for outside of case.

Describe alternatives you've considered
Would be neat to add a 2d QR code that people could scan that would take them to the website.

Additional context
Might help with promotion/discussions/awareness.

PapaDuck example mqtt topic size off by 1 char

Describe the bug
MQTT was not working for me (ie my client was not matching on the given topic

"iot-2/evt/status/fmt/json" or "iot-2/evt/health/fmt/json". Turns out the topic was being shortend to "iot-2/evt/status/fmt/jso" as the null terminator was missing in "topic"

In my private branch, I changed (line 193)
int len = loc.length(); to
int len = (loc.length() + 1) and now my client matches on the correct topic stream.

I also ran tcpdump on my mqtt broker and verified the topic is now the full topic

Add Encryption

Under normal testing purposes, protecting data over the CDP network is not an issue. However, in a natural disaster situation, personal information might be required to be shared for emergency personnel to respond to the situation. Therefore, the CDP needs the capability to protect the information until it reaches the first responders. This is also important for protecting information sent over different networks.

There are some challenges for security as a distributed network. How do we prevent someone from spoofing a Duck on a secured network? How do we prevent malicious data being transmitted over the network? How do we recognize Ducks in the same “family”?

Some approaches we have looked into in the past:

• 256-bit encryption
• Byte-codes (like a code/sync word)

Ghost Packets

Describe the bug

when looking at the serial log of a mamaduck, there are currently "ghost packets" that are received "right after" sending a packet.

Additional context

@nfeuer confirmed seeing these too, so it doesnt seem to be a local issue.

hexprinting both sent and received pkts shows the ghosts clearly contain some info from the pkts sent, but are mostly garbled.

watching with a receive-only duck does not show these ghosts, so they are probably mama-internal.

since they appeared only after switching to radiolib, it is more likely to be a software issue than an actual RF or hardware one.

Suggestions

next step is creating a standalone sketch demonstrating the problem.

my plan is to build that based on https://github.com/gozu42/LoRaDumper (which uses the old lora lib) by adding a send component, then switch to radiolib in a "lots of defines" way (so the same "app" can be switched between the two libs easily) to see that the issue is really lora-lib specific.

this should allow to either find the issue "on the way" or have a good repro/demo sketch for filing an issue with radiolib.

Since this is mostly debug work, i do not expect any merge-conflict potential.

Document the solar panel/upgraded case changes

Is your feature request related to a problem? Please describe.
There was a video update to changing the case from the small clear/plastic case to something larger that has a larger battery and solar panel. Can the parts be shared? (hopefully with where to source them, too like Amazon links)

Describe the solution you'd like
Update the doc (either README.md and/or PDF with updated parts)

Describe alternatives you've considered
Documenting on this project's Wiki might be another option, too. Could also add docs to Slack channel. But, too many places and it is hard to maintain.

Additional context
More info is better! :-)

Improve the Arduino library import process

Is your feature request related to a problem? Please describe.
There are libraries in a zip file in the repo. This is not a "best practice", but completely understand why/how it is the way it is.

Describe the solution you'd like
Create project files/some way to refer to other projects.

Describe alternatives you've considered
Ensure there are minimal binary/zip files in repo. Document steps to create a simple Mama/Papa from nothing (in regards to the library files)

Additional context
Looks like you can create dependencies (see https://github.com/arduino/Arduino/wiki/Library-Manager-FAQ )

Need to limit amount of chars allowed to be entered in the captive portal

Describe the bug
There is a limit to the allowed message size when sending. If it is exceeded the msg will not be sent and the Duck will PanicDuck. Unfortunately the user will likely not realize their disaster msg did not get sent.
To Reproduce
Steps to reproduce the behavior:

1. Go to the captive portal (I used the MamaDuck example config)
2. Enter a lot of char's in each field, especially the comments section. I just added a bunch of '1's in each field
   Hit send
   9:22:37.776 -> Topic: status
   09:22:37.776 -> uuidCreator 9xba1mpe
   09:22:37.776 -> Warning: message is too large!
   09:22:37.776 -> sendPayloadStandard packetIndex 1423
   09:22:37.776 -> LORA SND millis:27293 size:1423 data:f5025200f60939786261316d706500f7713131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131312a31313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131312a3131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131312a313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131312a31313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131312a312e3131313131313131313131313131313131652b39332a6f6b2a32342a31372a32332a31313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131312a313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131313131312a00f3025200e30773746174757300
   09:22:38.015 -> Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.
   09:22:38.015 -> Core 1 register dump:
   09:22:38.050 -> PC : 0x400d67c7 PS : 0x00060c30 A0 : 0x800d47d6 A1 : 0x3ffd8ca0
   09:22:38.050 -> A2 : 0x00e67ec0 A3 : 0x00e67ec0 A4 : 0x00e67ec0 A5 : 0x00000000
   09:22:38.050 -> A6 : 0x00000000 A7 : 0x00000000 A8 : 0x8008c0ba A9 : 0x3ffd8cc0
   09:22:38.050 -> A10 : 0x3ffc8d78 A11 : 0x3ffdfff9 A12 : 0x3ffdf0f4 A13 : 0x3ffdfc24
   09:22:38.050 -> A14 : 0x3ffdeb55 A15 : 0x3ffdfc25 SAR : 0x00000010 EXCCAUSE: 0x0000001c
   09:22:38.083 -> EXCVADDR: 0x00e67ec4 LBEG : 0x400014fd LEND : 0x4000150d LCOUNT : 0xffffffff
   09:22:38.083 ->
   09:22:38.083 -> Backtrace: 0x400d67c7:0x3ffd8ca0 0x400d47d3:0x3ffd8cd0 0x400d4871:0x3ffd8cf0 0x400d4937:0x3ffd8d20 0x400d57ff:0x3ffd8d40 0x400d615f:0x3ffd8d60 0x400d342a:0x3ffd8da0 0x400d36ec:0x3ffd8dd0 0x400d3aa6:0x3ffd8e50 0x400ddd05:0x3ffd8ef0 0x400dc0d1:0x3ffd8f30 0x400dc161:0x3ffd8f70 0x400deef9:0x3ffd8f90 0x400def79:0x3ffd8fd0 0x400df58a:0x3ffd8ff0 0x40088b7d:0x3ffd9020
   09:22:38.117 ->
   09:22:38.117 -> Rebooting...

Expected behavior
Each field should have a reasonable limit on the number of allowed char's and the sum of the field lengths should be <= total allowed number.

quickstart example

Is your feature request related to a problem? Please describe.
i would like to have an exampleduck that makes it really easy to get started with something.
currently there are waaaaaay too many steps/decisions required for a "first quack".

Describe the solution you'd like
first goal would be to have an exampleduck that can be compiled as-is.

• a mamaduck,
• using last 16 bit of mac-addr as duckid (4 chars),
• with a statuspage on the webserver showing the last N quacks received

perhaps it is even possible to get rid of having to define a board-type by looking at what the arduino framework defines for this.
the esp32 has enough ram that adding this "status page" should be doable, there should be no impact on builds with this feature disabled.

once the basics are in place this can easily be extended to add a basic "lora CB-like chat".

Describe alternatives you've considered
i dont see any alternatives since no amount of documentation can replace a zeroconf experience.

Additional context
the merge-conflict potential seems low since this is mostly an additional example plus an added page in the webserver.
i did not look at the webserver code close enough yet, if it looks problematic i will open another enhancement issue for splitting the captive-portal code from
the protocol code.

Enhance the beginner experience with simple steps on getting started

Overview
As more people join the community, it's good to be able to get them engaged as soon as possible. Whether that's by adding a demo video or overview showing how things work, or by creating short tutorials with simulated pieces (like using Paho to send MQTT messages that are later shown on their own DMS Lite, or by showing how to receive and interpret a message). The goal would be for them to be able to use the technology or contribute to it very quickly.

Additional context
After Call for Code submissions end on July 31, we'd like to provide IBM Developer visitors to go explore and contribute to previous projects that are now open source. They should be able to use the tech, or clearly see how they might open an issue or submit a pull request for basic improvements.

Initial bringup of DuckLinkExample and DuckDetector, Issues noted:

Noted two different things:

1. Was told that the DuckDetector should NOT have the web server portal running nor should the emergency page be available.

2. With the DuckLinkExample code running on a Heltec LoRa 32 and the DuckDetector code running on another Heltec LoRa 32 the DuckLinkExample is never seen. The two units are about 3 feet apart.

When attached to the SerialPort of the DuckDetectorand reset is pressed I see:
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:1216
ho 0 tail 12 room 4
load:0x40078000,len:9720
ho 0 tail 12 room 4
load:0x40080400,len:6352
entry 0x400806b8
Serial start 115200
Starting LoRa......
LoRa online, Quack!
Listening for quacks
Created Wifi Access Point
Created local DNS
Setting up Web Server
Detector Online
Ping!LORA SND millis:4068 size:4 data:f4023000
startTransmit Packet sent
Ping!LORA SND millis:7068 size:4 data:f4023000
startTransmit Packet sent

... endless repeat ...

Ping!LORA SND millis:10068 size:4 data:f4023000
startTransmit Packet sent
Ping!LORA SND millis:13068 size:4 data:f4023000
startTransmit Packet sent
Ping!LORA SND millis:16068 size:4 data:f4023000
startTransmit Packet sent

I would expect to see:

Ping! Pong! from the DuckLinkExample running. It never happens.

Change the “UPRM Emergency Network” value in index.h

Is your feature request related to a problem? Please describe.
This project was developed with Puerto Rico in mind, but I don't think you want new Duck networks sprouting up over the world with that name. Perhaps use generic text?

Describe the solution you'd like
Change the "“UPRM Emergency Network” value to something generic or specific to the user needs.

Describe alternatives you've considered
Not really sure what to use, perhaps something like "OWL Emergency Experimental Network" or something like that.

Additional context
None

Payload as JSON

To make data analysis easier, serializing data as JSON would make it easier to transform the data on the cloud side. However, for the captive portal that has many fields, this can use up a lot of the limited number of bytes that can be sent over LoRa. Therefore, the solution needs to simplify the JSON document as much as possible so that we have the most available bytes for emergency messages/data.

One idea was to have a code in the root that would correspond to a template stored on the cloud. While this could work well in simplifying the JSON doc, it increases the complexity needed on the cloud side.

MaMaDuck example captive portal form not publishing when fully filled out

Describe the bug
while testing, I noticed the captive portal would fail to publish (PaPaDuck => MQTT) when it was fully or mostly filled out. It will fail when I just add my full name and address but succeed if I just have a few char's in multiple fields.

This smells like a "size" or number of char issues - I'll try to narrow it down

Working publish with just a few random chars entered in the form

16:19:32.879 -> handlePacket pSize 47
16:19:32.915 -> LORA RCV millis:94378 rssi:-47.00 snr:6.25 fe:11618.22 size:47 data:f5025200f609793933787572396c00f711742a2a732a7364662a2a2a2a2a2a2a2a00f3025200e30773746174757300
16:19:32.915 -> getPacketData senderId_B Len = 2
16:19:32.915 -> getPacketData User ID: R
16:19:32.915 -> getPacketData messageId_B Len = 9
16:19:32.915 -> getPacketData Message ID: y93xur9l
16:19:32.915 -> getPacketData payload_B Len = 17
16:19:32.915 -> getPacketData Message: ts*sdf******
16:19:32.915 -> getPacketData path_B Len = 2
16:19:32.915 -> getPacketData Path: R
16:19:32.949 -> getPacketData topic_B Len = 7
16:19:32.949 -> hit2
16:19:32.949 -> getPacketData len0 Topic: status
16:19:32.949 -> hit3
16:19:32.949 -> iot-2/evt/status/fmt/json{
16:19:32.949 -> "DeviceID": "R",
16:19:32.949 -> "MessageID": "y93xur9l",
16:19:32.949 -> "Payload": "ts*sdf******",
16:19:32.949 -> "path": "R,Papa"
16:19:32.949 -> }
16:19:32.949 -> Publish to Cloud ok
16:19:32.949 -> {
16:19:32.949 -> "DeviceID": "R",
16:19:32.949 -> "MessageID": "y93xur9l",
16:19:32.949 -> "Payload": "ts*sdf******",
16:19:32.949 -> "path": "R,Papa"
16:19:32.949 -> }
16:19:32.949 -> Publish To Local MQTT ok

Example of a more fully filled out form failing publish
16:52:04.810 -> handlePacket pSize 95
16:52:04.810 -> LORA RCV millis:2046266 rssi:-50.00 snr:6.25 fe:11785.99 size:95 data:f5025200f6096c3236627764306900f7415472656e742a42726f776e2a37333120526f626572746f204176652a53616e746120426172626172612a39333130392a383035373239313831312a2a2a2a2a2a00f3025200e30773746174757300
16:52:04.842 -> getPacketData senderId_B Len = 2
16:52:04.842 -> getPacketData User ID: R
16:52:04.842 -> getPacketData messageId_B Len = 9
16:52:04.842 -> getPacketData Message ID: l26bwd0i
16:52:04.842 -> getPacketData payload_B Len = 65
16:52:04.842 -> getPacketData Message: TrentBrown731 Roberto AveSanta Barbara931098057291811*****
16:52:04.887 -> getPacketData path_B Len = 2
16:52:04.887 -> getPacketData Path: R
16:52:04.887 -> getPacketData topic_B Len = 7
16:52:04.887 -> hit2
16:52:04.887 -> getPacketData len0 Topic: status
16:52:04.887 -> hit3
16:52:04.887 -> iot-2/evt/status/fmt/jsonPublish to 'client' failed
16:52:04.887 -> Publish to 'LocalMqttServer' failed

To Reproduce
Steps to reproduce the behavior:
Fill out the captive portal form on the MaMaDuck example and hit send. Monitor the serial output on the PaPaDuck.

Expected behavior
the Form information should be published regardless of the amount of information entered on the form

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Add static code analysis tooling

Is your feature request related to a problem? Please describe.
Add some sort of lint/static code analysis

Describe the solution you'd like
Create a way to run some static code analysis on the code via command line.

Describe alternatives you've considered
Just a quick google shows these projects:
https://clang.llvm.org/extra/clang-tidy/
http://cppcheck.sourceforge.net/

Additional context
Would be nice to then connect this to either https://travis-ci.org/ or use a github action.

Sensor Integrations

The ClusterDuck Protocol provides a foundation that can enable a variety of sensor applications for different purposes. An example of this are the SolarDucks in Puerto Rico - many of these have BMP180 (temperature, pressure, altimeter) sensors or an Air Quality PM2.5 sensor. Most sensors that can be integrated into Arduino can be integrated into a DuckLink.

This issue focuses on two main goals

1. Constructing the process for future integrations of sensors
2. Building the integration of numerous common sensors already used. There is already an example for the BMP180 sensor. Others we are looking to build out

• MQXX Gas sensors (MQ-1, MQ-2, MQ-3, etc).
• Accelerometers
• Air Quality PM2.5 sensors
• Infrared heath thermometers
• Motion/visual sensors

Update Captive Portal

The captive portal is one of the many ways to interface with the CDP network. The default build of the CDP comes with the emergency captive portal template. While this is useful for the original mission, there is an opportunity to improve the capabilities and make it easier to customize.

Next steps:

• Add developer testing console

• Add data output template

• Create a customizable template

• Add more documentation on how it works

Make the Build-a-Duck documentation platform independent

Per the conversation on #47

FYI @gozu42 @Salzi

document the IBM Watson cloud pieces

Is your feature request related to a problem? Please describe.
Add info on how to get started using the cloud.ibm.com stuff

Describe the solution you'd like
Create a step-by-step doc for going from nothing to seeing stuff happen in

Describe alternatives you've considered
May consider using other cloud-stuffs. But, since this project was sponsored and uses IBM and IBM's cloud is free for lowest tier, I think it would be best to document how to get started with it.

Additional context
create new account, add device in ibm cloud, show how to get org/token, how to get messages flowing, etc.

improved duck telemetry

Is your feature request related to a problem? Please describe.
currently the alive-quacks do not have any real information content.
there also is no real "monitoring of duck health" beyond it.

Describe the solution you'd like
i would like to add some telemetry info to the alive quacks.

• basic - device independent infos (only depends on arduino framework)

• • uptime (millis())
• • loopcount (have a counter at the beginning of loop(), this is a simple way to measure how "busy" an arduino device is)
• • memory stats (free mem, max block, low water mark,...)

• advanced - device specific infos (needs to be part of a board definition)

• • battery level (adc pin)
• • power stats (axp192 on the newer t-beams)

these could all be added in simple string format as space-separated tag=value pairs to the alive message.

Describe alternatives you've considered
this could be done in a separate "telemetry quack" message, but i do not see the advantage over just sending it as part of / replacement for the alive-quack.
the text-format is not important to me, if there is an advantage for dms/backend in making the telemtry json right away, just holler.
something like the loopcount is "more natural" to do in the duck-app-sketch, but can also be contained inside the duck-class.

Additional context
the basic infos have very low merge-conflict potential since it mainly just changes the alive function.
i would like some basic feedback on format and suggestions for other infos to include.
advanced infos can be added later once it is more obivous how/which boards are supported/used.

improved logging

Is your feature request related to a problem? Please describe.
currently the logging is just Serial.print all over the place which makes it hard to focus on individual areas when needed.

Describe the solution you'd like
i would like to introduce a log function that is similar to the classic unix syslog facility+severity concept.

so each of these serial.print gets classified by "what is this about" (facility, like "lora" or "parser" or "web" or "uplink"...) and "how important is this" (severity, like "debug", "info", "warning", "error", "kaboom"...).

and the developer/user can decide what they want to see, like "web at full debug, everything else warning and above".

the simple way to do this is a lot of defines.

Describe alternatives you've considered
i looked at a few arduino logging frameworks, but they were all either lacking in relevant functionality or were really heavy-handed.
and adding yet-another-external-dependency for a developer-only feature seems bad.

the define approach has the advantage that for the simple case it is compile-time evaluated (meaning disabled messages dont are not even in the compiled binary).
and at the same time it allows to retrofit anything under the hood without touching the main code again, so if someone wants their papa to log to mqtt, no need to change the lib.

Additional context
this has maximum merge-conflict potential since it needs to touch every single Serial.print currently in the code.
the concept/idea can be demonstrated as part of a out-of-tree special-purpose-duck, then do the main change as a compact 1-3 hour session.
i suggest i do the demonstrator, get feedback/approval from people, then coordinate a change-window via slack.

App Development

When communication systems go down it is impossible to call for help let alone download an app. That is why each Duck is equipped with the Captive Portal. While the Captive Portal can work as a pseudo app experience, it is very limited. Things such as getting the phone’s GPS and communicating live data are either not possible or easy to implement.

Having a native app that can communicate with the Ducks over WiFi or Bluetooth opens up many ways to interact with the CDP network. The functionality we are currently most interested in:

• OTA updating

• Debugging

• Data/telemetry collection

• Send data over network (similar to captive portal)

We currently have an app example that is able to send and receive data over WiFi. We also have a Frankenstein app example for the same over Bluetooth (DiscoDuck).

add info about distances/speed (actual vs. theoretical)

Is your feature request related to a problem? Please describe.
Can info be shared about what was learned during some deployments? (ex: 400m with clear line of sight between MomaDucks got this much bandwidth/error rate, etc.)

Describe the solution you'd like
Some info on what was found during deployments. Perhaps what is "optimal/maximal" vs. actual.

Describe alternatives you've considered
WiFi should be fast (somewhat normal 2.4 wifi network speeds) between DuckLinks, but from Mama to Papa, since going over 900MHz is limited. Document/share these speeds/info.

Additional context
Might be cool to show info on how using pringle cans and/or low-tech antennas might improve. (like lofting some Mama's via balloons in the video)

Send data over RockBlock Instead of WiFi - Iridium Satellite Network

Support RockBlok
Iridium Satellite Communication

The goal is to connect the Papa Duck to the Iridium Satellite Network, to send data from places where there is no LTE or WiFi Available.

We want to integrate the Rockblock module to the different ESP32 boards and add the IRIDIUMSBD library for the RockBlock Module.

Currently you can only send data from the papa straight to an MQTT server by WiFi, so adding the Rockblock we would be able to send data an alternative route.

• Setup RockBlock module

• When a message comes in we need to convert to a const char so we can send with RockBlock

• When we converted message we can send the message to the network

• Write documentation for future wiki, about tokens/use of iridium

Update template text for captive portal to be generic

Describe the bug
These currently say UPRM on them. Should the portal be more generic?

• https://github.com/Code-and-Response/ClusterDuck-Protocol/blob/master/index.h#L8
• https://github.com/Code-and-Response/ClusterDuck-Protocol/blob/master/index.h#L143

add comments to all functions

Is your feature request related to a problem? Please describe.
Add comments to all functions. Bonus points if they are doxygen-compatible.

Describe the solution you'd like
Add/format comments on each function in ClusterDuck code.

Describe alternatives you've considered
Doxygen-based formatting is probably the best option, but might consider using something like https://www.sphinx-doc.org

Additional context
Start small. Add comments and some auto-doc gen stuff.

PaPi - PapaDuck + Raspberry Pi - an Offline Solution

While sending data to the cloud is very powerful, it is also valuable to be able to view data locally. After a natural disaster internet or satellite connectivity might not be available. That's why we felt it was important to have a terminal that can store incoming data and be viewable locally.

The approach we took was to use a Raspberry Pi for the terminal. The Rasberry Pi is great as it can be used as a normal computer and is very configurable. To connect the Pi to the network, we opted to connect a PapaDuck to the Pi's WiFi access point and push the data over MQTT. The Pi runs DMS-Lite locally to make viewing and tracking data easy. While this has worked pretty well so far, there are other options that might work better. Those are Bluetooth, Serial connection, and LoRa hat.

Project Owl has been working on the PaPi project with a team of students from the University of Mayaguez in Puerto Rico.

We see this expanding further than just a data terminal. Open to ideas and other applications that can run on the PaPi.

DishDuck Hybrid Inconsistency Issues

Describe the bug
I have been doing testing on the DishDuck Hybrid this past week. I have found multiple issues as a result of this testing that needs to be fixed. First is the "Hybrid Portion" when you are out of range of WiFi Rockblock is suppose to send the messages, Well I tested this and while it does work it is really inconsistent. Meaning that it does not always switch from one to the other. The second issue is that if the DishDuck is flashed without an SSID and Password, You should be able to goto PAPA Duck WiFi setup and set one. But that does not work at all as of right now get a watchdog error and TTgo T-Beam just reboots. Also (SSID Available Scan and Connect) hangs pretty often not allowing the T-Beam to carry on with operations.

This is the WatchDog error when trying to use PAPA WiFi Setup Page

Update the README and PDF to specify exact ZIP file to import

Describe the solution you'd like
Update the README and Make a Duck Manual to mention the release ZIP file as the thing that can be imported into the Arduino environment (in addition to the source copying option). Right now it just says there's a ZIP file (which can be confused for Libraries.zip inside the repo).

This can point to either the stable release or latest from the master branch, depending on which the user wants to run:

• https://github.com/Code-and-Response/ClusterDuck-Protocol/releases
• https://github.com/Code-and-Response/ClusterDuck-Protocol/archive/master.zip

DuckLink Positioning System (DPS)

We are developing the DPS which locates lost Ducks in unknown locations. It calculates the location of a lost Duck using RSSI of Ducks stationed in known locations. (If a lost Duck receives a high RSSI, then that means it is close to that stationed Duck. If the RSSI is low then it is far away from the stationed Duck.)

Currently we have a vanilla firmware to deploy the DPS. Now we have to integrate the firmware into the CDP. For more details on DPS, check the DPS wiki.

Make Configurables more configurable

Is your feature request related to a problem? Please describe.

currently everything is hardcoded and in just two files which can be very annoying when for example european ducks should use different RF parameters.

Describe the solution you'd like

i would like to move the currently hardcoded configurables out of the cdp library code.

as step one, just replace them all by CDP_something defines that go into a separate cdp-config.h in the same dir.
while this is not a longterm-best solution, it is one that is really low effort and unlikely to break anything and already solves the "i commited the wrong frequency/pinout by accident" issue.

this also bundles all configurables in one place which is a good step towards making them actualy configurable.

next step could be to move some of them into even more separate .h files to keep pinout specs for different boards or sensor loadouts or deployment-specific wifi setups in one place each.

this also makes it more obvious which parts could be useful to have runtime-configurable.

Describe alternatives you've considered

there is certainly potential to use an "if xml is not solving your problem, you are not using enough of it" approach.
but i dont see any way to make it as low-effort, gradual and "as needed" scalable.
the define-and-refine way also leaves all options open for runtime-config-storage later, so we can see what works best when we get there.

Additional context

candidates, grouped by "dynamic range":

• unlikely to change in a dynamic way

• • hardware details (pinout)
• • hardware presence (do we need to init display, powerchip, vext,...)

• unlikely to ever change within a deployment

• • RF parameters (radio type, frequency, bandwidth, spreadfactor, coding,...)
• • timings (sensor frequency, alivequacks,...)

• more likely to change

• • wifi details (both papaduck-uplink and allducks-AP)
• • password(s) (config page, wifi,...)

this change will be all over the cdp codebase for the first step, so will have maximal merge-conflict potential there. otoh, that first step is 1-3 hours max.
i would like to get an OK for this from @nfeuer and suggest we coordinate a suitable timeslot (where no one else has major changes open) on slack.

after that first step, the potential for merge conflicts is greatly reduced, so any additional steps (like adding the definition for another board) should require almost no coordination.

Fix Network Pathing/Hopping

Each time a Duck sends a message, it adds its own digital signature to the “path” portion of the message. Since each message is broadcasted to all nearby devices, a Duck will receive a message it has already sent. To prevent the same message from being sent infinitely in a loop, the Duck will check the “path” section for its digital signature using idInPath(). If the Duck id is in the "path", it will ignore the message. This has worked well in linear and smaller-scale networks. However, this becomes an issue in large scale networks with many devices and long device ids. Each LoRa message cannot exceed 256 bytes per packet. While splitting packets could work to extend the message size, it increases device busy time and network traffic.

Proposed solution
One idea to fix this is by assigning each Duck with a value dependant on how far away it is from the PapaDuck. If receiving a message from a Duck with a higher value than its own assigned or self-assigned value it won't send the message.

Here is a paper that touches on the idea:
ProximAid: Proximal adhoc networking to aid emergency response
Message me on Slack if you are interested in learning more about this paper.

Utilize Packet Object More

Make the Packet object the main mode of moving Packet data beyond just received packets. This will make buffering messages easier as well as other integrations.

Configure SSL certificate for http://clusterduckprotocol.org/

Describe the solution you'd like
The http://clusterduckprotocol.org/ site needs a valid SSL certificate to support HTTPS. We can add one through Let's Encrypt or another service and configure the web host.

BlueBird | Duck to PWA + Native Mobile App over BLE

Is your feature request related to a problem? Please describe.
The feature proposes an offline first solution to view and interface with outgoing Duck data without the need of additional hardware client dependencies.

Describe the solution you'd like
View and interface Duck data directly on a progressive web app, via Web BLE or a native mobile app.

Using TOA for DuckLink Positioning System instead of RSSI

The DuckLink Positioning System (DPS) is used to locate other Ducks in GPS-denied environments. For more details read the wiki on DPS.

Currently the DPS uses RSSI values to locate where the lost Duck is. Another possible method is to use Time of Arrival (ToA). ToA calculates the distance by using the time it takes a signal sent by a Duck to reach another Duck.

We would like to explore ToA to see if it would be a better process than using RSSI.

Build out contribution model

Create a system that leverages GitHub projects, wiki, milestones, and issues in order to clearly show open bugs and future directions. This will support the community by showing cutting edge work, help needed, and release schedule.

• Start with laying out known issues here, along with thoughts around cause and solutions
• Next capture future directions in the roadmap to track towards in issues. This could be features or refactoring goals
• Lay out milestones to target those issues at, whether a specific date or event

The key first dependency is to ensure the very latest code is here so that everyone knows they are working on the latest.

Add info on how to troubleshoot

Is your feature request related to a problem? Please describe.
After you create a MamaDuck, what should you see in Arduino Serial Monitor? (using 115200 baud)
After you create a PapaDuck, what should you see in Arduino Serial Monitor?

Describe the solution you'd like
A clear and concise description of what you want to happen.
Document some "errors"/problems.
Be sure to change the "Z" in the MamaDuck to a realistic value. (I used mac address without colons and it seemed to work ok.) If not, then you may see Authentication errors on PapaDuck serial output.
If only see "i", check four values in PapaDuck.

Describe alternatives you've considered
Add some troubleshooting/helpful info.

Additional context
Show info on what you should see on IBM side.

Build a buffer for incoming mesages for PAPA

Is your feature request related to a problem? Please describe.
When a message comes into the papaDuck but the papaDuck is trying to connect to the internet the incoming messages get lost.

Describe the solution you'd like
We need to build some sort of storage/queue where the incoming messages can sit temporarily

Additional context
When I was developing for the DishDuck I came across a problem where it takes a little while before the RockBlock is booted so the incoming messages get lost during that time.

Add support for WS2812 LED

I would like to give feedback from the Ducks Health by changing the color of a LED.

For example, When the battery is almost dead (get info from AXP202x Chip on the Te beam) we turn the LED red and when its fully charged Green.

• Make example code for simple LED

• add support for FastLED Library.
  To run different WS2812 LED chips we need to add FastLED library support.

• Intergrate with AXP202X Lbrary for changing colors

Get more feedback from AXP192 when using T-Beam or other AXP enabled devices

Is your feature request related to a problem? Please describe.
With the build in AXP192 we can get data like Voltages, Watts, Is charging, battery percentage, etc. Which can be very useful to see when the ducks are deployed in remote areas.

Describe the solution you'd like
Integrate AXP Library to start integrating the Pre-build methods for retrieving power data

Additional context
V1 is developed and I will make a PR ASAP.

add source for generating the pdf

Is your feature request related to a problem? Please describe.
Would like to help update the info in the .pdf, but do not know where the source for it is. Can that be shared?

Describe the solution you'd like
The "source" for the pdf to be shared so it can be updated. (even if it is a word doc or something like that)

Describe alternatives you've considered
The pdf may be able to be generated from doxygen/sphinx or other toolings.

Additional context
Just want to help improve this cool project. :-)