sparkfun/sparkfun_datalogger

Documentation and firmware for the SparkFun DataLogger IoT line of products.

SparkFun DataLogger IoT

Documentation and Firmware for the SparkFun DataLogger IoT line of products

This repository contains the latest firmware for the SparkFun DataLogger IoT products. The DataLogger IoT product line are pre-programmed to automatically log data from 50+ SparkFun Qwiic sensors, all without requiring any hardware setup or code development. Just plug in a qwiic board and log data. The DataLogger IoT automatically detects the connected sensor, configures the device and enables logging information locally, or to a variety of IoT data services.

The current list of SparkFun boards supported by the SparkFun DataLogger IoT are listed in the DataLogger Supported Device List.

SparkFun DataLogger IoT products include Internet connectivity built in, and includes connections for a variety of IoT service services and end-point types. Connection types include MQTT, HTTP, AWS IoT, Azure IoT, ThingSpeak, Arduino Cloud. To enable connectivity, connect the DataLogger IoT to a WiFi network, enter the required credentials for the specific IoT service being used, and the DataLogger IoT will do the rest - no programming required.

The DataLogger IoT products are highly configurable via an easily to use serial interface. Simply plug in a USB C cable and open a terminal at 115200 Kbps. By default the logging output is automatically streamed to the serial terminal - pressing any key will bring up the menu system. Device settings are also configurable via a JSON file contained on an inserted SD card - enabling rapid configuration for multiple devices.

Configuration details are outlined in DataLogger IoT Documentation.

The functionality of the DataLogger IoT firmware will continue to grow as SparkFun develops new Qwiic sensor boards, adds additional IoT services and product features. The board is easily updated via the on-board settings system. See below for future information on updates and the latest Firmware release.

Latest Firmware

• Version 01.1.01 - Overview of the Release

Documentation

• Hookup Guide - Basic hookup guide for the Datalogger IoT 9DoF and DataLogger IoT
• GitHub Hardware Repo
  • SparkFun DataLogger IoT - 9DoF
  • SparkFun DataLogger IoT

Supported Products

• DEV-22462 - SparkFun DataLogger IoT, v1.1
• DEV-20594 - SparkFun DataLogger IoT - 9DOF, v1.1

Firmware Updates

This repository contains the latest firmware for the DataLogger IoT products. The firmware is installed on a DataLogger IoT board using the built in over-the-air update functionality if the board is connected to the internet, or by loading the firmware off an SD card installed on the DataLogger IoT device.

Instructions on updating the DataLogger firmware are detailed here.

Latest Release

Details on the latest Firmware release are listed on the Release Page

sparkfun_datalogger's Issues

Duplicate configuration values?

Greetings,

In the Serial configuration interface, what is the difference between "/Settings/Application Settings/Output" and "/Settings/Logger/Output"?

Do they serve a separate function or is it simply a duplicate?

Best regards,

Marek J.

Feature Request - output logs over serial/USB

Hi,
Would it be possible to add support for accessing saved logs from the SD card over the serial connection (so the device doesn't need to be turned off and the SD card manually removed)? It would be nice for cases where the logger loses access to a serial connection or wifi.

Essentially, it could be the same thing as what was done for the Artemis logger here: sparkfun/OpenLog_Artemis#33

FEATURE REQUEST - Operational metadata

I'm wondering if it would be possible to send metadata about the logger with every data request

something like this

"General"  : {
   [...]
   "uptime": 123456,
   "wifi_wrssi" : -56,
   "wifi_ssid" : "homeWifi"
}

this would enable someone to build some operational excellence and monitor the performance and stability of the device.

[update] updated to use the General key in the JSON [/update]

DataLogger IoT upload format

I bought this board https://www.sparkfun.com/products/20594 SparkFun DataLogger IoT with a SF TMP117 temp sensor plugged in the board. I am using the adafruit IO mqtt service. After configuring the parms for the mqtt service, wifi and the logging interval everything seems to start and connect ok:

SparkFun DataLogger IoT (c) 2023 SparkFun Electronics
Version: 01.00.04 - build 0000bb

Restoring System Settings ...restored from ESP32 Preferences
Connecting to WiFi...Connected to NETGEARMR1100
NTP startup [5 secs]...enabled
MQTT Client: connecting to MQTT endpoint io.adafruit.com:1883 ...connected
Loading devices ... 2 devices detected
TMP117 - TMP117 Precision Temperature Sensor {qwiic}
MAX17048 - MAX17048 LiPo Battery Fuel Gauge {qwiic}

Time: 2024-01-01T10:59:47
Uptime: 0 days, 00:00:09.756
External Time Source: NTP Client
Board Name: SparkFun DataLogger IoT
Board ID: SFD248966B65B7A0
[I] SD card not available
[I] WiFi - Connected SSID: NETGEARMR1100 IP Address: 192.168.1.83 Signal: Fair
[I] Battery - Not Connected
[I] System Deep Sleep: disabled
Sleep Interval: 60 seconds
Wake Interval: 120 seconds
[I] Logging Interval (ms): 60000
[I] Serial Output: Disabled
Baud Rate: 115200
[I] SD Card Output: Disabled
Rotate Period: 24 Hours

[I] IoT Services:
MQTT Client : enabled
MQTT Secure Client : disabled
HTTP IoT : disabled
AWS IoT : disabled
Azure IoT : disabled
ThingSpeak MQTT : disabled
MachineChat : disabled

[I] Connected Devices [2]:
TMP117 - TMP117 Precision Temperature Sensor {qwiic}
MAX17048 - MAX17048 LiPo Battery Fuel Gauge {qwiic}

I called the mqtt topic:
Settings for: /SparkFun DataLogger IoT/Settings/MQTT Client/MQTT Topic
Edit the value of MQTT Topic - data type
When complete, press to accept, to discard
MQTT Topic = zzzzzzzz/feeds/datalogger.tmp117

I disabled the outputs for the 3 fields of the max17048 device in /SparkFun DataLogger IoT/Devices Settings/MAX17048

Settings for: /SparkFun DataLogger IoT/Devices Settings/MAX17048
Outputs:

1. Voltage (V) - Battery voltage (Volts)
2. State Of Charge (%) - Battery state of charge (%)
3. Change Rate (%/hr) - Battery charge change rate (%/hr)

When I open the web console of adafruit IO I am seeing updates once a minute but in the 'value' of the tmp117 feed I see this as the value:

{"TMP117":{"Temperature (C)":27.96875},"MAX17048":{}}

I just want to see the data value for "Temperature (C)" of 27.96875 not the entire string above

Is there a config setting for that somewhere ?
thanks

Same module, different addresses

Greetings,

We have found out that in case we use two of the same modules on different addresses (by soldering the address bridge), for example, two buttons. The device lists two buttons connected, but outputs only data for single button.

It would be convenient to include all devices of the same type in the output and also add the possibility of outputting the I2C address of the device (at least in JSON format).

Best regards,

Marek J.

BME68x VOC calculations on the iot logger

I wanted to use my BME68x to calculate a air quality index. After some research on the breakout board, I discovered there is an embedded library by Bosch that is commonly used which uses the different sensor readings.

That method of getting an air quality value cannot be done based on the raw data which is logged.

Would it be possible to include the Bosch library ( https://www.bosch-sensortec.com/software-tools/software/bme680-software-bsec/) and calculate the air quality on the iot logger? Otherwise the "gas resistance" value provided by the sensor is not very useful.

How to do NTP sync ?

I'm using version 01.01.01 - build 0000f4.
I've enabled NTP with servers: pool.ntp.org and time.nist.gov, and I'm connected to wifi, but the main clock quickly falls behind.
To resync the datetime, I've tried to restart device, disconnect/reconnect wifi, disable/re-enable NTP...
Nothing works.

I'm guessing the NTP won't resync unless the timedelta is above a certain threshold ?
How can I resync the clock ?

Crashes faster than 100ms Logging Interval

The firmware 01.01.01 crashes when the logging rate is set faster than 100ms logging interval. The error is:

Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandle
d.

Core 1 register dump:
PC : 0x40223bc6 PS : 0x00060b30 A0 : 0x800ec90f A1 : 0x3f
fb2250
A2 : 0x00000000 A3 : 0x3ffc9730 A4 : 0x3ffb941c A5 : 0x00
0000ff
A6 : 0x3ffb9454 A7 : 0x3ffc8c04 A8 : 0x80223bcc A9 : 0x3f
fb2230
A10 : 0x3ffc54a8 A11 : 0x3ffc53e8 A12 : 0x3ffc51e8 A13 : 0x00
000000
A14 : 0x3ffb8a6c A15 : 0x3ffd58b4 SAR : 0x0000000c EXCCAUSE: 0x00
00001c
EXCVADDR: 0x0000011b LBEG : 0x4008563d LEND : 0x40085645 LCOUNT : 0x00
000027

Backtrace: 0x40223bc3:0x3ffb2250 0x400ec90c:0x3ffb2270 0x401602bc:0x3ffb2290

FEATURE REQUEST - ADS1015 Unsigned Mode

The "DataLogger IoT" currently faces an issue related to the ADS1015 analog sensor, impacting the accuracy and usability of analog data acquisition. This feature request aims to resolve the problem by introducing an "unsigned mode" for the ADS1015 sensor.

1. Current Behavior: The ADS1015 sensor, as integrated into the "DataLogger IoT" project, exhibits a behavior where it outputs signed values ranging from -2047 to 2047. This signed representation poses challenges when collecting analog data, as it limits the effective utilization of the sensor's full 12-bit range.

2. Impact on DataLogger IoT: The existing behavior of the ADS1015 sensor negatively impacts the accuracy of analog data collected by the "DataLogger IoT" project. This deviation from expected behavior leads to data inaccuracies, which can undermine the reliability of the data logging system.

3. Request: We propose the implementation of an "unsigned mode" specifically within the "DataLogger IoT" project for the ADS1015 sensor. In this mode, the sensor would output values in an unsigned format, utilizing the entire 12-bit range from 0 to 4095. This change will enhance the compatibility and accuracy of the ADS1015 sensor within the context of the "DataLogger IoT."

4. Comparison to OpenLog Artemis: It is worth noting that the OpenLog Artemis, a related device, operates with an unsigned output format, which simplifies data acquisition and ensures consistency. Aligning the ADS1015 behavior within the "DataLogger IoT" project with the OpenLog Artemis would improve the overall user experience.

Investigate providing alternate I2C addresses for the ACS37800 Power Meter

Based on this Forum post: https://forum.sparkfun.com/viewtopic.php?p=245235#p245235

Investigate if it is possible to add alternate addresses for the ACS37800. Remembering that it will likely clash with the SCD30 and SCD40.

Support for presence sensor AK9753 https://www.sparkfun.com/products/14349

I have an AK9753 presence sensor breakout board from sparkfun (https://www.sparkfun.com/products/14349) but there is no support for that device.

Is there a way I can write a "driver" for that chip myself? I was hoping the datalogger firmware was completely open source but I can only find firmware dumps :(

Add list of POSIX time zones to the docs?

Maybe I missed it, but I couldn't find a link to a list of POSIX time zones in the docs.

There is a list here: https://github.com/nayarsystems/posix_tz_db/blob/master/zones.csv

I believe that list is copied and pasted from Linux /usr/share/zoneinfo/ , so maybe we could use a workflow action to auto-generate the list periodically?! ;-)

Issue with TIME - jumping by four hours

Based on this customer comment:

https://www.sparkfun.com/products/20594#comment-65305d4e8e19f166d1429ed5

"
I'm having the worst time with TIME. I'm logging the epoch time on the data logger and the time from the RTC (8803) at 1Hz. It seems every hour (3600 samples) the time jumps by 4 hours. It starts at the correct time. Both the RTC time and the epoch time do the same jump at the same sample. Here is a plot:

"

Firmware release v1.1.0 causing preferences not to restore from on-board storage on some/older DataLogger IOT 9DOF boards

An issue occurs with some 9DOF boards that prevent saved preferences from being restored from on-board (on device flash storage).

The issue has been identified and a hot-fix is being verified/tested.

Do you plan to upload source for system?

We are exploring using the Datalogger_IoT_9DoF. We deploy our systems in secured locations and need to be able to present our source code as part of the project to get clearance. Do you plan to release the code for this system, or do we need to create our software solution?

Using Flash Download Tool of Espressif to update the firmware.

Hello.
I accidentally download my own firmware to the board. Therefore, the firmware update instruction provided by SparkFun doesn't work. Could you please support me to recover my board? I need the compiled binary files that can be used with the Espressif Flash Download Tool .

Thank you very much!

Are the firmware sources available?

Hello,

I just discovered this interesting product and I would like to understand if

1. The source code is available.
2. If so, under which licenses are the sources released

So far I could only find the binaries (in this repository under /firmware/bin) .

Thanks in advance!

Delay before probing Qwiic bus

Greetings

Would it be possible to add a customizable delay on boot/startup before probing the Qwiic/I2C bus for external modules?

We would like to use the DataLogger in an application where the external modules are not powered exactly at the same time as the DataLogger, but with a slight delay (several seconds).
In our application, since the modules are powered up a little after the DataLogger, it does not register them and thus does not read the data from the modules afterwards.

Best Regards,

Marek J.

NAU7802 Inconsistent Readings on Device Reset/Reboot

Description

Issue Summary:
Our IoT data logger setup, integrating the SparkFun Qwiic Scale - NAU7802, is experiencing significant variations in sensor readings immediately after the device undergoes a reset or reboot. This challenge persists regardless of the power supply or load cell used and is not mitigated by the fact that calibration remains intact. Notably, the required taring process post-reboot is not feasible for our headless, production-environment devices, leading to unreliable data collection and compromising data integrity and analysis.

Expected Behavior:
Sensor readings should be consistent and accurate before and after device resets or reboots, with minimal variation that adheres to the expected tolerance levels specified in the sensor's datasheet, without the need for manual intervention such as taring.

Actual Behavior:
Post reset or reboot, readings from the NAU7802 vary significantly, despite the calibration being intact. The issue occurs even with no load cell connected and without any change in the measured weight. For example, readings can exhibit a substantial margin of variation (e.g., more than 10% variation) immediately following a device reset, indicating a need for taring that cannot be addressed in our headless device configuration.

Steps to Reproduce

1. Power on the device and allow it to stabilize for a few minutes.
2. Record baseline readings from the NAU7802 for a known weight (or with no load to observe variance).
3. Reset or reboot the device without altering the setup.
4. Once the device is back online, immediately take new readings from the NAU7802.
5. Note the significant variations in readings, indicating a need for taring which is not feasible.

Environment

• Hardware: SparkFun DataLogger IoT
• Firmware Version: Latest (v1.1.01)
• Power Supply: Issue persists across various power supplies (battery, usb)
• Load Cells: Observed with multiple load cells and with no load cell connected

Additional Information

• Calibration Status: Calibration checks confirm that the calibration remains intact post-reboot/reset.
• Comparison with SparkFun OpenScale (HX711): This issue does not occur with the SparkFun OpenScale board (HX711) under similar conditions.
• Frequency of Issue: Consistently occurs with every reboot/reset, especially noticeable when "System Sleep" is enabled.
• Troubleshooting Attempts: Troubleshooting across different power supplies, load cells, and with no load cell connected confirms the issue is not related to calibration drift but to the inability to perform taring in a headless production environment.

Possibility of Releasing the Firmware as a Dynamic Library

Hello,

I just found out that the firmware is closed-source.

Would it be possible to provide it as a dynamic library with a corresponding interface contained in a header file?

That way we would be able to interact with the firmware and develop on top of it. For example, you could provide us with a function to add a function pointer to the list of procedures that need to run when logging.

An example code could be:

string readVoltage() {...}

firmware.log(&readVoltage);
firmware.log(&addStuff);
firmware.start();

I really think this would improve the usability and extensability of the product while not divulging the information you want to keep closed-source.

Thank you!

Parameter/Character limit

Hello

There seems to be some kind of parameter or character limit for the output.

We were experimenting with 6 modules on the I2C bus, but for some reason when adding modules, the last parameters are left out of the message, at least in the JSON format.

Some modules have many parameters, which causes some parameters to be left out from the output, even from the on-board sensors.
Even adding timestamp to the message causes loss of at least 1 parameter output.

Would it be possible to remove this limit or increase the number of parameters in the output to some larger number?

Best regards,

Marek J.