This project contains tools for sending IMU data over a WiFi or Bluetooth connection from an ESP, and consuming it in a web page.
it contains these components:
-
A MicroPython program (in
./pyboard) that runs on an ESP8266 or ESP32. It publishes sensor data from a connected to BNO055 IMU, a simulated IMU, to an MQTT connection, or the serial port. -
Web pages (in the
webdirectory) that display IMU data as graphs, charts, and 3D models using React, p5.js, Highcharts, and D3. These serve as examples of how to consume the IMU data. -
A fleet management dashboard (also in the
webdirectory) that displays a list of connected device, with frame rates. -
Command-line tools (in the
scriptsdirectory, and invoked viapoetry) to report, simulate, and measure the sample rate of IMU data, and relay it to a named port. -
Additional command-line scripts for managing MicroPython/ESP development.
-
An npm package imu-tools npm package that that can be included in a web page in order to subscribe to an MQTT broker that is relaying data in the format provided by the tools in this directory.
-
An experimental demonstration of rigging a Blender model to IMU output.
The ESP program in this repository sends data over MQTT/WiFi. The rest of the code in this repository is also designed to work\ with https://github.com/osteele/Arduino-BLE-IMU, which uses Bluetooth Low Energy (BLE) connection instead.
Local MQTT
Gateway. An ESP32 connects via WiFi to an MQTT broker running on a developer
laptop. A web app, such as a React or p5.js application, can connect to this
broker to read the sensor data from all the IMUs on the network.
. This architecture MQTT Gateway allows a web
page to draw information from many MCUs.
. It also allows several computers to subscribe
to data from the same sensors. This is useful for an installation.
.
Remote MQTT Gateway. The same as above, but the MQTT gateway can run on a server
with greater availability than a laptop. This has the advantage that multiple
laptops can all subscribe to data from a fleet of sensors. It has the
disadvantage that the development machine must be on the same LAN as the MQTT
broker, or it must be remote for less security and greater latency than a local
deployment.
. Alternate ESP firmware, in
https://github.com/osteele/Arduino-BLE-IMU, which publishes sensor data via
BLE instead of MQTT. This permits use at lower power levels, without WIFi, and
without the need for an MQTT broker.
Currently, only the Chrome browser supports Web Bluetooth; and, the user has to manually initiate each BLE connection each time the page is reloaded. This is a bother during development, and when working multiple devices. As a workaround, the system can also be used in the mode:
. The BLE -> MQTT
Relay is a command-line application that connects as a Central BLE device to IMU
BLE Peripherals, and relays their sensor data to a (local or remote) MQTT
broker.
The Serial -> MQTT Gateway that publishes information from an MCU that is connected to a computer's serial port, to a local or remote MQTT broker.
-
Clone this repo.
-
Install poetry. On macOS running Homebrew, you can also install it via
brew install poetry. -
Run
poetry install -
Install RabibitMQ, or find a RabbitMQ broker. Either: (1) Follow the instructions here to install RabbitMQ on your computer; or (2) get an MQTT hostname, username, and password from your instructor, and use it in the instructions below.
-
Download a GENERIC image from MicroPython Downloads. For example,
esp32-idf3-20191220-v1.12.bin. -
Flash the ESP. If you downloaded the image to
images/esp32-idf3-20191220-v1.12.bin, then run:./scripts/flash images/esp32-idf3-20191220-v1.12.bin
-
Upload the Python source code to the ESP. In a terminal window in this directory, run:
./scripts/py-upload
Run ./scripts/webserver to start a web server.
http://127.0.0.1:8000 displays a directory of web pages in the web directory
directory.
http://127.0.0.1:8000/barchart.html displays a live bar chart of sensor data.
http://127.0.0.1:8000/chart.html uses HighCharts to display another live graph, that automatically scales the y axis as data arrives.
http://localhost:8000/3d-model.html displays the bunny, with its orientation yolked to the IMU orientation. The model is red before the sensor is minimally calibrated, and it fades out when sensor data is not being received.
https://github.com/osteele/Arduino-BLE-IMU is an alternative to the
MicroPython program in ./pyboard, that sends IMU data over a Bluetooth Low
Energy (BLE) connection instead of MQTT.
Follow the instructions in that repository to upload it to an ESP32.
There are two means of using this data:
- In a browser that supports Bluetooth
(e.g. Chrome or Edge; not Safari or Firefox). In Chrome, any of the demos (or
a program that uses the imu-tools npm package) will include a “Connect
Bluetooth" button. Clicking on this button opens the browser's Bluetooth
connection dialog. The button disappears once a device has been connected;
press the
bkey to bring up the connection dialog in order to connect to a second (and third, etc.) device. - Run the BLE->MQTT gateway.
- Once: in the command line, run
npm install. - Each time you start a development session: run
npm run ble2mqtt.
- Once: in the command line, run
poetry run sub runs an MQTT client that subscribes to IMU messages that are
sent to the MQTT broker, and relays them to the terminal.
poetry run pub publishes an MQTT message. The message is a simulated sensor
sample.
./scripts/simulate simulates a device. It continuously publish messages from a
simulated IMU until the user kills (^C) the script. (This is the same as the
--continuous option to poetry run pub.) The simulated sensor readings change
over time.
./scripts/simulate --help gives a list of command-line options. Use
--device-id to simulate a particular ID; you can use this to run multiple
simulations, in different terminals, with different ids.
./scripts/py-upload copies the code in pyboard to the attached ESP, and then
reboots the board.
./scripts/py-repl opens an rshell
REPL to the attached ESP. Press ⌃X to exit the ./scripts/py-repl command.
./scripts/serial-term is an alternative to ./scripts/py-repl, that uses the
GNU screen command instead of
rshell. serial-term connects to the board more quickly than py-repl.
Consult the documentation for screen to see how to exit this command.
MicroPython Development Notes contains notes on developing MicroPython on the ESP.
While running a REPL on the board, press ⌃D to reboot the board. (⌃ is the
control character. Hold it down while the D is pressed, and then release them
both.)
As a proof of concept, IMU data can be used to control the orientation of a rigged joint in Blender.
In a terminal, run: ./scripts/mqtt2pipe
In another terminal, launch Blender with the --python blender/motion.py option:
/Applications/Blender.app/Contents/MacOS/Blender model.blend --python blender/motion.py
Note: If the pipe buffer fills (for example, because Blender is closed), the
mqtt-sub process will hang. You will need to force quit it (^C) and launch it
again.
An ESP that is running the firmware in this directory connects to a WiFi network
and an MQTT broker, and continuously publishes MQTT messages to the
imu/${device_id} topic, where ${device_id} is a unique device id for the
ESP. (It is currently the lowercase hexadecimal representation of the device's
MAC address.) The message payload is a JSON string with properties whose names
are the values of the IMU sensors, fused data, and other data: accelerometer,
gyroscope, magnetometer, euler, quaternion, calibration, temperature. The
message also includes a "timestamp" value, which is the millisecond counter.
This can be used to compute the relative time between samples.
It periodically prints the sample rate to the serial port.
It can optionally be configured to instead send the sensor data.
The serial port format is compatible with osteele/microbit-sensor-relay.
BNO055.py and functools.py are adapted from Radomir Dopieralski's
deshipu/micropython-bno055.
This uses MicroPython and Paho MQTT, and uses rshell.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent