nixes/iothub

A node js application for collecting and visualising data gathered from sensors, pretty MEAN.

Hi there 👋

• 🔭 I’m currently working on a news reliability analysis service
• 🌱 I’m currently learning Dart + Flutter, Spring, Nest, Unity 3D
• 💬 Ask me about architecture, datastructures, AWS infrustructure, devops.
• Looking to collaborate on open source VR projects.
• 📫 How to reach me: email in my profile
• 💖 My favorite tools/frameworks:
  • FE: VueJs (w/typescript)
  • BE: Spring Boot (w/kotlin), Symfony, NestJS
  • DevOps: Terraform, CircleCI

iothub's Issues

allow users to add sensors to the overview page, from sensor settings

add and wire mongoose into api calls

attempt to move data filtering into mongodb from js

Write tests for actor api

fix request by time return results

Currently results returned:

• Hour Returns: 3am to 12:44
• Day Returns: Day
• Week Returns: Week
• Month returns: Month

sensor deletion should be recursive

all objects that reference this sensor id should also be deleted

fix graph styling on mobile devices

currently the graph will be tiny, making bad use of available screen real estate.

when too much data available, return averages

Since the resolution of the display device is limited anyway, we should provide only enough data for the available pixels. To do this and improve data quality we should average data points until we have a small enough number.

implement authentication

Using Oauth2, this should allow me to develop a companion mobile app too.
Oauth2 Server Middleware: https://www.npmjs.com/package/oauth2orize

To allow server to process bearer auth tokens provided by Oauth2 use: https://github.com/jaredhanson/passport-http-bearer

implement versioning

use something like a /api/version endpoint

point hover is borked

something to do with not setting correct background colours

unable to get angular-chart.js working

might be related to some wrong versioning in npm? http://stackoverflow.com/questions/37143278/angular-chart-js-it-doesnt-display-the-chart?noredirect=1&lq=1

In behaviour ui change name of action to response

This makes more sense for the user who has no idea about how things are named internally, and better describes what it will do in terms of a behaviour.

Objectives

• Queryable Input (sensors)
• On Demand Output (actors?)
• IFTTT like commands
• Authentication

MongoDB data queries by time period (hour / day / week)

Sensor data types?

Need ideas for sensor input types, this has implications for what graphs are suitable.

• Number
• Enum / State
• Boolean
• String

How to implement actors?

In this system an actor is defined as anything that performs an action in the world.
The basic example of this would be switching a light on or off. But can include things like another program.

Current thinking is to have each actor provide a compatible REST interface. Actors should register for an id once, then check-in on powerup to update its recorded ip address.

Tests failing

Output:

Data.id expected regex: /^[a-f\d]{24}$/  actual: failed to parse body: invalid character 'N' looking for beginning of value  
--- FAIL: POST /api/sensors 
 api_docs.md:19 - Data.id doesn't match  

Looks like a simple regex fix

add continuous integration for API testing

using http://frisbyjs.com/ to perform API testing

add tagging support to allow grouping of related sensors

add a data summary table below sensor graph

add ability to delete a sensor node from sensor settings

simplify new behaviour creation process

The visual complexity is really a bit high for what should be an easy process, Some ideas for improving this include:

• Some kind of wizard process
• Dim parts of UI that are no longer relevant + guide stepwise

virtual sensors

these might be the result of an intermediary calculation, and should be hidden by default

new sensor update function does not match ID correctly

If id does not exist it seems to add data to the first document regardless of options. No errors are returned.

consider addition of websocket interface for faster WebUI updates

some good examples of using express-ws are found here: http://stackoverflow.com/questions/10112178/differences-between-socket-io-and-websockets

when listing sensors, don't include data

this would massively slow down requests when large amounts of data has been collected.

enable editing of existing behaviours

provide meaningful x-time labels

ability to export / import sensor data to csv

Tasks:

• Export Support
• Import Support

Add time virtual sensor to behaviour system

switch to using http error codes rather than json for success / failure

provide support for sending sensor data in batches

add protocol versioning

Required before MVP release.

deleted sensors remain in selection list until refresh

sometimes graph not rendered

No idea whats causing this at this point.

data limits

Add ability to set data limits for individual sensors, whereby data is dropped if it is older than a certain period or number the of samples reach a certain level.

switch to using reference to data document, rather than nesting data in sensor document

Apparently nested documents have a pretty high performance impact as the number of nested documents increases.
http://stackoverflow.com/questions/15208711/mongoose-subdocuments-vs-nested-schema

templating engine

Swig looks pretty good.

add main tab navigation

use this to massage bootrap tabs with angularjs routes

Add automatic SNYK support, to test security

This would be done much later as I haven't stabilized the api yet.
https://snyk.io/test/

if sensor not registered but data sent anyway server crashes

value was: 19.1
events.js:160
      throw er; // Unhandled 'error' event
      ^

TypeError: Cannot read property 'data' of null
    at /home/callum/iotHub/routes/api.js:204:15
    at Query.<anonymous> (/home/callum/iotHub/node_modules/mongoose/lib/model.js:3324:16)
    at /home/callum/iotHub/node_modules/kareem/index.js:259:21
    at /home/callum/iotHub/node_modules/kareem/index.js:127:16
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickCallback (internal/process/next_tick.js:98:9)

consider making a react fork for learning

react also seems to have initial server side rendering, which seems really attractive.

periodically update charts