ehie-center/prisms-wifi-sensor

Code for Utah Modified Dylos sensors

prisms-wifi-sensor's Issues

Log uncaught exceptions

Currently uncaught exceptions aren't logged in the file. This gives us an incomplete picture of what is going on with the sensor. We need to set up logging to catch all exceptions.

Sensor offline after disconnect from MQTT broker

When a sensor gets disconnected from the MQTT broker, it becomes offline. When it comes back online, sometimes the status does not change to online.

This could be a timing issue between the broker sending the offline last will and the sensor sending the online message.

Add watchdog timer

Under certain conditions, automatically restart the sensor. The two conditions that I can think of right now are no data from the Dylos sensor and unable to connect to WiFi.

Unable to connect to MQTT broker after restart

After restarting the sensor (because of another issue) the sensor is unable to connect to the MQTT broker. For many hours, the sensor keeps on trying but never succeeds.

2018-01-06 06:07:02,208:MainThread:ERROR:__main__:Connection failure...trying to reconnect...
Traceback (most recent call last):
  File "main.py", line 323, in main
    client.connect(mqtt_cfg['server'], mqtt_cfg['port'])
  File "/root/pyenv/versions/3.5.2/envs/dylos/lib/python3.5/site-packages/paho/mqtt/client.py", line 768, in connect
    return self.reconnect()
  File "/root/pyenv/versions/3.5.2/envs/dylos/lib/python3.5/site-packages/paho/mqtt/client.py", line 895, in reconnect
    sock = socket.create_connection((self._host, self._port), source_address=(self._bind_address, 0))
  File "/root/pyenv/versions/3.5.2/lib/python3.5/socket.py", line 693, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/root/pyenv/versions/3.5.2/lib/python3.5/socket.py", line 732, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

At the same time, the sensor is unable to ping the database:

2018-01-06 06:09:47,891:Thread-4:WARNING:sensors.ping:Ping error: ping: unknown host p1db-prisms-p1.bmi.utah.edu

or local network:

2018-01-06 06:09:51,549:Thread-3:WARNING:sensors.ping:Ping error: ping: unknown host gateway.local

For some reason the wireless sensor is not running. Wireless statistics are not getting pushed onto the queue:

2018-01-06 06:08:02,777:Thread-1:DEBUG:__main__:Pushing {'temperature': (69.35, '<C2><B0>F'), 'local_ping_errors': (11, 'num'), 'remote_ping_errors': (5, 'num'), 'sampletime': (1515218882.6456475, 's'), 'large': (91, 'pm'), 'small': (1185, 'pm'), 'remote_ping_packet_loss': (0, 'num'), 'sequence': (1, 'sequence'), 'remote_ping_latency': (0, 'ms'), 'local_ping_total': (11, 'num'), 'humidity': (38.12, '%'), 'local_ping_latency': (0, 'ms'), 'remote_ping_total': (5, 'num'), 'local_ping_packet_loss': (0, 'num'), 'queue_length': (9850, 'num')} into queue

The clock is up to date which makes me think it is connected to the Internet.

Set up ACL

Improve metadata

The following pieces of information would help to have in the metadata:

• Internal IP address
• Software version

Read-only File System

Sometimes a sensor stops working and when sshing, it says

-bash: test.txt: Read-only file system

This seems like a corrupt file system. Is there anything we can do to fix this or at least get notified when it happens?

Crash while getting wireless stats

While trying to get wireless stats, a crash occurs.

2017-12-18 00:00:14,616:Thread-1:ERROR:sensors.wireless:Exception occurred while getting wireless stats
Traceback (most recent call last):
  File "/root/dylos/sensors/wireless.py", line 48, in read
    stats = lines[0].split()
IndexError: list index out of range
2017-12-18 00:00:14,764:Thread-1:ERROR:sensors.wireless:Exception occurred while running iwconfig
Traceback (most recent call last):
  File "/root/dylos/sensors/wireless.py", line 68, in read
    timeout=5)
  File "/root/pyenv/versions/3.5.2/lib/python3.5/subprocess.py", line 626, in check_output
    **kwargs).stdout
  File "/root/pyenv/versions/3.5.2/lib/python3.5/subprocess.py", line 708, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'iwconfig ra0' returned non-zero exit status 237

We handle the crash, but the code that tries to reconnect to WiFi is unable to run. With this crash and if the sensor is disconnected to WiFi, it will never try to reconnect to WiFi.

Update README

README describes the old method (CoAP). We need a description of MQTT.

Allow script to be restart without WiFi restart

When you run

systemctl restart sensor.service

WiFi shutsdown and when the script runs, it reconnects. I would like to not have the WiFi shutdown when the script is stopped.