Website for databat.io
databat-io / sonar Goto Github PK
View Code? Open in Web Editor NEWGoogle Analytics for the real world.
Home Page: http://databat.io
License: Other
Google Analytics for the real world.
Home Page: http://databat.io
License: Other
We used chartit as a quick-win in the early version of Sonar. As it has now been deprecated, it would be desirable to replace it.
Here's where my head is at:
Ok, so I have finally after weeks of bashing away at everything gotten an external wifi adapter going, and running in monitor mode. I have gotten Sonar installed and working, I can access it via the browser.
How long should it be before it starts seeing devices? Our office is chock full of BLE devices, cellphones, even wifi equipment. However, it is doing nothing and showing nothing as far as devices are concerned.
Hi buddy,
So we just need a raspberry to run this all ? No additional sensor or hardware ?
We should create a mapping between vendors and operating system and display icons accordingly.
For instance:
Apple, Inc.
should return should show <i class="fa fa-apple" aria-hidden="true"></i>
Microsoft
should show <i class="fa fa-android" aria-hidden="true"></i>
As explained in #20, Sonar is using Python 2.7 because of historical reasons (that should now have been resolved). Before upgrading Django, we need to upgrade the stack to Python 3.
Right now the documentation is rather Resin-heavy as it is what was used during the development. There should be documentation for running it on a regular Rasbian installation too.
Is it on the roadmap to use a proper BLE sniffing dongle, like this one?
Right now, we're using the base image "resin/raspberrypi3-python".
As @dansku correctly pointed out, this is an old image and needs to be updated.
The most suitable candidate is likely "balenalib/raspberrypi3-debian:jessie-build". However, when performing that upgrade, the installation broke (as a lot of changes have been made since).
Ideally, it should be broken down with a multi-stage build to reduce the footprint further.
I volunteer at an event which runs over several buildings, and was planning on installing a Raspberry Pi Zero W at entrance to each building to capture the data. At the end of the event, i would hope to export all the data into a spreadsheet, remove duplicates (from say 3 devices) and then have a reasonable approximate of unique traffic through the event.
Feasible?
A very useful metrics would be return visitors. For instance, how many visitors return within 180, 60, 30 days etc.
Create a Jypiter Notebook that visualizes (perhaps with Plotly) some of the common reports.
Service exited 'celery sha256:XXX'
Running in Balena on Pi 4 through Balena Cloud
XXX is the sha
File "/usr/src/app/collector/celery.py", line 3, in
gunicorn from celery import Celery
gunicorn ImportError: cannot import name 'Celery' from 'celery' (/usr/local/lib/python3.7/dist-packages/celery/init.py)
gunicorn [2023-04-22 13:08:35 +0000] [25] [INFO] Worker exiting (pid: 25)
gunicorn [2023-04-22 13:08:35 +0000] [22] [INFO] Shutting down: Master
gunicorn [2023-04-22 13:08:35 +0000] [22] [INFO] Reason: Worker failed to boot.
Service exited 'gunicorn sha256:XXX'
In order to extract useful reports, we need to create some kind of front-end behind a login with a user friendly way to get graphs. D3.js appears to be a suitable tool.
Good day, i'm new to resin.io / balena, this is the first time trying something like this on the pi 3b+. I managed to push the code to my balena acc, the build finished but nothing happens after... i was thinking to access the interface would be pi ipadd:8000 but nothing, port 80 says bad gateway
11.12.18 20:49:24 (-0400) main 109 static files copied to '/usr/src/app/staticfiles', 151 post-processed.
11.12.18 20:49:24 (-0400) main Attaching hci0...
11.12.18 20:49:24 (-0400) main bcm43xx_init
11.12.18 20:49:24 (-0400) main Failed to write reset command
11.12.18 20:49:24 (-0400) main Can't initialize device: Success
11.12.18 20:49:24 (-0400) main Bring hci0 up...
11.12.18 20:49:24 (-0400) main Scan for devices...
11.12.18 20:49:40 (-0400) main Inquiry failed: Connection timed out
11.12.18 20:49:40 (-0400) main Initialization failed.
11.12.18 20:49:40 (-0400) main Soft error counter: 1/3
11.12.18 20:50:06 (-0400) main Attaching hci0...
11.12.18 20:50:06 (-0400) main bcm43xx_init
11.12.18 20:50:06 (-0400) main Failed to write reset command
11.12.18 20:50:06 (-0400) main Can't initialize device: Success
11.12.18 20:50:06 (-0400) main Bring hci0 up...
11.12.18 20:50:06 (-0400) main Scan for devices...
11.12.18 20:50:22 (-0400) main Inquiry failed: Connection timed out
11.12.18 20:50:22 (-0400) main Initialization failed.
11.12.18 20:50:22 (-0400) main Soft error counter: 2/3
11.12.18 20:50:42 (-0400) main Attaching hci0...
11.12.18 20:50:42 (-0400) main bcm43xx_init
11.12.18 20:50:42 (-0400) main Failed to write reset command
11.12.18 20:50:42 (-0400) main Can't initialize device: Success
11.12.18 20:50:42 (-0400) main Bring hci0 up...
11.12.18 20:50:42 (-0400) main Scan for devices...
11.12.18 20:50:58 (-0400) main Inquiry failed: Connection timed out
11.12.18 20:50:58 (-0400) main Initialization failed.
11.12.18 20:50:58 (-0400) main Soft error counter: 3/3
11.12.18 20:51:47 (-0400) main Hard error counter: 1/5
11.12.18 20:51:47 (-0400) main All tests failed. Taking hard action in 249 seconds.
A useful metric to track would be dwell, i.e. visitors that stay longer than say 15 minutes. This is a common metrics used in retail as a KPI.
Regardless if the data source is BLE or WiFi, we should be able to figure out what the manufacturer of the device is. This can then be used to create better reports.
Because the Bluetooth libraries failed in Python 3, the project is using Python 2.7. This appears to have been addressed now. Hence we should be able to upgrade to Python 3 and subsequently Django 2 (and now 3).
Some of the code is hard to test, but we should at least get some tests going (and hook it up with CI).
BLE provides good insight, but we can also collect metrics from WiFi devices in the surrounding area.
Out of necessity (to access the bluetooth chip), the celery
container must run as root. However, this is not true for the (publicly exposed) gunicorn
container. We should drop permission of this container to nobody. As part of that, we also need to alter the permission of of the SQLite database (chown nobody:nobody
) in order for the gunicorn
container to be able to interact with the database.
In the current iteration, Sonar will report any BLE devices it detects. That might be useful in some settings, but in other settings, you want to dial back on the sensitivity such that we only pick up on devices close by.
We have the ability to do this using Received Signal Strength Indication (RSSI). According to this article, there we can map the correlation between RSSI and distance. It won't be fully accurate, but good enough as a proxy.
Now with resin supporting multi-containers, we can make the project a bit more clean by adding deprecating the systemd hacks and split up the app into proper containers.
It would be desirable to collect WiFi metrics (see #10). However, it would also be great to collect WiFi data from something like a Unifi Controller using their rest api.
In order to do this, we need:
W: GPG error: http://security.debian.org/debian-security bullseye-security InRelease: At least one invalid signature was encountered.
E: The repository 'http://security.debian.org/debian-security bullseye-security InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian bullseye InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian bullseye InRelease' is not signed.
W: GPG error: http://deb.debian.org/debian bullseye-updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://deb.debian.org/debian bullseye-updates InRelease' is not signed.
W: GPG error: http://archive.raspbian.org/raspbian bullseye InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.raspbian.org/raspbian bullseye InRelease' is not signed.
W: GPG error: http://archive.raspberrypi.org/debian bullseye InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.raspberrypi.org/debian bullseye InRelease' is not signed.
The command '/bin/sh -c install_packages wget redis' returned a non-zero code: 100
ERROR: Service 'redis' failed to build : Build failed
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.