argeniss-software/rolaguard

RoLaGuard (Community Edition): a LoRaWAN Operations, Cybersecurity & Compliance Management Platform.

RoLaGuard Community Edition

A LoRaWAN Operations, Cybersecurity & Compliance Management Platform.

Docker Compose is needed to run RoLaGuard. To install docker-compose, go to this link and follow the instructions for your OS. Note that for Windows and MacOS, docker-compose is included in Docker desktop. For Linux, you'd have to download both programs separately.

Clone this project and its submodules with git clone --recurse-submodules https://github.com/Argeniss-Software/rolaguard && cd rolaguard

If you are using Windows, make sure to clone this project and execute the following commands from the linux shell with WSL.

Run RoLaGuard using Makefile

To start up the system locally with all the modules running inside docker containers, simply run make all command. This will clean up the database, pull the develop branch of all modules, and build the images. It'll take a time to finish its execution, so please be patient. Go to the Makefile to see other options.

Run RoLaGuard manually using docker-compose

After docker-compose is installed, follow these steps:

1. Fetch the latest changes for each submodule:

   git submodule foreach --recursive '
   if [ "$name" != "postgres" ]; then
       git checkout develop && git pull;
   else
       git checkout master && git pull;
   fi'

2. Execute docker-compose build. Running this command will build the images locally. It'll take a time to finish its execution, so please be patient.

3. Execute docker-compose up.

Access RoLaGuard

Once you have executed docker-compose up or make all, wait until all the components init (it may take a couple minutes) and access the RoLaGuard web interface following this link http://localhost:30000.

To access the system use these credentials:

• user: admin
• password: admin

Please, remember to change the password.

An open network server is pre-loaded, therefore, some alerts must appear.

All the data generated by the the system will be saved in ~/rolaguard, to be persisted when the system is shutdown. The folder used to persist data can be changed modifying the field services->postgres->volumes in the docker-compose.yml file.

Additional features

• If you intend to utilize RoLaGuard as a server accessible from other devices, navigate to the rolaguard\config\frontend.env file and replace the "${DOMAIN}" attribute with the appropriate IP address.

• If you wish to send email notifications, navigate to rolaguard\config\backend.env, change the field SEND_EMAILS to True, and modify the fields below (there are example comments on how they should be configured).

After making these changes, execute docker-compose build/up or make all again for them to take effect.

Documentation

• Instruction for TTN connection

• Instruction for TTN v3 connection

rolaguard's Issues

Documentation required on how to run without the built-in chirpstack server

Describe the bug

First of all, this is an amazing project, thank you so much for releasing it!

It would be great to know exactly which of the docker containers are needed for a "production" deployment of the tool, i.e. one without the built-in chirpstack.

For example, are RabbitMQ and Postgresql part of rolaguard, or are they only required for Chirpstack?

I'm happy to help write this up if you can point me in the right direction!

To Reproduce

N/A

Expected behavior

Documentation exists to help deploy this somewhere other than docker-compose with Chirpstack.

Screenshots

N/A

[Data Sources] 500 error when the user try to add a new Data source with a Type

Describe the bug
As a user I want to add a new Data Source with a type and the system is showing a 500 error

To Reproduce
Steps to reproduce the behavior:

1. Go to data source section
2. Complete the required files
3. add a type
4. Click on Test connection

Expected behavior
The system shoud test the connection and verify it

Actual result
The system is showing a 500 error

Additional context
This is only happening when the user add a Type. If the user doesn't add a type the system is working fine.

[Alerts] - Can't select an Start date different that today

Describe the bug
As a user I want to select a day in the past to filter the results but the date is not saving.

To Reproduce
Steps to reproduce the behavior:

1. Go to Alerts
2. Click on Start date on the filter section
3. Select a day in the past
4. Select an hour

Expected behavior
The system should filter the results between the dates that the user select

Screenshots

[Alerts] Error by clicking on the next page button

To Reproduce
Steps to reproduce the behavior:

1. Go to Alerts section
2. Click on Next page button on the grid

Expected behavior
If the grid has no items to show, the next button should be disable.
If the grid has two pages and the user clicks on the next button the next page should be shown

Screenshots

Desktop (please complete the following information):

• Browser chrome

Note: This is only happening if the grid is empty

Sortable titles on the grids

As a user I want to sort grids by the title. Asc, desc, default
So it is easily to find the data.

Failed message parsing

Describe the bug
The data-collector cannot parse the message from the gateway.

Log

--- Logging error ---
2021-05-17T13:25:01.857048206Z  Traceback (most recent call last):
2021-05-17T13:25:01.857050267Z    File "/root/app/auditing/datacollectors/LoraServerIOCollector.py", line 305, in on_message
2021-05-17T13:25:01.857052381Z      standard_packet['modu'] = data_rate.get('modulation')
2021-05-17T13:25:01.857054493Z  AttributeError: 'NoneType' object has no attribute 'get'
2021-05-17T13:25:01.857056511Z  
2021-05-17T13:25:01.857058665Z  During handling of the above exception, another exception occurred:
2021-05-17T13:25:01.857060818Z  
2021-05-17T13:25:01.857073146Z  Traceback (most recent call last):
2021-05-17T13:25:01.857075270Z    File "/usr/lib/python3.7/logging/__init__.py", line 1034, in emit
2021-05-17T13:25:01.857079975Z      msg = self.format(record)
2021-05-17T13:25:01.857082044Z    File "/usr/lib/python3.7/logging/__init__.py", line 880, in format
2021-05-17T13:25:01.857084244Z      return fmt.format(record)
2021-05-17T13:25:01.857086378Z    File "/usr/lib/python3.7/logging/__init__.py", line 619, in format
2021-05-17T13:25:01.857088692Z      record.message = record.getMessage()
2021-05-17T13:25:01.857090797Z    File "/usr/lib/python3.7/logging/__init__.py", line 380, in getMessage
2021-05-17T13:25:01.857096189Z      msg = msg % self.args
2021-05-17T13:25:01.857098317Z  TypeError: not all arguments converted during string formatting
2021-05-17T13:25:01.857100489Z  Call stack:
2021-05-17T13:25:01.857102545Z    File "/usr/lib/python3.7/threading.py", line 885, in _bootstrap
2021-05-17T13:25:01.857104659Z      self._bootstrap_inner()
2021-05-17T13:25:01.857106694Z    File "/usr/lib/python3.7/threading.py", line 917, in _bootstrap_inner
2021-05-17T13:25:01.857108781Z      self.run()
2021-05-17T13:25:01.857110758Z    File "/usr/lib/python3.7/threading.py", line 865, in run
2021-05-17T13:25:01.857113060Z      self._target(*self._args, **self._kwargs)
2021-05-17T13:25:01.857115287Z    File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3452, in _thread_main
2021-05-17T13:25:01.857117467Z      self.loop_forever(retry_first_connection=True)
2021-05-17T13:25:01.857119585Z    File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1779, in loop_forever
2021-05-17T13:25:01.857121716Z      rc = self.loop(timeout, max_packets)
2021-05-17T13:25:01.857123767Z    File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1181, in loop
2021-05-17T13:25:01.857125821Z      rc = self.loop_read(max_packets)
2021-05-17T13:25:01.857127844Z    File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 1572, in loop_read
2021-05-17T13:25:01.857130090Z      rc = self._packet_read()
2021-05-17T13:25:01.857132189Z    File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2310, in _packet_read
2021-05-17T13:25:01.857134383Z      rc = self._packet_handle()
2021-05-17T13:25:01.857136499Z    File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 2936, in _packet_handle
2021-05-17T13:25:01.857138583Z      return self._handle_publish()
2021-05-17T13:25:01.857147571Z    File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3216, in _handle_publish
2021-05-17T13:25:01.857149877Z      self._handle_on_message(message)
2021-05-17T13:25:01.857151923Z    File "/usr/local/lib/python3.7/dist-packages/paho/mqtt/client.py", line 3444, in _handle_on_message
2021-05-17T13:25:01.857154042Z      self.on_message(self, self._userdata, message)
2021-05-17T13:25:01.857156123Z    File "/root/app/auditing/datacollectors/LoraServerIOCollector.py", line 99, in <lambda>
2021-05-17T13:25:01.857158345Z      self.mqtt_client.on_message = lambda client, userdata, msg: self.on_message(client, userdata, msg)
2021-05-17T13:25:01.857160699Z    File "/root/app/auditing/datacollectors/LoraServerIOCollector.py", line 438, in on_message
2021-05-17T13:25:01.857162904Z      json.dumps(mqtt_messsage) if is_protobuf_message else msg.payload.decode("utf-8"))
2021-05-17T13:25:01.857165243Z  Message: 'Error creating Packet in Chirpstack collector:'
2021-05-17T13:25:01.857167438Z  Arguments: (AttributeError("'NoneType' object has no attribute 'get'"), 'Topic: ', 'gateway/00aeaefffe000016/event/up', 'Message: ', '{"phyPayload":"AL3GraPODsb57T4AAArKJQCfXCDdLOQ=","txInfo":{"frequency":868500000,"modulation":"LORA","loRaModulationInfo":{"bandwidth":125,"spreadingFactor":9,"codeRate":"4/5","polarizationInversion":false}},"rxInfo":{"gatewayID":"AK6u//4AABY=","time":"2021-05-17T13:25:01.577240Z","timeSinceGPSEpoch":"1305293119.577s","rssi":-99,"loRaSNR":-14,"channel":7,"rfChain":0,"board":0,"antenna":0,"location":null,"fineTimestampType":"ENCRYPTED","encryptedFineTimestamp":{"aesKeyIndex":0,"encryptedNS":"b+YoGvCf7xj81Xo+s2hvFA==","fpgaID":null},"context":"r85q/A==","uplinkID":"uM8nnPeORxqIJjWCz4ViPA==","crcStatus":"CRC_OK"}}')

supported network servers?

Hello

Which network servers are currently supported?

Is the things stack v3 ?
Chirpstack?

Very promising project

TTN gateway packet count issue

Describe the bug
TTN gateway packet count inconsistent with information in TTN console.

To Reproduce
Add a TTN gateway. Note packet count in TTN console. Wait a couple of hours and compare the counts in Asset 360 view of the gateway with the delta of the current value in TTN console and the previously noted value.

Expected behavior
Number of uplink and downlink messages should match the delta in TTN console or be close. After 23 hours runtime the 360 view reports 2 uplink and 5 downlink for a gateway that receives at least 30 messages each hour (uplinks) from a 'canary' node.

Desktop (please complete the following information):

• OS: Windows 10
• Browser firefox
• Version 82.0.2

Avoid logging too many errors on containers initialization

Describe the bug
The docker containers usually log some errors when starting. This happens because some parts of the system, like the database and rabbitmq, refuse the connection attempts made by the backend, the engine and the packer-writer. After some short time, the database and rabbitmq become ready to accept connections and everything start working fine. However, it would be better to avoid logging these initial errors as it can be confusing.

To Reproduce
Steps to reproduce the behavior:

1. Initialize docker containers with "docker-compose up -d"
2. Check the logs of the containers, for example: "docker logs backend"
3. Visualize error logs related with sqlalchemy and rabbitmq connections.

Expected behavior
Each service should try to be initialized a fixed number of times, without logging any errors between different attempts, and only if all these attempts fail, then an error should be logged and the initialization aborted.

Cannot access Rolaguard from other PC

Describe the bug
A browser on a PC on the same network as the PC with Rolaguard cannot login at Rolaguard while accessing from localhost is no problem. The progress circle keeps circling

To Reproduce
Steps to reproduce the behavior:

1. Start browser on other PC
2. go to :30000
3. login with admin credentials
4. See error

Expected behavior
Successfull login.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: Windows 10
• Browser Firefox
• Version 85.0

Add tls authentication and hostname support for chirpstack/mqtt

Is your feature request related to a problem? Please describe.
Currently, when I add a chirsptack server as datasource, I can't use a hostname for the MQTT server, only an IP address, and I can't specify tls authentication using a certificate and private key. This makes the MQTT server connection impossible for my chirpstack setup.

Describe the solution you'd like
I would like fields added where I can upload the tls certificates for my mqtt client connection (certificate, private key and ca cert), and change the server ip address, to also allow hostnames.

Additional context
Our MQTT server is an AWS-IOT server, and we are running chirsptack servers in EC2 instances. We won't be able to use RolaGuard without this.

Failed to start docker images or failed to login with default username and password

Describe the bug
On Fedora Workstation 33, when I try to start rolaguard, I see errors that the packet-writer and backend aren't able to connect to the postgresql server on port 5432. When I open http://localhost:30000/ and try to login, I get an error that the username or password are incorrect.

To Reproduce
Steps to reproduce the behavior:

1. clone the rolaguard repository (master branch)
2. docker-composer up
3. See errors in the terminal
4. optionally, visit http://localhost:30000 and try to login with admin admin
5. see the error

Expected behavior
The docker images start up without errors, and the login succeeds with the default username and password.

Screenshots

Desktop (please complete the following information):

• OS: Fedora Workstation 33
• Any

Additional context
Add any other context about the problem here.

Failed to add user

Describe the bug
logged in as admin. Cannot create new user. after adding the text appears: Wa are sorry. It has been an error while creating the user" No new user is displayed.

To Reproduce
try to add user in New user window

Expected behavior
A user is added

Cannot remove gateway entries from test network

Describe the bug
After setting up the system, the default collector is removed.
The gateways that have been "heard" by the system cannot be removed and keep cluttering mij system.

Expected behavior
A way to remove unwanted or faulty data.

[Inventory] Error by clicking device button

Describe the bug
As user when I click on the device button on the grid the system is showing a 500 error

To Reproduce
Steps to reproduce the behavior:

1. Go to Inventory
2. Click on device

Expected behavior

Actual result
The system is showing a 500 error

Note: This is happening only when the grid is empty