gut-space/satnogs

The goal of this project is to create a functional satellite ground station, able to receive VHF transmissions from NOAA sats and more!

The goal of this project is to build a fully functional automated VHF satellite ground station, loosely based on satnogs project.

Project founders: Sławek Figiel and Tomek Mrugalski

Project status

This is an old, unmaintaned fork. For up to date code, see https://github.com/gut-space/svarog

Documentation (old)

• Installation
• Architecture
• User Management
• Project report - a report from the early days when this was a team university project
• Project poster 1
• Project poster 2

satnogs's Issues

Pick the right box - RPi?

We need to choose the right computer hardware to run the software to drive the project.

Tomek has Raspberry Pi 2+ and is more than glad to donate it. Is this sufficient? Do we need a newer model?

CORRECTION: It's B+, not 2+.

Originally posted in #4 by Tomek Mrugalski at 2019-09-15T13:41:24.933Z

DB migrations

Our current approach to migrations is a bit primitive. Consider using something similar to: https://pypi.org/project/yoyo-migrations/ or https://pypi.org/project/raw-sql-migrate/

The goal of our project - what will we do with the data received?

Prof. Moszynski said the goal of our project cannot be just connecting to a satellite. We have to determine what exactly we want to achieve. What data do we want to receive and what to do with it?

The easiest way will probably be to receive and present some weather data or photos(?)

Originally posted in #9 by Ewelina Omernik at 2019-10-12T23:48:11.718Z

Observation API

We need a new API that will:

• receive an observation from reporting ground station
• rename uploaded file
• store an observation in a DB
• rename file using a scheme that ensures no conflicts
• lastobs field in the stations table should be updated

It seems that over time more and more parameters will be provided when adding observation. The schema should be easily extensible. One good example would be a POST method with a JSON data and JSON response. This would also ensure easy adaptation by third parties.

Task 4: Software automation design

Task 4 of our project plan covers the following:

**Design of the automated data acquisition, processing pipeline, data deployment **

Engineer responsible: @slawek

This task also covers writing the report section about the design.

Originally posted in #15 by Tomek Mrugalski at 2019-11-09T08:36:56.018Z

Connection between antenna and RPi

We need to decide where place the RPi and SDR. We have few options:

1. On the roof, near the antenna
2. In the laboratory, using long antenna cable
3. SDR on the roof, RPi in the laboratory

On the roof

In this case we put the RPi and SDR on the roof, probably in waterproof box, near (or on) the antenna (far ~0,5 m).
It should minimize the noises. But upgrade or manual reset of RPi is hard, because we cannot personally get on the roof.
Other problem is communication with RPi. We don't know if we have Ethernet connection. If we don't have a possibility to connect to the network then we need plug the GSM modem to RPi.
We know nothing about power supply on the roof too.

In the laboratory

Dr Siwicki says that exists the antenna cable from the roof to one laboratory on 7 level and we probably can use it.
But this solution increase the noises. But we will to have a free access to our electronic components.

SDR on the roof, RPi in the laboratory

The noises occur when we work with analog signal. We have analog signal in antenna, between antenna and SDR and partially in SDR. SDR produces digital signal. We may put the antenna and SDR on the roof to convert signal to digital and next lossless transfer it to RPi in the laboratory.
The main problem here is length of the cable. Some digital cables have limited length to few meters and it isn't enough. We don't know also if we have a possibility to install this cable.

To Do:

• Check if we have a Ethernet on the roof
• Check specification of the power supply on the roof
• Check how long cable we need between antenna place and laboratory
• Measure how noise increases depending on the distance
• Check if we have possibility to install any, new, own cable
• Research possible solutions to connect SDR to RPi (only USB? maybe Ethernet? or other?)
• Solve how we want to provide power for RPi
• Solve how to provide power for LNA (optional, only if we decide to use LNA)
• Analyse the static protection SDR or LNA

Originally posted in #5 by Sławek Figiel at 2019-10-03T20:26:40.911Z

Task 6: Test campaign

The 6th task specified in the project plan is the following:

Test specification, experimental assessment of the system performance, test report, improvement suggestions, conclusions

Engineer responsible: @ewe

Originally posted in #17 by Tomek Mrugalski at 2019-11-09T08:42:17.620Z

pagination exception when clicking last page

I managed to get this exception when browsing observations list:

ERROR:app:Exception on /obslist [GET]
Traceback (most recent call last):
  File "/home/satnogs/devel/satnogs/server/venv/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/satnogs/devel/satnogs/server/venv/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/satnogs/devel/satnogs/server/venv/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/satnogs/devel/satnogs/server/venv/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/satnogs/devel/satnogs/server/venv/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/satnogs/devel/satnogs/server/venv/lib/python3.7/site-packages/Flask-1.1.1-py3.7.egg/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/satnogs/devel/satnogs/server/venv/lib/python3.7/site-packages/webargs-5.5.3-py3.7.egg/webargs/core.py", line 455, in wrapper
    return func(*args, **kwargs)
  File "/home/satnogs/devel/satnogs/server/app/pagination.py", line 323, in wrapped
    limit_and_offset=limit_and_offset)
  File "/home/satnogs/devel/satnogs/server/app/controllers/obslist.py", line 15, in obslist
    obs["sat_name"] = satellites_dict[obs["sat_id"]]
KeyError: None

Steps to reproduce:

1. go to https://satnogs.klub.com.pl/obslist
2. click on Last page (or any other non-consecutive page, e.g. jump from 1 page to 5th).

You can see the exception on ~satnogs/logs/wsgi.log

Come up with a design for the content server

Our current "website" (that's a fancy name for index.php) is no longer suitable for the current state of the project. We now gather 15 observations a day.

We need a website that offers filtering and sorting. There should be about page and another one with documentation etc.

Pick the main satellite for observe

I think that we should select one, main satellite to observe and prepare first version of our station for it.
It will allow us to focus on the task and we will to have a clear goal.

This satellite should be high above horizon and fly often above Gdańsk.
The best if this satellite transmit the graphical data (for example weather maps).

On the same frequency and similar orbit we should find two another satellites as backup.

Originally posted in #7 by Sławek Figiel at 2019-10-10T15:32:59.329Z

Choose the LNA

We need the analysis the possibility to use Loise Noise Adapter.
It increase the received quality signal by decrease noise level.
This component is optionally, but it is highly recommended to use it.

The LNA must be compatible with used frequency and antenna and SDR connector.

I propose to wait with it until choose antenna and SDR.

Originally posted in #8 by Sławek Figiel at 2019-10-10T17:54:22.538Z

Choose the location

Ok, once we assemble the station, we need to deploy it somewhere. There are at least 3 possible locations:

• ETI faculty roof. This location has the advantage of being reasonably easy accessible by every participant. This is important, especially in the future if the project is successful and the team grows. The downside is that it requires permissions from unspecified ETI official.

• Balcony/roof of Tomek's apartment. Tomek has a large balcony, so we can start deploying the station there. The benefit is that the access is much easier than a roof. The disadvantage is that it's easily accessible only when Tomek is around. In some time in the future, this could be relocated to the roof, which would have better horizon exposure than a balcony. This would require an approval from the community (wspólnota mieszkaniowa), but I can probably take care of this. Building a fireplace required making a huge hole in the roof and I managed to do it.

• An undisclosed location in Bory Tucholskie. Tomek has a small piece of land. We can deploy anything there. There's plenty of space. The disadvantage is that any hardware modification requires a 70 mins drive from Gdansk. The benefit is that it's literally in the middle of nowhere, so there should be less interference, no wifi, no planes flyovers etc.

Originally posted in #3 by Tomek Mrugalski at 2019-09-15T13:39:35.587Z

Choose the right antenna

Ok, we also need to pick the right antenna. Currently I know nothing about antennas.

It seems reasonable to start with a fixed, non-moving omnidirectional antenna.

The goal of this ticket is to determine which antenna we want to buy and where to buy it, so shipment to Poland doesn't take ages or is not too complicated.

Originally posted in #2 by Tomek Mrugalski at 2019-09-15T13:31:45.522Z

flake8 failure: undefined name 'VersionNotSetError'

I've did some experiments with github actions. I've added an action to test python app. You can inspect it by clicking Actions on the main repo page. Here's the error being reported:

Run pip install flake8
Collecting flake8
  Downloading flake8-3.7.9-py2.py3-none-any.whl (69 kB)
Collecting mccabe<0.7.0,>=0.6.0
  Downloading mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Collecting pycodestyle<2.6.0,>=2.5.0
  Downloading pycodestyle-2.5.0-py2.py3-none-any.whl (51 kB)
Collecting entrypoints<0.4.0,>=0.3.0
  Downloading entrypoints-0.3-py2.py3-none-any.whl (11 kB)
Collecting pyflakes<2.2.0,>=2.1.0
  Downloading pyflakes-2.1.1-py2.py3-none-any.whl (59 kB)
Installing collected packages: mccabe, pycodestyle, entrypoints, pyflakes, flake8
Successfully installed entrypoints-0.3 flake8-3.7.9 mccabe-0.6.1 pycodestyle-2.5.0 pyflakes-2.1.1
./server/app/repository.py:342:19: F821 undefined name 'VersionNotSetError'
            raise VersionNotSetError()
                  ^
1     F821 undefined name 'VersionNotSetError'

Details here

Store obit information TLE with observations

Due to orbit perturbations, TLE data is changing slowly over time. We need to store that information with the observations. In the future, we'll use it to geotag photos and then possibly overlay country contours.

Task 3: System integration

Task 3 of our project plan covers the following:

Hardware (computing unit, SDR, antenna, wiring), assembly, base software installation (OS, SDR drivers, SDR software)

Engineer responsible: @ewe

That is mostly covered by #13, but we also need to write a section in the final report that will document what exactly was integrated and how.

Originally posted in #14 by Tomek Mrugalski at 2019-11-09T08:35:25.672Z

Timetable

We need a prepare timetable.

1. Analyse and choose a mission objectives [3 weeks]

2. Learn and test the software - prepare dev environment [2 weeks]

3. Prepare an order for hardware [2-3 days]

4. Delivery [1 week]

5. Testing the station [1 month]

6. Installation the station in target place [2 week]

7. Prepare software for automatically select satellite for observer

Originally posted in #6 by Sławek Figiel at 2019-10-10T08:04:47.233Z

Purchase TA-1 Turnstile antenna.

Following on the #2 discussions, we decided to purchase Winkler turnstile antenna. This issue covers the purchase process.

Originally posted in #12 by Tomek Mrugalski at 2019-10-20T18:20:19.871Z

# of observations statistic is incorrect

The number of observations for two stations is swapped.

1. Go to: https://satnogs.klub.com.pl/stations
2. It say's that ETI-1 has tons of observations, but in fact it's TKiS-1 does them.

Also, the registered and last observations are swapped, too.

Acquire SDR hardware

We need to get a reasonable SDR hardware. I've found this one: RTL-SDR v3 R820T2 on allegro. This seems like a good start.

I also have a very cheap (45pln) SDR DVB-T tv tuner on usb. I'm travelling right now and don't know the exact model. It's a cheap one with plastic case. But I think it's based on the same chip. So it could be used for early experiments, at least before the proper hardware arrives.

Originally posted in #1 by Tomek Mrugalski at 2019-09-15T13:29:40.488Z

Configure remote access to RPi

We need to set up Raspberry Pi in a way that it's remotely accessible.

This requires a global IP, setting up separate VLAN, configuring DHCP for new zone, punching a hole in a firewall and forwarding a port (and a couple other things).

Originally posted in #11 by Tomek Mrugalski at 2019-10-20T17:33:10.420Z

Nav menu should highlight the current page

Right now the top navigation panel is very simple and it always highlights Home. It should highlight the current page that is open.

README for station

We need to prepare README file with instructions how to use our console tool.

Searching, filtering by station

Once #19 is done, we can implement a better filtering and searching mechanisms. In particular:

• show reporting station on the list of all observations
• allow filtering for specific station
• on the station details page, show all reported observations
• pagination

Version isn't displayed on production.

Version and timestamp isn't present on production server.
We need fix it and add unittest for check if it works.

Deployment model - server

Deployment model (how to deploy new versions)

After accept changes the new version of software should be automatically build and install on our ground station and content server. Other ground stations connected to content server should get notification that update is required.
All necessary external dependencies must be installed automatically.

master doesn't work on python 3.7 and 3.6

The recent change in backend/app/repository.py imports TypedDict from typing. Unfortunately, on python 3.7.5 it does fails:

File "/home/thomson/devel/satnogs/backend/app/repository.py", line 9, in <module>
    from typing import Any, Dict, List, TypedDict, NewType, Sequence, NoReturn, Union, Optional, Tuple
ImportError: cannot import name 'TypedDict' from 'typing' (/usr/lib/python3.7/typing.py)

See https://docs.python.org/3.7/library/typing.html and https://docs.python.org/3.8/library/typing.html

must enable unit-tests for automation

Automation/ dir has a small (a total of 2, but we need to start somewhere) number of tests. Sadly, those are not run in our CI. They should.

Experiment with LRPT reception and decoding

Right now we're only able to decode APT transmission, which is a 60s technology being used by only 3 NOAA sats and it is being retired. Newer sats (e.g. NOAA-20) no longer use APT. Our current hardware should allow us to receive LRPT transmissions.

According to wikipedia, Metop and NPOESS use LRPT. Metor 2-M use LRPT, too. We should do some manual experiments and see how tricky it is to receive such transmissions.

Task 5: Software implementation

Our project plan covers the following task:

Implementation of the design specified in task #4 (#15), developed software deployment

Engineer responsible: @thomson

Originally posted in #16 by Tomek Mrugalski at 2019-11-09T08:39:10.559Z

Setup the dev environment

Thomas created the dev environment in his home.
We have there RPi B+, target SDR and simple antenna.
We need now to install software for tests.

• Install OS
• Configure network, firewall, ports, security, SSH
• Install drivers
• Install and configure GNU Radio
• Install additional software (GQRX? check if is possible run it with GUI by SSH)
• Write manual how to use it using SSH

Originally posted in #13 by Sławek Figiel at 2019-10-20T18:31:58.101Z

Store more than one artifact per observation

The server MUST be able to store more than one artifact per observation (decoded image for now, but in the future also waterfall, radar plot, recorded audio and possibly more).

Now our DB layout allow us to store only one photo attached to an observation.