szpajder/dumpvdl2

VDL Mode 2 message decoder and protocol analyzer

dumpvdl2's Issues

IQ File Piping

dumpvdl2 --iq-file <file_name> --centerfreq 136955000 136975000

Anyway to pipe in STDIN here? I tried the obvious:

./ss_client iq -r 176.176.220.12 -q 5555 -s 24000 -f 136725000 - | dumpvdl2 --iq-file - 136725000

but dumpvdl2 output:

dumpvdl2 2.0.0-65988a4 (libacars 2.0.1)
Sampling rate set to 1050000 sps
fopen(): No such file or directory

which tells me it's not aware of STDIN pipe when passed -.

You can get the spyserver_client at https://github.com/miweber67/spyserver_client if you'd like to try using it with a remote server when trying to test stdin pipe. I have an GH issue open there (miweber67/spyserver_client#6) as well.

OpenSky database support

Today it's impossible to find openly available up to date basestation.sqb
At same time OpenSky project keeps it's aircraft database up to date:

https://opensky-network.org/datasets/metadata/

It would be nice if this DB can be used with dumpvdl2

Latest version stops

Hi Tomasz,

I have been trying the latest version, but I run into some problems.
I'm running in debug mode and PLATFORM=rpiv3.

sometime the rpi stop responsing. The lastest error was: device #0: async read failed.

I have tried different dongles, different sd cards.
If I switch back to the old version, it runs without any problems.

Do you have any ideas?

All the best,

Albert

No messages at all

Hello
My problem is that i don;t get any messages (i readed FAQ).
When i start dumpvdl2 with dumpvdl2 --rtlsdr 0 --gain 19.7 --correction 23 --centerfreq 136650000 136725000 136775000 136975000 136875000
I use --gain 19.7 because my Lifeview LV5TDeluxe Fitipower FC0013 tuner cannot go more

It either find maybe 4 msgs or none at all after 20 min.

When i use on same devices vdlm2dec i get bunch of msgs just from the start.
Testedd 1.6.0 and 1.7.0

Whats going on?

Build failure with gcc 10

dumpvdl2 fails to build with recent gcc versions (>= gcc10) since gcc now defaults to -fno-common

[   29s] /usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-linux/bin/ld: CMakeFiles/dumpvdl2_base.dir/demod.c.o (symbol from plugin): in function `sbuf':
[   29s] (.text+0x0): multiple definition of `decoder_thread_active'; CMakeFiles/dumpvdl2_base.dir/decode.c.o (symbol from plugin):(.text+0x0): first defined here
[   29s] /usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-linux/bin/ld: CMakeFiles/dumpvdl2_base.dir/dumpvdl2.c.o (symbol from plugin): in function `sighandler':
[   29s] (.text+0x0): multiple definition of `decoder_thread_active'; CMakeFiles/dumpvdl2_base.dir/decode.c.o (symbol from plugin):(.text+0x0): first defined here
[   29s] /usr/lib64/gcc/x86_64-suse-linux/10/../../../../x86_64-suse-linux/bin/ld: CMakeFiles/dumpvdl2_base.dir/input-raw_frames_file.c.o (symbol from plugin): in function `input_raw_frames_file_process':
[   29s] (.text+0x0): multiple definition of `decoder_thread_active'; CMakeFiles/dumpvdl2_base.dir/decode.c.o (symbol from plugin):(.text+0x0): first defined here

See
https://gcc.gnu.org/gcc-10/porting_to.html
https://wiki.gentoo.org/wiki/Gcc_10_porting_notes/fno_common

Information about cpdlc

Good afternoon. I'm doing a project to decode Inmarsat Aero messages. A number of messages I can not decode, but your module decpdlc can. It is possible to compile .dll for visual studio c # or advise the source of information to decode such messages, could not find it. Thank you in advance.

Share Grafana dashboard

Consider sharing the Grafana dashboard. It looks great, but I can't recreate it as the text image in the readme is too blurry. Grafana definition would be ideal, but a readable screenshot would work for me to create my own.

thanks!

JSON Output - Enhancement

Hi Tomasz,

Would it be possible to add an additional output to dumpvdl2 to send JSON via UDP to an IP address ?
vdlm2dec already has this function & its very useful as the output to Planeplotter is limited in what it supplies.

JSON output would include Date, Time, Channel, Frequency, Aircraft ICAO, Ground Station Id, Direction/Uplink/Downlink indicator, Latitude, Longitude, Altitude and Destination.

Regards
Nigel

No output at local ACARS frequency

Issuing the following command:

dumpvdl2 --rtlsdr 0 --gain 40 --correction 2 131550000

displays no output at all, although acarsdec works properly with the same parameters. If I turn on the debugging mode ("all"), this is the kind of message that I get from time to time:

process_samples(): 131550000: noise_floor: -46.0 dBFS
process_samples(): 131550000: noise_floor: -46.5 dBFS
got_sync(): Preamble found at 5433096 (pherr[2]=16.727892 pherr[1]=3.969243 pherr[0]=39.243427 vertex_x=-3.703129 syncbufidx=61, syncpoint=57 syncpoint_phase=-1.542303 sclk=4 v->dphi=-1.124854 ppm=-14.289424)
demod(): DM_SYNC, v->sclk=4
demod(): 5433105: I: 0.000071 Q: -0.002219 symb_pwr: 0.000005 frame_pwr: 0.000005 dphi: 1.436756 * pi/4 idx: 1 bits: 1
demod(): 5433115: I: 0.001198 Q: -0.002062 symb_pwr: 0.000006 frame_pwr: 0.000005 dphi: 2.061582 * pi/4 idx: 2 bits: 3
demod(): 5433125: I: -0.004359 Q: 0.000336 symb_pwr: 0.000019 frame_pwr: 0.000010 dphi: 6.663925 * pi/4 idx: 7 bits: 4
demod(): 5433135: I: 0.001438 Q: -0.000886 symb_pwr: 0.000003 frame_pwr: 0.000008 dphi: 4.826840 * pi/4 idx: 5 bits: 7
demod(): 5433145: I: -0.002716 Q: 0.001127 symb_pwr: 0.000009 frame_pwr: 0.000008 dphi: 5.634742 * pi/4 idx: 6 bits: 5
demod(): 5433155: I: -0.000899 Q: 0.000505 symb_pwr: 0.000001 frame_pwr: 0.000007 dphi: 1.281379 * pi/4 idx: 1 bits: 1
demod(): 5433165: I: -0.002303 Q: -0.002552 symb_pwr: 0.000012 frame_pwr: 0.000008 dphi: 3.149053 * pi/4 idx: 3 bits: 2
demod(): 5433175: I: -0.001743 Q: -0.002439 symb_pwr: 0.000009 frame_pwr: 0.000008 dphi: 1.576923 * pi/4 idx: 2 bits: 3
demod(): 5433185: I: -0.001738 Q: -0.000158 symb_pwr: 0.000003 frame_pwr: 0.000007 dphi: 0.337649 * pi/4 idx: 0 bits: 0
demod(): bitstream len=27 requested_bits=25, launching frame decoder
bitstream_descramble(): descrambled from 0 to 26
decode_header(): received: 0x3ac32e syndrome: 0x12 error: 0x804000, decoded: 0xba832e
decode_vdl2_burst(): Rejecting decoded header with non-zero reserved bits
process_samples(): 131550000: noise_floor: -46.6 dBFS
process_samples(): 131550000: noise_floor: -45.8 dBFS

Sometimes there's a line that looks like this:

decode_vdl2_burst(): v->datalen=111857 v->syndrome=17 - frame rejected

but still no output to viewport, file logs, nor UDP.

custom oversampling

For hardware devices:

Note: The device must support a sampling rate of 2100000 samples per second to work correctly with dumpvdl2. It is therefore not possible to use devices which only support predefined, fixed sampling rates (notably Airspies). This limitation will be removed in a future release of dumpvdl2.

For recorded IQ files:

The symbol rate for VDL2 is 10500 symbols/sec. dumpvdl2 internal processing rate is 10 samples per symbol. Therefore the file must be recorded with sampling rate set to an integer multiple of 105000. Specify the multiplier value with --oversample option. The default value is 10, which is valid for files sampled as 1050000 samples/sec.

Is there a technical limitation for not allowing --oversample for rtl_sdr, soapysdr, etc? I'm sharing a receiver between dumpvdl2 and rtlsdr-airband (http://github.com/rpatel3001/docker-rtlmuxer) and would like to up the sample rate to a higher multiple of 105000 to squeeze in a few more audio channels.

I might try it out myself but it might take some time to get around to it, asking here to see if there's a particular reason it won;t work.

macOS/Darwin does not implement pthread barriers

Not sure how high priority this is, but dumpvdl2 doesn't compile without modification since macOS Darwin doesn't implement POSIX thread barriers. There seem to be projects out there that have implemented this functionality easily themselves to avoid this compatibility issue, and thought it might be great if new users could just compile without problems.

This project for instance implemented pthread barriers to fix the compatibility issue:
pmwkaa/ioarena#10

Another example implementation:
https://github.com/ademakov/DarwinPthreadBarrier

References to pthread barriers:

dumpvdl2.h:273:8: error: unknown type name 'pthread_barrier_t';
extern pthread_barrier_t demods_ready, samples_ready;

dumpvdl2.c:41:1: error: unknown type name 'pthread_barrier_t';
pthread_barrier_t demods_ready, samples_ready;

dumpvdl2.c:74:13: warning: implicit declaration of function 'pthread_arrier_init' is invalid in C99 [-Wimplicit-function-declaration]
if( (ret = pthread_barrier_init(&demods_ready, NULL, ctx->num_channels+1)) !=0 ||

dumpvdl2.c:573:3: warning: implicit declaration of function 'pthread_barrier_wait' is invalid in C99 [-Wimplicit-function-declaration]
pthread_barrier_wait(&demods_ready);

Also for users trying to compile on macOS:

To fix reference to endian.h in avlc.h:22, change:

#include <endian.h>

to

#include <machine/endian.h>

Request: Full streaming data

Glad to read that full JSON data is being considered.... But, I would like to ask about including all the information that you currently have be included in the streamed data.

For example, this is what shows in the terminal when running;
[2019-02-03 20:43:41 UTC] [136.650] [-19.6/-34.8 dBFS] [15.2 dB] [2.7 ppm]
A2AB41 (Aircraft, Airborne) -> 11208A (Ground station): Response
AVLC: type: S (Receive Ready) P/F: 0 rseq: 4

But only this latter part is streamed when using the option --output-acars-pp.
A2AB41 (Aircraft, Airborne) -> 11208A (Ground station): Response
AVLC: type: S (Receive Ready) P/F: 0 rseq: 4

I would find the date, time, frequency and signal strength of huge value in my website.
Seems strange to me that it is clearly shown but not streamed.
Perhaps I have a setting wrong?

No decoded data.

vdlm2dec and acarsdec both work Ok (thousands of messages per day) so I am confident that the antenna and RTLSDRv3 dongle are working correctly.

Built on a Pi 3 with the lite OS version.
Did an apt-get update and apt-get upgrade before starting the build.
No other ADSB etc software installed. Just 1 USB SDR dongle installed.

I carefully followed the instructions on the github page:

1. git clone git://git.osmocom.org/rtl-sdr.git
2. cd rtl-sdr/
3. mkdir build
4. cd build
5. cmake ../
6. make
7. sudo make install
8. sudo ldconfig
9. sudo apt install build-essential cmake git libglib2.0-dev pkg-config
10. sudo echo "blacklist dvb_usb_rtl28xxu" >> /etc/modprobe.d/blacklist.conf`
11. Rebooted the Pi.
12. git clone https://github.com/szpajder/libacars
13. cd libacars
14. mkdir build
15. cd build
16. cmake -DMIRISDR=FALSE -DSDRPLAY=FALSE -DSOAPYSDR=FALSE -DSQLITE=FALSE -DETSY_STATSD=FALSE -DRAW_BINARY_FORMAT=FALSE -DZMQ=FALSE ../
17. make
18. sudo make install
19. dumpvdl2 --rtlsdr -0 --gain 42 --correction 7 136.725 136.775 136.800 136.975

dumpvdl2 2.1.0 (libacars 2.1.3)
Sampling rate set to 1050000 sps
Found 1 device(s):
0: Realtek, RTL2838UHIDIR, SN: 00000001

Using device 0: Generic RTL2832U OEM
Found Rafael Micro R820T tuner
Exact sample rate is: 1050000.026077 Hz
[R82XX] PLL not locked!
[R82XX] PLL not locked!
Center frequency set to 136 Hz
[R82XX] PLL not locked!
Device #0: gain set to 42.10 dB
Device 0 started
Allocating 15 zero-copy buffers

After running for 2+ hours, there is no output. With vdlm2dec I see output within 15-20 seconds during the day.
I prefer the decoded text output rather than JSON and have dumpvdl2 running Ok from 2-3 years ago, just the newer version is not showing any output.

Thanks for your help.

Macos 12.6 Monterey: issue (solved) with lib "lacars-2"; Planeplotter alternatives?

Hi, thank you for this software! Just wanted to say I managed to compile on an Macbook Air M1 with 12.6 after solving the same make install question reported for dumphfdl. I had already installed libacars on the same machine, and the compiler wasn't apparently able to find it. So I just set LDFLAGS and CPPFLAGS to the correct path. That worked and dumpvdl2 is running flawlessly (as far I can see) with a RTL-SDR dongle.
Interpreting the messages, however, is far from trivial for me. What could I do to at least visualize their airborne sources on a map, without Planeplotter?

"reserved" destination

I got an interesting message today:

[2019-03-06 13:58:53 UTC] [136.975] [-36.1/-50.7 dBFS] [14.7 dB] [0.5 ppm] A2B8DB (Aircraft, On ground) -> 5031EA (**reserved**): Command AVLC type: I sseq: 0 rseq: 0 poll: 0 ACARS (warning: CRC error): Reg: .N2748U Flight: UA0872 Mode: 2 Label: H1 Blk id: 1 Ack: ! Msg no.: D1 A Message: #DFBACM01ACM304BSN2748U UAL872 RCTPKSFO060319 9ER125130582883110UALACMAD^M 118 384084028722873-2993-299434999350005516T, 7911 7842 8#3 83314883^M 475511111^U3615540000000 48 394410-1343427 1057 57 1114^MJ%-000

The "reserved" string comes from an array in avlc.c but there are multiple index entries that resolve to "reserved" ... maybe you could use "reserved0", "reserved2", etc. so that when they come up again we can look further to see what the deal was ...?

Service file networking dependency

dumpvdl2/etc/dumpvdl2.service

Line 4 in fc653ae

Wants=network.target

dumpvdl2/etc/dumpvdl2.service

Line 5 in fc653ae

After=network.target

Documentation says:

network.target has very little meaning during start-up. It only indicates that the network management stack is up after it has been reached. Whether any network interfaces are already configured when it is reached is undefined.
network-online.target is a target that actively waits until the nework is "up", where the definition of "up" is defined by the network management software. Usually it indicates a configured, routable IP address of some kind. Its primary purpose is to actively delay activation of services until the network is set up.

Adding standard input support

This came up as I was trying to integrate dumpvdl2 support into OpenWebRX:

Will it be possible to add support for standard input, same as in dumphfdl? If yes, what would be the minimal input bandwidth required?

libzmq version checking error

After adding ZMQ support to dumpvdl2 I get the following error in my environment, incorrectly stating that libzmq 4.0.5 is older than 3.2.0:

pi@wx-rx:~/vdl2 $ dumpvdl2 --output decoded:json:zmq:mode=server,endpoint=tcp://*:5555
dumpvdl2 2.1.0 (libacars 2.1.3)
output_zmq: error: libzmq library version 4.0.5 is too old; at least 3.2.0 is required
Invalid output configuration

This is caused by problematic version checking code from line 51 of output_zmq.c, which causes the error during the minor check (0<2).

	if(major < LIBZMQ_VER_MAJOR_MIN || minor < LIBZMQ_VER_MINOR_MIN || patch < LIBZMQ_VER_PATCH_MIN) {
		fprintf(stderr, "output_zmq: error: libzmq library version %d.%d.%d is too old; at least %d.%d.%d is required\n",
				major, minor, patch,
				LIBZMQ_VER_MAJOR_MIN, LIBZMQ_VER_MINOR_MIN, LIBZMQ_VER_PATCH_MIN);
		goto fail;
	}

One possible way to fix this is:

if((major*1000000 + minor*1000 + patch) < (LIBZMQ_VER_MAJOR_MIN*1000000 + LIBZMQ_VER_MINOR_MIN*1000 + patch)) {

which passes the check successfully:

pi@wx-rx:~/vdl2 $ dumpvdl2 --output decoded:json:zmq:mode=server,endpoint=tcp://*:5555
dumpvdl2 2.1.0-dirty (libacars 2.1.3)
No input specified
Use --help for help

Build environment is Raspberry Pi 3 Model B Rev 1.2, Linux wx-rx 4.19.40-v7+ #1217 SMP Tue May 7 17:22:08 BST 2019 armv7l GNU/Linux

Thanks for the great work on dumpvdl2! Apart from this minor problem configuration for a remote monitoring station was very smooth.

duplicate in statsd.c

Hi, I have absolutely no idea what I'm doing but I did notice a duplicated line in statsd.c at lines 48-49 which both declare "decoder.errors.bitstream",

I don't know if this is intentional or if it's had any downstream effects, good or bad, at all, but thought you might want to know.

Ta

P.S. Happy new year

Channel Sense

VDL receivers in airframes are designed to channel sense in order to determine whether or not to use the SITA or ARINC ground stations.

Here's an excerpt from Standards and Recommended Practices (SARPs) for very high frequency (VHF) digital link (VDL) Mode 2:

2.2.4 Channel sense algorithms

2.2.4.1 When running a carrier sense multiple access (CSMA) algorithm prior to transmitting data or packetized voice, the VDL Mode 2 receiver can determine if the channel is idle by using an energy sensing algorithm. However, because the local noise floor is not a constant, an estimator is needed. This section provides an example of one possible estimator.

Note.— The MAC sub-layer declares the channel idle after the channel sense algorithm reports that the received signal power level has crossed below the busy threshold.

2.2.4.2 Channel quiescent value. Whenever the VDL Mode 2 is not transmitting or receiving a message,

Figure 2-2. Message encoding block diagram

the VDL calculates a channel quiescent value, Tb, according to the following algorithm:

Tq[n+1] = Tq[n] + k(L,Tq[n]) (L – Tq[n]), L ≤ Tmax
              Tmax                                   otherwise

where
L is the rms received signal level in hard μvolts calculated over the preceding 1 ms;
Tmax is a value in the range of 15 to 30 hard μvolts (exact calibration is not necessary);
Tq[n] is the estimate of the noise floor at time n;

and
k ( L,Tq ) = max ( 0.01(LITq), 1/16384 )

Note.— The non-linear function k() is designed to quickly adjust to a reduced noise level and to slowly adjust to an increased level.

2.2.4.3 Channel busy threshold. The channel busy threshold, Tb, is defined as 1.4 Tq. The channel is declared idle until the rms received signal value, L, exceeds Tb. Then the channel is declared busy and the channel sense algorithm is suspended while synchronization is attempted.

2.2.4.4 Synchronization. If synchronization is not achieved (i.e. the unique word is not detected) within 2.5 milliseconds, then a new value is taken for the rms received signal level, L, and processed to yield the channel quiescent value, Tq, and the channel sense determination.

2.2.4.5 Receiver/transmitter interactions. 

I am not very familiar with the deep internals to the decoder, but is there any value to incorporate such channel sensing across the body of known VDL frequencies in order to detect channel use more effectively?

Airspy Usage

Just occurred to me after reading your front page that there’s probably no way to currently use an Airspy R2? I’m thinking it’s the sample rate warning you mention. I don’t suppose this has been changed in the current release?

Output to network may fail on startup

Hello!

I dump to network using this parameter:

--output decoded:text:udp:address=myhost.net,port=1111

On startup of my Orange Pi dumpvdl seems to start before Pi is connected to WiFi. Because of this it starts dumpvdl2 normally but there is zero networking traffic. I have to fix it manually by restarting the service.

Possible solution is to change lines at

https://github.com/szpajder/dumpvdl2/blob/f671d9ecf8d4fc60190c097daaad5df9404622b2/etc/dumpvdl2.service

from

Wants=network.target
After=network.target

to

Wants=network-online.target
After=network-online.target

Difference is described in detail here: https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/