777arc/pysdr

PySDR.org textbook source material, feel free to post issues/PRs

PySDR Textbook Source Material

This repo contains the source content used to generate the textbook PySDR: A Guide to SDR and DSP using Python hosted at https://pysdr.org.

Feel free to submit an issue, or even a Pull Request (PR) with fixes or improvements. Those who submit valuable feedback/fixes be permanently added to the acknowledgments section. Not good at Git but have changes to suggest? Feel free to email Marc at [email protected].

Building

Note that the website is now automatically built and deployed with each push/merge into master branch, using the GitHub action build-and-deploy.yml and the GitHub pages system for hosting the actual textbook.

For testing changes to the textbook locally, you can build using the following steps:

Ubuntu/Debian

sudo apt update
sudo apt install -y texlive-latex-extra pdf2svg
pip install -r requirements.txt
make html
make html-fr
make html-nl
make html-ukraine
make html-zh

In _build there should be an index.html that represents the main page of the English site

Note: on one machine I had to add ~/.local/bin to PATH

Windows

Install pre-requisite software with:

1. Install Tex Live from the install-tl-windows.exe link towards the beginning of https://www.tug.org/texlive/acquire-netinstall.html (after opening it, choose the Install option). When a box pops up, click Advanced. It will mention Installation root C:/texlive/2023 which we need later. Uncheck Install TeXworks front end because we wont need it. Under Scheme click "change" and choose basic scheme. Hit Install. It will take a while because it installs a lot of common latex packages, you can do the remaining steps while this installs. You know it worked if you now have a C:\texlive\2023\bin\windows\latex.exe.
2. From the Microsoft Store install Python 3.10 (3.8-3.12 is fine too if you already have it installed).
3. In a PowerShell terminal (click start menu then type powershell, or open a terminal in VSCode) run pip install sphinx sphinxcontrib-tikz patreon
4. cd to the directory you cloned PySDR
5. Separately, open "Tex live command-line" app from start menu, then run tlmgr install dvisvgm pgf (pgf is tikz).

Build the English version only using:

python -m sphinx.cmd.build -b html -D imgmath_latex="C:\texlive\2023\bin\windows\latex.exe" -D imgmath_dvisvgm="C:\texlive\2023\bin\windows\dvisvgm.exe" . _build

The first time running this it might take a while because it has to download LaTeX packages.

Creating a PDF Export

Not fully working yet due to animated gifs, they all need to be removed for this to not error out:

sudo apt-get install -y latexmk
sphinx-build -b latex . _build/latex
make latexpdf

Misc

Ideas for future chapters:

• Equalization, would be the last step needed to finish the end-to-end comms link
• Cyclostationary analysis
• OFDM, simulating OFDM and CP, show via Python how it turns freq selective fading into flat fading
• How to create real-time SDR apps with GUIs in Python using pyqt and pyqtgraph, or even just matplotlib with updating
• Python code that lets the Pluto (or RTL-SDR) act as an FM receiver, like with sound output
• End to end example that shows how to detect start of packet and other concepts not covered in RDS chapter

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 Unported License.

pysdr's Issues

USRP `sc16`, `sc8` CPU format in Python API

First, thanks for the great book! It helps me a lot to get started in SDR.

In the USRP chapter, you said that the UHD only supports fc64 and fc32 CPU data types when using the Python API. In my experiment, I found that all CPU data types are supported in python api, though you need to do some extra job when using the sc16 or sc8 as CPU format.

In short, the following codes can be use to rx and tx data with sc16 CPU data type in python.

Receiving

import uhd
import numpy as np

usrp = uhd.usrp.MultiUSRP()

center_freq = 100e6  # Hz
sample_rate = 1e6  # Hz
gain = 50  # dB

usrp.set_rx_rate(sample_rate, 0)
usrp.set_rx_freq(uhd.libpyuhd.types.tune_request(center_freq), 0)
usrp.set_rx_gain(gain, 0)

# Set up the stream and receive buffer
st_args = uhd.usrp.StreamArgs("sc16", "sc16")
st_args.channels = [0]
metadata = uhd.types.RXMetadata()
streamer = usrp.get_rx_stream(st_args)
recv_buffer = np.zeros((1, 1000), dtype=np.int32)  # use int32 to hold the iq sample

# Start Stream
stream_cmd = uhd.types.StreamCMD(uhd.types.StreamMode.start_cont)
stream_cmd.stream_now = True
streamer.issue_stream_cmd(stream_cmd)

# Receive Samples
streamer.recv(recv_buffer, metadata)

# Stop Stream
stream_cmd = uhd.types.StreamCMD(uhd.types.StreamMode.stop_cont)
streamer.issue_stream_cmd(stream_cmd)

samples = np.zeros(recv_buffer.shape[-1], dtype=np.complex64)
recv_buffer = recv_buffer.reshape(-1).view(np.int16)  # view the buffer as i16,q16 array
samples.real = recv_buffer[::2]
samples.imag = recv_buffer[1::2]

print(len(samples))
print(samples[0:10])

When using sc8 as CPU format, the recv_buffer needs to be initialized as np.int16, and be viewed as np.int8 after receiving samples.

Transmitting

import uhd
import numpy as np

# Configuration Parameters
tx_freq = 915e6  # Hz
tx_gain = 25  # Transmit gain in dB
tx_rate = 1e6  # Sample rate in samples per second

usrp = uhd.usrp.MultiUSRP()

st_args = uhd.usrp.StreamArgs("sc16", "sc16")
st_args.channels = [0]

# Set up TX streamer
tx_streamer = usrp.get_tx_stream(st_args)

# Configure USRP parameters
usrp.set_tx_rate(tx_rate)
usrp.set_tx_freq(uhd.libpyuhd.types.tune_request(tx_freq), 0)
usrp.set_tx_gain(tx_gain, 0)

# create random signal
samples = 0.1 * np.random.randn(10000) + 0.1j * np.random.randn(10000)

# convert the float complex samples to i16,q16 buffer
data = np.zeros(len(samples) * 2, dtype=np.int16)  
data[::2] = np.real(samples) * 2 ** 15
data[1::2] = np.imag(samples) * 2 ** 15
data = data.view(np.int32)

# Create a metadata object for sending samples
md = uhd.types.TXMetadata()
md.start_of_burst = True
md.end_of_burst = False

# Stream the signal
tx_streamer.send(data, md)

# Indicate end of burst
md.start_of_burst = False
md.end_of_burst = True
# Send a few zeros to flush the buffer
tx_streamer.send(np.zeros((1000,), dtype=np.int32), md)
print("Transmission completed.")

When using sc8 as CPU format, the data needs to be initialized as np.int8, be set via np.real/imag(samples) * 2 ** 7, and be viewed as np.int16 before sending.

Why the code works?

Here is the underlying code of tx_streamer.send(nparr, md) in the UHD library. It shows that the UHD library only get the underlying bytes of the input numpy array and pass it to the cpp api, which means (I guess) we can use any numpy array as the input of the tx_streamer.send as long as the item size of the numpy array is same as the CPU format. So when using sc16 as CPU format, the input numpy array can be np.int32 or np.uint32, and when using sc8 as CPU format, the input numpy array can be np.int16 or np.uint16.

The C version of tx example also shows that the UHD library only cares about the underlying bytes of the input buffer.

Full E2E Simulation for Sync Chapter

At the end of the chapter add a full python simulation showing reading some input data (eg an ASCII message), generate QPSK, simulate a channel, receive/demod/decode the ASCII, and compute BER

Issues from old repo

Before transitioning to new repo there were a few good issues - https://github.com/777arc/textbook/issues

Discrepancy Between Impulse Response Equation and Python Code Implementation

The equation that defines the impulse response of the raised-cosine filter is available at:
https://github.com/777arc/PySDR/blob/master/content/pulse_shaping.rst?plain=1#L104

However, the Python code implementation, found at:
https://github.com/777arc/PySDR/blob/master/content/pulse_shaping.rst?plain=1#L197
appears to be missing the term 1/Ts.

After conducting a search on the internet, I concluded that the equation is correct. However, upon further examination, I realized that the Python code is also correct. This is because when the term 1/Ts is added, the result is approximately 0.2 instead of approximately 1.

Could you please add an explanation in this section to describe why there is a difference between the equation and the Python code?"

@accessbox777

Clarify first image in Carrier and Downconversion

clarify what an unmodulated carrier looks like, and remove the cos and arrow.

Section or chapter on NOAA HRPT

eg adapting https://github.com/radio-satellites/python-hrpt and it would use a recording of HRPT off IQEngine

Synchronization of TX and RX Signals in BladeRF 2.0 xA9

I want to implement a FMCW or a pulsed radar using a SDR I tried using Pluto sdr but that does not have any synchronization mechanisms, I do have a BladeRF in my laboratory and I saw that in the API to control the bladeRF there is a bladerf_sync_config(), bladerf_sync_rx(), bladerf_sync_tx() functions. So, I wanted to know that if I use these functions in my code then will the TX and the RX signals be tine synchronized.

End-to-End example doesn't seem to work properly

I've tried to use examples from Textbook to process my own signal, after I tried full code example and provided file, I'm still getting wrong results as I can assume from constellation diagrams.
Any suggestions where I go wrong way or misunderstood something?

Code I've used:

import matplotlib.pyplot as plt

import numpy as np
from scipy.signal import resample_poly, firwin


def main():
    x = np.fromfile('waves/fm_rds_250k_1Msamples.iq', dtype=np.complex64)
    sample_rate = 250e3
    center_freq = 99.5e6
    plt.scatter(x.real, x.imag, marker='.', label="Original")

    x = 0.5 * np.angle(x[0:-1] * np.conj(x[1:]))

    N = len(x)
    f_o = -57e3
    t = np.arange(N) / sample_rate
    x = x * np.exp(2j * np.pi * f_o * t)

    taps = firwin(numtaps=101, cutoff=7.5e3, fs=sample_rate)
    x = np.convolve(x, taps, 'valid')
    plt.scatter(x.real, x.imag, marker='.', label="After filtering")

    x = x[::10]
    sample_rate = 25e3

    x = resample_poly(x, 19, 25)
    sample_rate = 19e3

    samples = x
    print(len(x))
    samples_interpolated = resample_poly(samples, 32, 1)
    sps = 16
    mu = 0.01
    out = np.zeros(len(samples) + 10, dtype=np.complex64)
    out_rail = np.zeros(len(samples) + 10, dtype=np.complex64)
    i_in = 0
    i_out = 2
    while i_out < len(samples) and i_in + 32 < len(samples):
        out[i_out] = samples_interpolated[i_in * 32 + int(mu * 32)]
        out_rail[i_out] = int(np.real(out[i_out]) > 0) + 1j * int(np.imag(out[i_out]) > 0)
        x = (out_rail[i_out] - out_rail[i_out - 2]) * np.conj(out[i_out - 1])
        y = (out[i_out] - out[i_out - 2]) * np.conj(out_rail[i_out - 1])
        mm_val = np.real(y - x)
        mu += sps + 0.01 * mm_val
        i_in += int(np.floor(mu))
        mu = mu - np.floor(mu)
        i_out += 1
    x = out[2:i_out]

    print(len(x))
    sample_rate /= 16

    samples = x
    N = len(samples)
    plt.scatter(x.real, x.imag, marker='.', label="After time sync")
    phase = 0
    freq = 0

    alpha = 8.0
    beta = 0.002
    out = np.zeros(N, dtype=np.complex64)
    freq_log = []
    for i in range(N):
        out[i] = samples[i] * np.exp(-1j * phase)
        error = np.real(out[i]) * np.imag(out[i])

        freq += (beta * error)
        freq_log.append(freq * sample_rate / (2 * np.pi))
        phase += freq + (alpha * error)

        while phase >= 2 * np.pi:
            phase -= 2 * np.pi
        while phase < 0:
            phase += 2 * np.pi
    x = out
    plt.scatter(x.real, x.imag, marker='.', label="After frequency sync")
    plt.legend(loc="upper right")
    plt.show()

support forum for students?

Hi,

More a question then a "issue".

Is there a student forum for pysdr?

I am learning about SDR using the pysdr website as a reference.

It would be interesting if there would be a forum so students can ask questions and talk about SDR, without the need to contact the author directly, or use github (which I think is not surely not the place for that).

I propose a room on matrix as that is an open-source / open-standard conference system.

Thx :-)

C4FM Modulation in Python

Hello , i'm trying to implement C4FM modulation ( input symbols -> raised cosine filter -> inverse sinc filter-> FM Modulator). What should i write in the code in python to get the desired output c4fm modulated waveform?