nisar-solid/atbd

Notebooks for NISAR Solid Earth Algorithm Theoretical Basis Document

atbd's Issues

Variogram

Statistical functions for comparison

• Deramp --> Raider deramp
• Calculate experimental variogram --> RAiDER variogram
• Calculate binned variogram with functional fitting --> RAiDER binning and RAiDER fitting
• Calculate double-difference --> RAiDER pairs

@cmarshak made this repository with @dbekaert to break out these functional components. There are few more things I need to add, but hopefully this is a clear starting point. Thanks!

func fitting to LOS GNSS and InSAR time-series

Function(s) related to the time function(s) estimation of the LOS GNSS and InSAR time-series, in the post-processing. This includes the following functions:

• translate the model (time func) setup to design matrices
• estimate the time function parameters given the time-series data, optimization param and model setup

1 - get_design_matrix4time_func()

Translate the input time function setup into the design matrix.

Inputs

• date_list - list of str or datetime objects
• polynomial - int, polynomial order, e.g. 1 (linear), 2 (quadratic), 3 (cubic), etc.
• periodic - list of float, period(s) in years, e.g. 1.0 (annual), 0.5 (semi-annual), etc.
• step - list of str or datetime objects, dates of the step functions
• exp - dict with key for onset time in YYYYMMDD and values for char times in days
• log - dict with key for onset time in YYYYMMDD and values for char times in days

Outputs

• design_matrix - 2D np.ndarray in size of (num_date, num_param)

Status

• mintpy.utils.time_func.get_design_matrix4time_func() supports polynomial, periodic, step
• the same function above now supports multiple exp / log functions [update at Jun 15, 2021]

2 - estimate_time_func()

For a 1D displacement time-series, estimate the time function parameters given the time-series data, time func setup and optimization setup.

Inputs

• date_list - list of str or datetime object, list of time info
• dis_ts - 2D np.ndarray in float32, displacement in meters in size of (num_date, num_site) where num_site can be 1 for one GNSS or one InSAR pixel.
• time_func - dict of time function setup.
• solver - str, optimization / solver names, e.g. l1norm, l2norm, etc.

Outputs

• G - 2D np.ndarray, design matrix in size of (num_date, num_param)
• m - 2D np.ndarray, estimated parameters in size of (num_param, num_site)
• kwargs, e.g.:
  • e2 - sum of residual square (estimation quality parameters)
  • residual time-series [can be calculated from G and m with input dis_ts]
  • covariance matrix of time func parameters via:
    • a) fitting residual assuming normal distribution in time (link) OR
    • b) linear propagation from covariance matrix of time-series (link).

Status

• mintpy.utils.time_func.estimate_time_func() supports l2norm.
• mintpy/timeseries2velocity.py --ts-std-file now supports linear propagation of covariance (STD) from time-series to time func parameters [update at Jun 22,2021]

Pull Request for permafrost validation notebook and utilities

The remote sensing group at University of Alaska Fairbanks is submitting permafrost NISAR Cal/Val functionality this repository. Please see the corresponding Pull Request from myself.

The primary notebook is "methods/permafrost/Permafrost_Requirement_Validation.ipynb" and it follows as much of the format as the other Solid Earth validation notebooks when possible. Functions for the time series spline fitting algorithm used in this validation approach is included in "solid_utils/permafrost_utils.py"

As it is now, we use Hyp3 to produce S1 interferograms rather than ARIA as in the other Solid Earth approaches. A very short notebook is also included for users to produce and download the interferogram stack(s) used in the notebook.

Additionally, we are providing a set of field data surface displacement observations, as measured by GNSS and surveying. That data is in the process of being added to the cal/val database. However, until it is added, the summer 2023 data is included in a small .zip file in this pull request which contains 8 short .csv files.

fetch GNSS time-series

Function(s) to fetch the GNSS time-series with documentation on the correction levels. This should include two functions:

• search the available GNSS sites
• for each site, download and read the time-series data

1 - search_gnss()

Search the list of available GNSS sites given a certain set of criteria.

Inputs

• SNWE - list of 4 floats in (south, north, west, east) in degrees
• start_datetime - datetime object
• end_datetime - datetime object
• solution_provider - str, GNSS solution provider, e.g. UNR, JPL, SOPAC
• kwargs - dict, usability metric, e.g. ?
  • min_num_observation
  • fraction

Outputs

• site_names - list of str, the list of names/IDs of available GNSS sites
• site_lats - list of float, the list of latitudes of available GNSS sites
• site_lons - list of float, the list of longitudes of available GNSS sites

Status

• mintpy.objects.gps.search_gps() supports the UNR solution.

2 - get_gnss_displacement()

For each GNSS site, download the data file and read them. The output should be one common format as defined below, regardless of the solution providers.

Inputs

• site_name - str, GNSS site name/ID
• start_datetime - datetime object
• end_datetime - datetime object
• solution_provider - str, GNSS solution provider, e.g. UNR, JPL, SOPAC

Outputs

• site_lat/lon - float, latitude and longitude of the GNSS site in degree
• site_hgt - float, height of the GNSS site in meter
• E/N/U time-series - 1D np.ndarray in float32, displacement time-series in meters
• E/N/U time-series uncertainty (STD) - 1D np.ndarray in float32, uncertainty of displacement time-series in meters
• reference_frame - str
• correction_level - str or list of str [require more detailed definitions/descriptions]
• step_time_coseismic - list of datetime objects, times with co-seismic jumps
• step_time_antenna - list of datetime objects, times with antenna changes/maintainances.

Status

• mintpy.objects.gps.GPS() supports UNR solution
• GeoGateway.getDisplacement supports JPL solution

Add the latest version of ATBD doc

It would be helpful to keep the latest version of the solid Earth ATBD documents in PDF here in the repo, similar to what the cryo team has (https://github.com/fastice/nisarCryoCalValNotebooks/tree/main/ATBD-Documents).

Turn the ATBD repo into a JupyterBook hosted on GitHub Pages

This repository (https://github.com/CIMR-Algos/TerrestrialSnowArea_ATBD) shows it's possible to combine a bunch of markdown files and Jupyter notebook files together into a Jupyter book (https://cimr-algos.github.io/SeaIceThickness_ATBD/intro.html), an HTML website hosted in GitHub Pages and compiled from the repo automatically. It will be really cool if we could do the same here.

Mark this repository as deprecated

It seems like we should mark this repository as deprecated and point to the GitLab version that is the new main version, so people don't get confused.

list of functional components

This issue lists the consolidated functional components needed for the ATBD. For each functionality, give a brief description, put a link to an existing code, otherwise create an independent issue with a detailed description of the functionality and its input/output parameters.

GNSS related

• Fetching time-series data and document correction levels --> #3

Generic time series analysis for GNSS and InSAR

• Functional fitting to LOS GNSS and InSAR time-series --> #4 [Done]
• Functional fitting to 3D GNSS time-series --> optional secondary priority

GNSS and InSAR comparison

• Project 2D/3D GNSS to InSAR LOS direction:
• Retrieve InSAR observations at GNSS locations:

Statistical functions for comparison

• Deramp --> Raider deramp
• Calculate experimental variogram --> RAiDER variogram
• Calculate binned variogram with functional fitting --> RAiDER binning and RAiDER fitting
• Calculate double-difference --> RAiDER pairs

InSAR related

• SBAS --> mintpy.ifgram_inversion.py
• plotting
  • interferogram coherence as a function of Bperp
  • interferogram coherence as a function of Btemp
  • interferogram pairs in Btemp-Bperp plane --> mintpy.plot_network.py

UAVSAR related