jingmengzhiyue/hyperevalsr

An open source python package for super-resolution/recovery quality evaluation of hyperspectral images, including RMSE, ERGAS, SSIM, RSNR, PSNR, CC, DD, and SAM.

---

HyperEvalSR

HyperEvalSR is an open-source Python library designed for reading hyperspectral images, assessing the quality of various indices used in unmixing, denoising, and super-resolution tasks, and providing algorithms for fusing hyperspectral and multispectral images. In the future, it will also offer algorithms for unmixing, material identification, classification, segmentation, denoising, change detection, and target detection in remote sensing (hyperspectral) images.

Installation

You can install the HyperEvalSR package directly from PyPI:

pip install HyperEvalSR

Alternatively, if you wish to access the latest features, you can clone this repository and install it manually:

git clone https://github.com/jingmengzhiyue/HyperEvalSR.git
python setup.py install

Data

Data Loading

The data loading module supports direct reading of TIFF and MAT files. Support for other file formats will be added gradually.

from HyperEvalSR import data
img = data.load(file_path)

file_path (str): The path to the image file. Supported file extensions: .tiff and .mat.

Image Display

You can display a hyperspectral image using the show function:

from HyperEvalSR import data
data.show(HSI, band_set=None, show=True, save=False, path=None)

HSI (ndarray): Hyperspectral image to display.

band_set (list or None, optional): List of 3 band indices to compose the pseudo-color image. Defaults to None.

show (bool, optional): Whether to display the image immediately. Defaults to True.

save (bool, optional): Whether to save the image. Defaults to False.

path (str, optional): Path to save the image if save is True.

Fusion of HSI and MSI

The sr module supports the fusion of high spatial resolution multispectral images and high spectral resolution hyperspectral images to reconstruct images with both high spatial and spectral resolutions simultaneously. Currently, it supports the CNMF algorithm based on coupled non-negative matrix factorization.

CNMF Algorithm

The CNMF (Coupled Non-negative Matrix Factorization) algorithm is used for fusing hyperspectral and multispectral data. You can use it as follows:

from HyperEvalSR import algorithms as algo
out = algo.CNMF(MSI, HSI, mask=0, verbose='off', MEMs=0)

MSI (numpy.ndarray): Multispectral (MS) image data, shape (rows1, cols1, bands1).

HSI (numpy.ndarray): Low-spatial-resolution hyperspectral (HS) image data, shape (rows2, cols2, bands2).

mask (int or numpy.ndarray, optional): Binary mask for processing (rows2, cols2) (0: mask, 1: image). Defaults to 0.

verbose (str, optional): Verbosity mode ('on' or 'off'). Defaults to 'off'.

MEMs (int or numpy.ndarray, optional): Manually defined endmembers (bands2, num_endmembers). Defaults to 0.

References:

[1] N. Yokoya, T. Yairi, and A. Iwasaki, "Coupled nonnegative matrix factorization unmixing for hyperspectral and multispectral data fusion," IEEE Trans. Geosci. Remote Sens., vol. 50, no. 2, pp. 528-537, 2012.
[2] N. Yokoya, N. Mayumi, and A. Iwasaki, "Cross-calibration for data fusion of EO-1/Hyperion and Terra/ASTER," IEEE J. Sel. Topics Appl. Earth Observ.Remote Sens., vol. 6, no. 2, pp. 419-426, 2013.
[3] N. Yokoya, T. Yairi, and A. Iwasaki, "Hyperspectral, multispectral, and panchromatic data fusion based on non-negative matrix factorization," Proc. WHISPERS, Lisbon, Portugal, Jun. 6-9, 2011.

Quality Assessment Metrics

The metrics module provides various quality assessment metrics for evaluating the performance of unmixing, denoising, and super-resolution algorithms.

from HyperEvalSR import metrics

• Peak Signal to Noise Ratio (PSNR): Measures the ratio between the maximum possible power of a signal and the power of corrupting noise.

  metrics.PSNR(ref_img, rec_img)

  ref_img (numpy.ndarray): The reference image.

  rec_img (numpy.ndarray): The reconstructed image.

• Reconstruction Signal-to-Noise Ratio (RSNR): Evaluates the signal-to-noise ratio of the reconstructed image.

  metrics.RSNR(ref_img, rec_img, mask=None)

  ref_img (numpy.ndarray): The reference image.

  rec_img (numpy.ndarray): The reconstructed image.

  mask (numpy.ndarray, optional): A mask to apply to the images. Defaults to None.

• Degree of Distortion (DD): Represents the level of distortion in the image.

  metrics.DD(ref_img, rec_img)

• Spectral Angle Mapper (SAM): Measures the spectral similarity between two images using the angle between their spectral vectors.

  metrics.SAM(ref_img, rec_img)

• Root Mean Squared Error (RMSE): Computes the square root of the average squared differences between the reference and reconstructed images.

  metrics.RMSE(ref_img, rec_img)

• Erreur Relative Globale Adimensionnelle de Synthèse (ERGAS): Calculates the relative global dimensionless synthesis error.

  metrics.ERGAS(ref_img, rec_img, downsampling_scale)

  downsampling_scale (int): The downsampling scale factor.

• Structural Similarity Index (SSIM): Assesses the structural similarity between the reference and reconstructed images.

  metrics.SSIM(ref_img, rec_img, k1=0.01, k2=0.03, L=255)

  k1 (float, optional): Constant for stability. Defaults to 0.01.

  k2 (float, optional): Constant for stability. Defaults to 0.03.

  L (int, optional): Dynamic range of the images. Defaults to 255.

• Cross-Correlation (CC): Measures the similarity between two images using the correlation coefficient between their pixels.

  metrics.CC(ref_img, rec_img, mask=None)

  mask (numpy.ndarray, optional): A mask to apply to the images. Defaults to None.

• Universal Image Quality Index (UIQI): Calculates the Universal Image Quality Index (UIQI) between two images.

  metrics.UIQI(ref_img, rec_img)

Mathematical Formulations

This section provides the mathematical formulations for the quality assessment metrics implemented in the metrics module.

Peak Signal-to-Noise Ratio (PSNR)

The PSNR measures the ratio between the maximum possible power of a signal and the power of corrupting noise. It is expressed in decibels (dB), and a higher value indicates a higher similarity between the reconstructed and original images.

$$ \mathrm{PSNR} = 10 \log_{10} \left( \frac{\max^2\left( \widehat{\mathbf{X}} \right)}{\mathrm{MSE}\left( \mathbf{X}, \widehat{\mathbf{X}} \right)} \right) $$

where $\mathrm{MSE}$ represents the mean squared error, calculated as:

$$ \mathrm{MSE} = \frac{1}{N_w N_h} | \widehat{\mathbf{X}} - \mathbf{X} |_{F}^{2} $$

In these formulas, $N_w$ and $N_h$ represent the width and height of the image, respectively, and $\mathbf{X}$ and $\widehat{\mathbf{X}}$ denote the reference and reconstructed images, respectively.

Root Mean Squared Error (RMSE)

The RMSE is a commonly used indicator to describe the degree of difference between the reconstructed image and the reference image. Smaller errors result in smaller RMSE values. When the reconstructed image and the reference image are exactly the same, the RMSE equals 0.

$$ \mathrm{RMSE} = \sqrt{\mathrm{MSE}} $$

Reconstruction Signal-to-Noise Ratio (RSNR)

The RSNR is commonly used to measure the spatial quality of the reconstructed image. Higher RSNR values indicate smaller differences between the reconstructed and original images, and thus better image quality.

$$ \mathrm{RSNR} = 10 \log_{10} \left( \frac{| \mathbf{X} |{F}^{2}}{| \widehat{\mathbf{X}} - \mathbf{X} |{F}^{2}} \right) $$

Degree of Distortion (DD)

The Degree of Distortion (DD) is an indicator used to describe the degree of signal distortion, typically used to evaluate the distortion during signal transmission or storage. Smaller distortions result in smaller DD values, with the optimal value being 0.

$$ \mathrm{DD} = \frac{1}{N_w N_h} | \mathrm{vec}\left( \widehat{\mathbf{X}} \right) - \mathrm{vec}\left( \mathbf{X} \right) |_{1}^{2} $$

Spectral Angle Mapper (SAM)

The Spectral Angle Mapper (SAM) compares the similarity between the reconstructed and reference images by measuring the spectral angle of each pixel. The higher the similarity, the smaller the SAM value.

$$ \mathrm{SAM} = \frac{1}{M} \sum_{n=1}^{M} \arccos \left( \frac{(\widehat{\mathbf{x}}[n])^{\mathrm{T}} \mathbf{x}[n]}{| \widehat{\mathbf{x}}[n] |{2} \cdot | \mathbf{x}[n] |{2}} \right) $$

In this formula, $\mathbf{x}[n]$ represents the $n$-th column of $\mathbf{X}$, and $M$ represents the number of spectral bands.

Erreur Relative Globale Adimensionnelle de Synthèse (ERGAS)

ERGAS is a relative error indicator that can be used to compare the quality of reconstructed remote sensing images with different resolutions and sizes, as well as to evaluate image quality at different compression ratios. Smaller ERGAS values indicate higher spatial and spectral similarity between the reconstructed and reference images.

$$ \mathrm{ERGAS} = \frac{100}{r} \sqrt{\frac{1}{M} \sum_{m=1}^{M} \frac{\mathrm{RMSE}{m}^{2}}{\mu{\mathbf{X}^{(m)}}^{2}}} $$

In this formula, $r$ represents the spatial down-sampling ratio, $\mu_{\mathbf{X}^{(m)}}$ represents the mean of the $m$-th row of $\mathbf{X}$, and $\mathrm{RMSE}_m$ represents the RMSE value of the $m$-th spectral band.

Structural Similarity Index (SSIM)

The Structural Similarity Index (SSIM) is an indicator used to evaluate the similarity between two images and to quantitatively assess the degree of image distortion. The SSIM value ranges between $[-1, 1]$, with larger values indicating greater similarity.

$$ \mathrm{SSIM} = \left[ l\left( \widehat{\mathbf{X}}, \mathbf{X} \right) \right]^{\alpha} \left[ c\left( \widehat{\mathbf{X}}, \mathbf{X} \right) \right]^{\beta} \left[ s\left( \widehat{\mathbf{X}}, \mathbf{X} \right) \right]^{\gamma} $$

In this formula, $l\left( \widehat{\mathbf{X}}, \mathbf{X} \right)$, $c\left( \widehat{\mathbf{X}}, \mathbf{X} \right)$, and $s\left( \widehat{\mathbf{X}}, \mathbf{X} \right)$ represent the luminance, contrast, and structure similarity, respectively, and are defined as:

$$ l\left( \widehat{\mathbf{X}}, \mathbf{X} \right) = \frac{2 \mu_{\widehat{\mathbf{X}}} \mu_{\mathbf{X}} + c_1}{\mu_{\widehat{\mathbf{X}}}^{2} + \mu_{\mathbf{X}}^{2} + c_1} $$

$$ c\left( \widehat{\mathbf{X}}, \mathbf{X} \right) = \frac{2 \sigma_{\widehat{\mathbf{X}} \mathbf{X}} + c_2}{\sigma_{\widehat{\mathbf{X}}}^{2} + \sigma_{\mathbf{X}}^{2} + c_2} $$

$$ s\left( \widehat{\mathbf{X}}, \mathbf{X} \right) = \frac{\sigma_{\widehat{\mathbf{X}} \mathbf{X}} + c_3}{\sigma_{\widehat{\mathbf{X}}} + \sigma_{\mathbf{X}} + c_3} $$

In these formulas, $\mu$ represents the mean of the image, $\sigma$ represents the standard deviation of the image, and $\sigma_{\widehat{\mathbf{X}} \mathbf{X}}$ represents the covariance between the two images. The constants $c_1$, $c_2$, and $c_3$ are typically set to $1$.