fiduceo/fcdr_avhrr

FCDR AVHRR python code

• Copyright (C) 2017 J.Mittaz University of Reading
• This code was developed for the EC project: Fidelity and Uncertainty in
• Climate Data Records from Earth Observations (FIDUCEO).
• Grant Agreement: 638822
• Reviewed and approved by <name, instituton>,
• This program is free software; you can redistribute it and/or modify it
• under the terms of the GNU General Public License as published by the Free
• Software Foundation; either version 3 of the License, or (at your option)
• any later version.
• This program is distributed in the hope that it will be useful, but WITHOUT
• ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
• FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
• more details.
• A copy of the GNU General Public License should have been supplied along
• with this program; if not, see http://www.gnu.org/licenses/

FCDR_AVHRR

Development code for AVHRR FCDR Uncertainties

Written by J.Mittaz, University of Reading 31/08/2017 Revised by M.Taylor, University of Reading 10/11/2017

Code here will create the pre-beta FIDUCEO FCDR. The data will be equator-to-equator data with all fields defined as in the specification. Note that the Fortran code is dependent on having the UoR GBCS (Generalised Bayesian Cloud Screening) installed from the ESA CCI SST project.

The files in the directory are

equator-to-equator.py : top level script to run the process. Accepts instrument name and year month day and will create all FCDR files for that day.

write_easy_fcdr_from_netcdf.py : python code to convert temporary file created by write_fcdf.exe into FIDUCEO netcdf format using Tom Block's writer available from FIDUCEO/FCDRTools/.

Makefile: Makefile set up to make .exe file on CEMS. The Makefile assumes that the GBCS is installed as a GBCS directory in the source directory.

write_fcdr.f90 : Top level program to make FCDR data from an input list of AVHRR Level 1B files.

combine_orbits.f90 : Code to merge AVHRR L1B files rogether into a continuous stream replacing bad scan lines from one with good scan lines from another file. Then cuts out the equator-to-equator data and recalibrates it.

fiduceo_uncertainties.f90 : Code to create uncertainties for easy FCDR. Also specifies sensor specific channel correlation matrices and spatial correlation scale. Writes temporary output netcdf file which is then converted using write_easy_fcdr_from_netcdf.py within the code.

quickplot.py : Code to automate quick time series plot of reflectance and IR channel data (chx.png) as well as the independent uncertainties (chx_independent.png) and structured uncertainties (chx_structured.png).

fcdr_avhrr's Issues

Loss of a significant number of processed orbit files

The latest build is generating a number of empty orbit files and/or not processing all available input data.

It is suspected that this is possibly caused by an error in the logic applied to select orbit files overlapping or contained with each between-ECT window processed on a particular day.

Fix NaN fraction calculation in robust statistics code calc_stats.py

The code used to generate global (sensor series level) robust statistics is incorrectly returning:

1. the calculated fraction of NaN values (it is presently returning the value 0.5)
2. the minimum value of the median (it is presently returning the fill value -32767)
3. the maximum value of the robust standard deviation (RSD) is too high

It is suspected that the cause of these issues is at the orbit level caused possibly by incorrect setting of fill values in the write_easy_fcdr_from_netcdf.py.

Fix Ch3a uncertainty allocation problem for AVHRR/1 and AVHRR/2

Global statistics suggests that data is being incorrectly stored in memory for the independent and structured uncertainties on the Ch3a of the earlier editions of the AVHRR (prior to NOAA-15).

It is suspected that this is due to incorrect initiatialisation of the Ch3a array in the AVHRR_Data type in fiduceo_uncertainties.f90.