noaa-gfdl/fre-nctools

Tools for manipulating and creating netCDF inputs for FMS managed models

fre-nctools's Issues

poly_area() gives wrong values for area when the side of polygon crosses a pole

Describe the bug
The exchange grid tool uses a function poly_area(), which returns the area of a polygon on globe with given vertices, to calculate the (exchange) grid cell areas. This function gives an unreasonably large values when the side of such polygons crosses a pole. This is because the line inegral formula which the function is based on is not valid when such crossing happens and is bigger than what it should be by pi*R**2 (which explains why the returned area is almost half the area of the Earth).

This pole crossing happens mostly for stretched ATM grids (non-GCA) giving rise to tiling errors for those grids and holes in the Antarctic land_mask.

To Reproduce
Get a version of frenctools with more verbosity for the exchange grid creation tool:

bash
git clone -b nnz_NONGCA_strechedGridIssues https://gitlab.gfdl.noaa.gov/fre/fre-nctools.git fre-nctools_gitlab
cd fre-nctools_gitlab
git checkout adde039  #adds calculated atm_area to land_mask files
export CONFIG_SITE=site-configs/gfdl-ws/config.site 
source site-configs/gfdl-ws/env.sh 
module load gcc/6.2.0
export PATH=.:$PATH
autoreconf -i
./configure 
make

Now create a coupled mosaic with a stretched grid for ATM such as c256r25tlat32.0_om4p25.
Explore the values of area_atm array along the longitude in the tile that contains the south pole (land_mask_tile3.nc) . See figure below.

Expected behavior
area_atm should not give huge values.

System Environment
fre-nctools on PAN

Additional context
See issue #44 for the inaccuracies in poly_area even for non-stretched grids.

Figure below shows the value of area_atm in land_mask_tile3.nc

pathname320='/work/Niki.Zadeh/MOM6-examples_myfork/ice_ocean_SIS2/OM4_025/preprocessing.20201028/mosaic_c256r25tlat32.0_om4p25_githubMaster_Verbose_20210121/'          
land_mask0 = nc.Dataset(pathname320+'land_mask_tile3.nc')
mask320= np.array(land_mask0.variables['mask'])
area_atm320= np.array(land_mask0.variables['area_atm'])
plt.plot(area_atm320[128,:],marker='o')

Same as above after a fix to poly_area formulation to take into account the pole crossing for a polygon edge.

There is still the discontinuity mentioned in issue #44

Bring in last updates from GFDL's gitlab

GFDL's gitlab had been pushing new updates, which stopped working in mid-June due to new large committed files triggering github's pre-commit hook which refuses files larger than 100 MB.

Missing updates from GFDL's gitlab (https://gitlab.gfdl.noaa.gov/fre/fre-nctools) need to be added here.

Any large files can be symlinked to GFDL's filesystem.

These updates are mostly updates for the Bronx-18 hydrology grid generating tool.

run.cr_simple_hydrog.csh fails and wipes the entire FTMPDIR

When I try to run run.cr_simple_hydrog.csh from a sub-directory of $FTMPDIR, it fails with the following messages:
an106: /vftmp/Sergey.Malyshev/pid20895/hydro > ./run.cr_simple_hydrog.csh
ERROR: invalid platform
In addition, everything in the $FTMPDIR directory is wiped out by the script. This could potentially result in significant loss of work, if there are other directories in $FTMPDIR that are being used for, say, data accumulation or processing. The "invalid platform" error message appears to be spurious, caused by the fact that in my case the current working directory was also removed.
Ideally, a script should create its own unique temporary directory, if necessary.
@slm7826
@ngs333 This is gitlab issue #65

Add env.sh for hera

The use of lmod on hera does not allow a user to have two compiler modules loaded. This causes a problem when building with the Intel compilers. The Intel compilers use the version of gcc that appears first in PATH to determine what C standard to use when compiling. FRE-NCtools is using the C11 standard. The system GCC on hera is too old and doesn't have a full C11 implementation.

To get past this problem, an env.sh similar to what is used on GFDL and gaea would help a user get past the issue.

Update build instructions for case using site-config env.sh

The build instructions should be updated to reflect the recent site-config env.sh changes.
More specifically, they should be augmented (with commands) for the case of building on
some GFDL sites where the site-config env.sh can be used.

NCtools should compile with debug options with intel compiler.

Function mppnccombine does not compile out of the box with the Intel 19 compiler with common
debug options. With CFLAGS set to "-g -O0 -sox -ftrapuv -qopenmp", the compiler yields
the error "undefined reference to `min'" wherever the function min is used. Merely changing the
flags to "-O3 -sox -qopenmp -debug minimal" will result in compilation sucess.

Its as if with the debug option, inline functions are not placed (do to a definition) in the translation unit and
wherever they are not actually inlined (at compile time, whcih happens sometimes since "inline"is merely a suggestion), then there is no function for them to link to.
One possible solution is to add
"extern inline int min(int a, int b)" underneath the definition of min.
BTW, min is declared as:
inline int min(int a, int b)
{
if (a<b) return a;
return b;
}

make_coupler_mosaic breaks with --wave_mosaic flag after bronx-11

Describe the bug
The make_coupler_mosaic program fails due to a memory error if run with the --wave_mosaic flag. This issue appears in bronx-12 and subsequent releases, but is absent in bronx-11 and several prior versions.

To Reproduce

A simple idealized domain with the --wave_mosaic flag will reproduce the error. The following set of commands is successful in bronx-11, but fails in more recent versions:

make_hgrid --grid_type regular_lonlat_grid --nxbnd 2 --nybnd 2 --xbnd 100,110 --ybnd 10,20 --nlon 20 --nlat 20 --grid_name ocean_hgrid
make_solo_mosaic --num_tiles 1 --dir ./ --mosaic_name ocean_mosaic --tile_file ocean_hgrid.nc --periodx 360 --periody 360
make_solo_mosaic --num_tiles 1 --dir ./ --mosaic_name atmos_mosaic --tile_file ocean_hgrid.nc --periodx 360 --periody 360
make_solo_mosaic --num_tiles 1 --dir ./ --mosaic_name land_mosaic --tile_file ocean_hgrid.nc --periodx 360 --periody 360
make_solo_mosaic --num_tiles 1 --dir ./ --mosaic_name wave_mosaic --tile_file ocean_hgrid.nc --periodx 360 --periody 360
make_topog --mosaic ocean_mosaic.nc --topog_type rectangular_basin --bottom_depth 4000
make_coupler_mosaic --atmos_mosaic atmos_mosaic.nc --land_mosaic land_mosaic.nc --ocean_mosaic ocean_mosaic.nc --wave_mosaic wave_mosaic.nc --ocean_topog topog.nc --mosaic_name grid_spec

Expected behavior
The make_coupler_mosaic program fails with a segmentation fault. On Gaea, with the --verbose --check flags, the error is slightly different depending on whether running with a different wave_mosaic.nc and ocean_mosaic.nc or supplying the same mosaic file for each.

With --wave_mosaic wave_mosaic.nc:
"Error in `make_coupler_mosaic': munmap_chunk(): invalid pointer: 0x000000" when run

With --wave_mosaic ocean_mosaic.nc:
"Error in `make_coupler_mosaic': free(): invalid pointer: 0x000000000b2ac900"

System Environment
The bug is found using default installations via module load fre/bronx-15 (or any release after 11) on both Gaea and the GFDL workstations.

split_ncvars.pl should use relative-path perl

Is your feature request related to a problem? Please describe.
split_ncvars.pl currently uses the system Perl (/usr/bin/perl) via its shebang. It would be better to use the first perl in the user's PATH, and would support more varied system environments.

#!/usr/bin/perl

Describe the solution you'd like
Change split_ncvars.pl's shebang to use the first perl in the user's environment:

#!/usr/bin/env perl

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Add automake test for C99 standards compliance

Is your feature request related to a problem? Please describe.
A recent gcc version is required, even if non-GNU compilers, in order to support modern C99 C syntax.

Describe the solution you'd like
A automake configure check to warn users that their compiler isn't supported. (Otherwise, the first indication of a problem will be a compiler error on the new C99 syntax)

Until then, documentation describing this requirement should be added to README.md

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

fregrid writing with incorrect nc file type

Describe the bug
Fregrid is is not writing files as it indicated in its the usage instructions. According to the instructions,
for the format argument, "When format is not specified, will use the format of input_file."
What it is actually doing is that when the format is not specified, it is using the mpp_io global
in_format, which was set when the last file was opened, which is not necessarily the same as
the in_format of the input_file file. This contributes to the GFDL issue ticket #5019381
("bronx-18: fregrid fails to remap tiled files with netcdf3/netcdf4 cconfusion")

For further information on the problem see the GFDL ticket.

Issues with simple_hydrog and netCDF-Fortran

Describe the bug

There is a GNU make issue with how a couple directories in simple_hydrog handle netCDF-Fortran. They assume netCDF-Fortran was built in a specific way. For example I had to change these:

FRE-NCtools/tools/simple_hydrog/rmvpr/Makefile.am

Line 9 in 214e569

LDADD = $(NETCDF_LDFLAGS) $(NETCDF_LIBS) -lnetcdff -lnetcdf -lhdf5_hl -lhdf5 -lz -ldl

FRE-NCtools/tools/simple_hydrog/postp/Makefile.am

Line 9 in 214e569

LDADD = $(NETCDF_LDFLAGS) $(NETCDF_LIBS) -lnetcdff -lnetcdf -lhdf5_hl -lhdf5 -lz -ldl

in the final Makefile to look like:

LDADD = $(NETCDF_LDFLAGS) $(NETCDF_LIBS) $(NETCDF_FCLIBS)

In our netCDF we have a looooooot of stuff we built our netCDF-Fortran with:

$ /discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/bin/nf-config --flibs
-L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lnetcdff -lnetcdf -lm 
-L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lnetcdf -ljpeg -lmfhdf -ldf -ljpeg -lhdf5_hl -lhdf5 -lm 
-L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lmfhdf -ldf -lsz -ljpeg -lgpfs 
-L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lcurl -lssl -lcrypto -lssl -lcrypto -lz -lm -ldl -lm 
-L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lcurl -lssl -lcrypto -lssl -lcrypto -lz -lm

so I think it's best to use NETCDF_FLIBS if passed in by the user.

To Reproduce

I ran make and got:

mpiifort -I../../../tools/libfrencutils -real_size 64 -I/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/include/netcdf -g   -o rmv_parallel_rivers rmv_parallel_rivers.o ../../../tools/libfrencutils/libfrencutils.a -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib  -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lnetcdf -ljpeg -lmfhdf -ldf -ljpeg -lhdf5_hl -lhdf5 -lm -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lmfhdf -ldf -lsz -ljpeg -lgpfs -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lcurl -lssl -lcrypto -lssl -lcrypto -lz -lm -ldl -lm -lnetcdff -lnetcdf -lhdf5_hl  -lhdf5 -lz -ldl
ld: /discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib/libnetcdf.a(hdf4file.o): in function `hdf4_rec_grp_del':
/gpfsm/dswdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/src/netcdf/libhdf4/hdf4file.c:70: undefined reference to `SDendaccess'
ld: /discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib/libnetcdf.a(hdf4file.o): in function `hdf4_read_att':
/gpfsm/dswdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/src/netcdf/libhdf4/hdf4file.c:300: undefined reference to `SDattrinfo'
ld: /gpfsm/dswdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/src/netcdf/libhdf4/hdf4file.c:322: undefined reference to `SDreadattr'
ld: /discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib/libnetcdf.a(hdf4file.o): in function `hdf4_read_dim':
/gpfsm/dswdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/src/netcdf/libhdf4/hdf4file.c:359: undefined reference to `SDgetdimid'
ld: /gpfsm/dswdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/src/netcdf/libhdf4/hdf4file.c:361: undefined reference to `SDdiminfo'
ld: /discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib/libnetcdf.a(hdf4file.o): in function `hdf4_read_var':
/gpfsm/dswdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/src/netcdf/libhdf4/hdf4file.c:502: undefined reference to `SDselect'
ld: /gpfsm/dswdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/src/netcdf/libhdf4/hdf4file.c:506: undefined reference to `SDgetinfo'
ld: /gpfsm/dswdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/src/netcdf/libhdf4/hdf4file.c:511: undefined reference to `SDgetchunkinfo'
ld: /gpfsm/dswdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/src/netcdf/libhdf4/hdf4file.c:522: undefined reference to `SDgetfillvalue'
ld: /discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib/libnetcdf.a(hdf4file.o): in function `NC_HDF4_open':
/gpfsm/dswdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/src/netcdf/libhdf4/hdf4file.c:629: undefined reference to `SDstart'

and it kept going. It was missing the link to cURL (we build with OpenDAP support), the link to the zip and szlib libraries, etc.

Expected behavior
The file should link.

System Environment
Describe the system environment, include:

• OS: SLES 12 SP3
• Compiler(s): Intel 19.1.2 IMPI 19.1.2
• netCDF Version: 4.7.4 (C), 4.5.2 (Fortran)
• Configure options:

  $ ./configure --prefix=/discover/swdev/gmao_SIteam/other/SLES12.3/FRE-NCtools/2020Nov10 --with-netcdf=/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux --with-netcdf-fortran=/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux --with-mpi=yes CC=mpiicc FC=mpiifort NETCDF_LIBS=-L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lnetcdf -ljpeg -lmfhdf -ldf -ljpeg -lhdf5_hl -lhdf5 -lm -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lmfhdf -ldf -lsz -ljpeg -lgpfs -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lcurl -lssl -lcrypto -lssl -lcrypto -lz -lm -ldl -lm NETCDF_FCLIBS=-L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lnetcdff -lnetcdf -lm -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lnetcdf -ljpeg -lmfhdf -ldf -ljpeg -lhdf5_hl -lhdf5 -lm -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lmfhdf -ldf -lsz -ljpeg -lgpfs -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lcurl -lssl -lcrypto -lssl -lcrypto -lz -lm -ldl -lm -L/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux/lib -lcurl -lssl -lcrypto -lssl -lcrypto -lz -lm

This is a bit long, so in other words I do:

BASEDIR=/discover/swdev/gmao_SIteam/Baselibs/ESMA-Baselibs-6.0.16/x86_64-unknown-linux-gnu/ifort_19.1.2.254-intelmpi_19.1.2.254/Linux

NETCDF_LIBS=$($BASEDIR/bin/nc-config --libs)
NETCDF_FCLIBS=$($BASEDIR/bin/nf-config --flibs)

PREFIX=/discover/swdev/gmao_SIteam/other/SLES12.3/FRE-NCtools/2020Nov10

./configure --prefix=${PREFIX} --with-netcdf=${BASEDIR} --with-netcdf-fortran=${BASEDIR} \
   --with-mpi=yes CC=${CC} FC=${FC} NETCDF_LIBS="${NETCDF_LIBS}" NETCDF_FCLIBS="${NETCDF_FCLIBS}"

where I use the output of nc-config --libs and nf-config --flibs to tell the configure script what NETCDF_LIBS and NETCDF_FCLIBS will be as we build netCDF with support.

NOTE: Due to a fun bug in netCDF-Fortran 4.5.3, nf-config --flibs is broken, so that's why I'm not using the newest version. The fix is in their main branch, so hopefully all is well in the next version.

Switch GFDL site configurations to GCC

NCTools needs to document and set GCC compiler flags and settings for use after the 18.1 release.
A good set of flags will consider the parameters of reproducible with Bronx-18, speed of execution, and help in debugging working executables, and help in adding and maintaining code.

Both the config.site files and documentation may need changes.

Make and Make Install running into errors

Background
I have data from an AM4 run and want to regrid the six .nc files from cubed-sphere to a lat-lon grid.
The data is stored on an HPC in a folder I can write to. The HPC has netCDF and other key modules already installed. The only thing I'm missing is this package/repo, FRE-NCtools, but I'm having trouble installing it.
The documentation suggests that I have the permissions to do, so I believe as long as I install it to a home folder, I shouldn't run into issues with needing root privileges (documentation).

What I've have tried

1. I forked the repo and created a new folder in /FRE-NCtools/site-configs/ to match the conventions of the HPC. The files are viewable here (My Site Folder). I may have over-added modules because I couldn't find nccmp in the list available. I had trouble finding much information about it online (not sure what the abbreviation is...). It ran further when I got rid of the config.site file, hence why there isn't one.
2. I cloned the fork into my home folder (/home/users/)
   git clone <my_url>
3. I moved into the new directory cd ./FRE-NCtools/
4. Set the environment variable CONFIG_SITE and set the recommended environment
   export CONFIG_SITE=/home/users/mborrus/FRE-NCtools/site-configs/stan/config.site source /home/users/mborrus/FRE-NCtools/site-configs/stan/env.sh
5. autoreconf -i
6. ./configure --prefix=/home/users/mborrus
7. make
   Errors incoming! It's long, so I copied and pasted it here.
8. I tried pressing on, not knowing if these were fatal errors, so I tried a make install
   More errors! These are linked here. .

I can't quite decipher what's going wrong here, or what may be causing it. Is it due to my lack of config.site file?

What's a good way to check if the install is successful?

Thanks for any help and let me know if I'm forgetting anything...

Wrong nc-config found.

Describe the bug
The ax_lib_netcdf.m4 macro's search for nc-config and nf-config may find the wrong n[cf]-config when searching for the netcdf config application.

To Reproduce
Have a nc-config installed in /usr/bin along with one installed in a different location. (For example on a GFDL workstation). Check the config.log to see which config application was found.

Expected behavior
The nc-config in the first directory listed in PATH should be returned.

System Environment
Describe the system environment, include:

• OS: RHEL 7.9
• Compiler(s): All
• netCDF Version: N/A
• Configure options: N/A

Additional context
This is likely due to how the macro attempts to set a prefix bin location in AC_PATH_PROGS.

Add env.sh for orion

The use of lmod on orion does not allow a user to have two compiler modules loaded. This causes a problem when building with the Intel compilers. The Intel compilers use the version of gcc that appears first in PATH to determine what C standard to use when compiling. FRE-NCtools is using the C11 standard. The system GCC on orion is too old and doesn't have a full C11 implementation.

To get past this problem, an env.sh similar to what is used on GFDL and gaea would help a user get past the issue.

gaea modulefile environment env.sh needs CLE7 update

Describe the bug
The environment setup file for gaea (site-configs/ncrc/env.sh) needs to be updated to reflect gaea's current software environment.

To Reproduce
. site-configs/ncrc/env.sh on gaea should load a functional Intel compilation environment

Expected behavior
. site-configs/ncrc/env.sh on gaea should load a functional Intel compilation environment

System Environment
Describe the system environment, include:

• OS: [e.g. RHEL 7.2]
• Compiler(s): Type and version [e.g. Intel 19.1]
• netCDF Version: For both C and Fortran
• Configure options: Any additional flags, or macros passed to configure

Additional context
Add any other context about the problem. If applicable, include where any files
that help describe, or reproduce the problem exist.

Checking nc files for a null character

Is your feature request related to a problem? Please describe.
This issue can be found in this FMS issue. Users will need a way to check if the attributes on a netcdf file have a null character added to the end of their strings.

Describe the solution you'd like
A program that will detect the null character

Describe alternatives you've considered
none

Additional context
This is needed so users can figure out which files are exercising a bug in mpp_io so that they can be prepared when their answers change when switching to fms2_io.

Does make_topog need MAXXGRID, nthreads?

Is your question related to a problem? Please describe.

I am trying to generate a topography:

make_topog --mosaic ocean_mosaic.nc \
           --topog_type realistic \
           --topog_file ${topo_file} \
           --topog_field ${topo_var} \
           --bottom_depth 7500.0 \
           --min_depth 10.0 \
           --scale_factor -1 \
           --verbose \
           --output ocean_topog.nc

And I get following output:

NOTE from make_topog ==> the topog_type is: realistic
NOTE from make_topog ==> x_refine is 2, y_refine is 2


 ************************************************************

NOTE from make_topog ==> input arguments

NOTE from make_topog ==> min_depth is: 10.000000
NOTE from make_topog ==> topog_file is: /discover/nobackup/sakella/MOM6-GFDL/Grids/etopo1/etopo1_n.nc
NOTE from make_topog ==> topog_field is: z
NOTE from make_topog ==> scale_factor is: -1.000000
NOTE from make_topog ==> num_filter_pass is: 1
NOTE from make_topog ==> kmt_min is 2
NOTE from make_topog ==> fraction_full_cell is 0.200000
NOTE from make_topog ==> min_thickness is 0.100000
NOTE from make_topog ==> no vgrid_file is specified
NOTE from make_topog ==> not allow non-advective tracer cells
NOTE from make_topog ==> open this cell
NOTE from make_topog ==> adjust topography


 ************************************************************

**************************************************
Begin to generate topography

==>NOTE from get_boundary_type: x_boundary_type is cyclic

==>NOTE from get_boundary_type: y_boundary_type is fold_north_edge
FATAL Error: nxgrid is greater than MAXXGRID/nthreads, increase MAXXGRID, decrease nthreads, or increase number of MPI ranks
Abort(-1) on node 0 (rank 0 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, -1) - process 0

Describe what you have tried

ocean_hgrid.nc was generated with:

make_hgrid --grid_type tripolar_grid \
           --nxbnd 2 --nybnd 2 \
           --xbnd -280,80 --ybnd -90,90 \
           --dlon 5.0,5.0 --dlat 5.0,5.0 \
           --center c_cell \
           --grid_name ocean_hgrid

And then used that to generate the ocean_mosaic.nc

make_solo_mosaic --num_tiles 1 \
                 --dir . \
                 --tile_file ocean_hgrid.nc \
                 --periodx 360. \
                 --mosaic_name ocean_mosaic

I get the exact same error with make_topog_parallel as well. I also tried mpirun -np 10 make_topog_parallel ... all args as above.

@mathomp4 would be able to fill in the build details on NCCS NASA. Thank you @mathomp4

Appreciate your help, Thanks!

Review and enhance all test scripts

The 30-or-so test scripts currently run on PRs test basic functionality of most of the tools. However, some of the tests have commented sections that probably were done to get tests to pass while improving the github testing environment. Other commented sections were probably left from the original test development.

In some cases the commented-out sections should be returned to active use. For example, Test12 (mppncscatter) scatters a test input file, then combines using mppnccombine. However, the nccmp to compare the resulting files is commented out.

In other cases, the commented-out sections were for tests that were too unwieldy to run (too many or too large or too slow). These should be assessed for what functionality was being tested, and determine if a smaller test can be added to replace the unwieldy test.

In addition to resolving commented out test script sections (either delete, uncomment, or adapt), test scripts should be reviewed to verify they are testing what they appear to be testing, and that we are testing the important functionality of each tool. Additional tool options and build options should be considered as well.

exchange grid clipper tool does not work correctly around the tripolar fold

Describe the bug
The tool to create the exchange/overlap grid between two given grids gives wrong answers for certain grid cells adjacent to ocean tripolar fold in the vicinity of the North pole. This gives rise to issues (large tiling errors) for exchange grids, particularly involving a stretched ATM grid and a tripolar OCN grid (see FMS issue 621).

To Reproduce
I added these end cases as unit tests in create_xgrid.c
On gfdl-ws:

bash
git clone -b nnz_NONGCA_strechedGridIssues https://gitlab.gfdl.noaa.gov/fre/fre-nctools.git fre-nctools_gitlab
cd fre-nctools_gitlab
export CONFIG_SITE=site-configs/gfdl-ws/config.site 
source site-configs/gfdl-ws/env.sh 
module load gcc/6.2.0
export PATH=.:$PATH
autoreconf -i
./configure 
make
cd tools/libfrencutils
make
icc -Dtest_create_xgrid -Ddebug_test_create_xgrid -g -O0 ./create_xgrid.c -o test_create_xgrid -L. -lfrencutils
./test_create_xgrid

Expected behavior

Tests 16,17,18 are giving wrong answers. In particular:

• test 17: The first grid box totally contains the second grid box and should return the corners of second grid box.
• test 18: The first grid box totally outside the second grid box (except for common corner at pole and should return nothing.

System Environment
Describe the system environment, include:

• OS: tested at gfdl workstation and PAN
• Compiler(s): icc
• netCDF Version: For both C and Fortran
• Configure options: Any additional flags, or macros passed to configure

Additional context
The issue happens if the bigger ATM cell side is crossing the tripolar fold at the pole
and the smaller OCN grid is along the return path of the fold. This is perhaps important and at the heart of the issue since the tripolar fold is know to have weird symmetry properties.

In the following figures the red box is the ATM grid cell, the blue is the OCN and the circles are the corners of the clipped cell. Note that longitude 0 is along the vertical line going down from the pole and the OCN tripolar fold runs diagonally along 60 longitude.

The following figure is what test 16 produces. The clipped cell is outside the red input box. It should be the corners of the overlap!

The following figure shows a correct overlap found if the ATM grid cell is mostly "above" the fold.

The following figure is what test 18 produces. The clipped cell is totally outside the red input box. It should return no overlap corners!

fregrid bilinear interpolation gives different lat bounds than expected

Describe the bug
When specifying the non-global output grid with fregrid (using the --latBegin, --latEnd, --lonBegin, --lonEnd options), the output grid is different depending on the interpolation method. In particular, the bilinear option gives a different output grid than the others.

To Reproduce
Use fregrid as usual to regrid a grid to a non-global output, using the --(lat|lon)(End|Begin) options. Use the regular option first (conserve_order1), then try bilinear.

Expected behavior
The output grid should be the same regardless of interpolation option.

System Environment
Any

Additional context
Add any other context about the problem. If applicable, include where any files
that help describe, or reproduce the problem exist.

README instructions do not build on GFDL workstations and PP/AN

Describe the bug
A clear and concise description of what the bug is

To Reproduce
GFDL-WS

. site-configs/gfdl-ws/env.sh
export CONFIG_SITE=/path/to/FRE-NCtools/site-configs/gfdl-ws/config.site
autoreconf -i
./configure
make

PP/AN

. site-configs/gfdl/env.sh
export CONFIG_SITE=/path/to/FRE-NCtools/site-configs/gfdl/config.site
autoreconf -i
./configure
make

Recent updates to mosaic_util.h require a more recent version of gcc, and result in compile errors such as

create_xgrid.c(2190): error: expression preceding parentheses of apparent call must have (pointer-to-) function type
        fint = f1 + (f2-f1)*(fac-dphi1)/fabs(dphi);

Expected behavior
Successful compilation.

System Environment
Describe the system environment, include:
GFDL workstations (RHEL-7):
Currently Loaded Modulefiles:

1. hdf5/1.8.8 3) intel_compilers/18.0.5 5) mpich2/1.5b1
2. netcdf/4.2 4) nccmp/1.8.2.0

PP/AN analysis host (RHEL-6):
Currently Loaded Modulefiles:

1. globus/system-or-6.0 3) intel_compilers/18.0.5 5) hdf5/1.8.8 7) nco/4.5.4
2. slurm/20.02 4) mpich2/1.2.1p1 6) netcdf/4.2 8) nccmp/1.8.2.0

Additional context
Add any other context about the problem. If applicable, include where any files
that help describe, or reproduce the problem exist.

make_hgrid segfaults in Test04-grid_coupled_nest.sh (on GFDL and GFDL-WS sites)

Describe the bug
make_hgrid segfaults when using particular options and compiled on GFDL or GFDL-ws. This may be a recent problem, possibly since #63

To Reproduce
Compile as usual on GFDL or GFDL-WS. Run Test04, or run the failing test directly which is:

# Create C144 grid for land
make_hgrid 
 --grid_type gnomonic_ed 
 --nlon 288 
 --grid_name land_grid 
 --do_schmidt 
 --target_lat 48.15 
 --target_lon -100.15 
 --halo 3 
 --stretch_factor 3 
 --great_circle_algorithm 
 --nest_grid 
 --refine_ratio 3 
 --parent_tile 0

Expected behavior
Pass the tests.

System Environment
GFDL and GFDL-ws as set in their env.sh files

Additional context
Add any other context about the problem. If applicable, include where any files
that help describe, or reproduce the problem exist.

Report compiler type/version and library information with --version option

Is your feature request related to a problem? Please describe.
Similar to #52, it would be useful for debugging user issues to quickly determine which compiler and library versions were used during compilation.

Describe the solution you'd like
The tools could report this information in a format similar to cdo:

>cdo --version  
Climate Data Operators version 1.7.2 (http://mpimet.mpg.de/cdo)
Compiled: by Hans.Vahlenkamp on hnv (x86_64-unknown-linux-gnu) Jul 11 2017 22:44:06
Compiler: gcc -std=gnu99 -g -O2 -fopenmp 
 version: gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18)
Features: DATA PTHREADS OpenMP3 HDF5 NC4/HDF5 UDUNITS2 SSE2
Libraries: HDF5/1.8.8
Filetypes: srv ext ieg grb nc nc2 nc4 nc4c 
     CDI library version : 1.7.2 of Jul 11 2017 22:43:14
 CGRIBEX library version : 1.7.5 of Jun  3 2016 14:44:00
  NetCDF library version : 4.2 of May 23 2012 12:14:03 $
    HDF5 library version : 1.8.8
 SERVICE library version : 1.4.0 of Jul 11 2017 22:43:00
   EXTRA library version : 1.4.0 of Jul 11 2017 22:42:56
     IEG library version : 1.4.0 of Jul 11 2017 22:42:59
    FILE library version : 1.8.2 of Jul 11 2017 22:42:56

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Reduce repository size

Is your feature request related to a problem? Please describe.
At 1.6 GB currently, this repo is a bit unwieldy to clone.

Describe the solution you'd like
Keep the test files somewhere else (ftp server or git-lfs) so that the repo is much smaller.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Discussion on plans and instructions related to future code sanitation and PRs

PR #67 suggests that perhaps more of the code base needs additional sanitation. This may be for code that was modified and that there is a PR, or just code in general if time permits. PR 67s make_hgrid only core dumps in some situations, but even in situations where it doesn't core dump, Intel Inspector shows
that there are invalid memory reads. Additionally, compiling NCTools with maximum warnings shows (at least with gcc) a large number of warnings for numerous gcc apps. This issue is to discuss if there is to be any testing beyond what is commonly currently done, and/or or instructions issued concerning related testing. Here are some possible items:
a) Testing with valgrind or intel inspector and removing all critical warnings.
b) Keeping a history of apps and results that were tested under a)
c) Compiling with additional warnings. Provide instructions for doing so (Telling the user to look at the warnings; to ask for instructions. Keeping a list giving guidance of acceptable warning types. And asking users
d) Encouraging more unit tests.
e) Doing any of the above at PRs? Doing them on a schedule?

Improve affinity.c conditional compilation and remove duplicate.

There are two versions of the affinity.c functions:
/tools/simple_hydrog/lakes/affinity.c
/tools/libfrencutils/affinity.c
They actually try to do the same thing so one of them should be removed. One should consider putting them in
a utility directory, and also having it be as similar as possible to that in libFMS.

Additionally, they are conditionally compiled depending on available OS libraries.
The flag HAVE_GETTID is used to control the compilation. This flag is being picked up (through a # include statement in the .c files) from the autotools generated config.h file. It is preferable that instead autotools have the CFLAGS and FCFLAGS
determine a -D HAVE_GETTID compile option flag and add that to the the CFLAGS and FCFLAGS.

ax_lib_netcdf.m4 has some typos

Describe the bug
The ax_lib_netcdf.m4 has some typos causing it to fail in strange ways on some systems. The typos are from poor copy/paste reproduction of sections for C and Fortran. These sections are nearly identical, but use slightly different variable names. Some were not modified correctly.

To Reproduce
Build with autoconf on affected systems. On these systems, some of the include/library directories will not be prefixed with the -I/-L compile flags.

Expected behavior
All include/library options will be prefixed with the correct flag

System Environment
Mac OS X is one of the affected systems

remap_land needs to work with lm4.1 and lm4.2 data

This is an enhancement ticket to upgrade NCTools remap_land application to work with lm4.1 and lm4.2 (CANDIDATE) files in addition to the lm4.0 with which it currently works. It appears that the existing remap_land application makes several assumptions about the fields, their dimensions and indices that are not valid assumptions for the data of lm4.1 and lm4.2. A version working for lm4.1 and lm4.2 is needed by Sergey Malyshev and Meiyun Lin of GFDL.

After analyzing data from lm4.2, it was determined that new dimensions need to be supported:
nspecies (float type)
textlen (float)
soilCCohort (float)
litterCCohort (float)

The lm4.0 had these combinations of dimensions supported:
[zfull, tile_index]
[ tile_index]
[cohort_index]
And they had an optional “time” index on the left.

Inspection of lm4.2 data used in the original ticket indicates lm4.2 needs these dimensions additionally:
[soilCCohort, zfull, tile_index]
[ literCCohort]

There are numerous int or double (float) variables or fields using the above dimensions. Additionally, there is one character field in lm4.2 :
species_names (type char; dimensions : nspecies , textlen).

Testing:
Sergey Malyshev has suggested these two tests for identifying inconsistencies between grid spec and the created restart files:
A)
Check if the masks of the generated remapped restart files of glacier/lake/soil/vegetation data match the land mask of the specs of the input destination (dst) grid . To this end I’ve created a comparison program that compared land_mask_tile<i>.nc files produced by remap_land to the corresponding files in dst_grid. Note that some of these files are pre-processed by the “decompress-ncc” utility
[ update 1/25/2021 More specifically, the comparison is the field "mask" of file /dst_grid/CXY_grid_tile<i>.nc against the field "frac" (and independently the field "vegn") from file /dst_cold_start/land_mask_tile<i>.nc and also from file /dst_restart/land_maks_tile<i>.nc.]

[ Update 1/15/2021 it can be noted that the remap_land app produces a land mask_tile file that is made from both the cold_start land mask file and the grid tile. For example, see this line
land_frac_dst[ntile_dst * i + pos] = soil_frac_src[ntile_src * p + l] * soil_frac_cold[i] ]

B)
 Sergey suggest that the ultimate test is to run the lm4p model code using as input the dst grid spec and the remapped restart files. If the spec and the restart files don't match, then the model will issue errors. Sergeys lm4p model code that is in branch user/slm/dev is used. The xml to run the model is a copy of this one:
/ncrc/home1/Sergey.Malyshev/devel/ppa/2020.03/rts.xml.

My copy is located at :
/ncrc/home1/Miguel.Zuniga/remap/sergey/lm4P_xmls/rts.xml

The experiment is a copy of the original experiment “LM4p2c2-SPEAR-ESM4-test”, but it has the gridSpec and initCond specified to use files used by and created by a run of the land_remap app :

<experiment name="LM4p2c2-SPEAR-ESM4-remap
   <description>
      <dataFile label="initCond" target="INPUT/" chksum="" size="" timestamp="">
        <dataSource 
site="ncrc"> /lustre/f2/dev/Miguel.Zuniga/remap/lm4p2c1_SPEAR_18700101.remapped.v20201215.tar </dataSource>
      </dataFile>
      <dataFile label="gridSpec" target="INPUT/" chksum="" size="" timestamp="">
            <dataSource site="ncrc">/lustre/f2/dev/Miguel.Zuniga/remap/mosaic.tar</dataSource>
      </dataFile>

Testing status Jan 4 2020:

Test type B) above suggest that there are mismatches. It indicates that there non-zero points on the remap restart file that are zero on the the grid spec. In contrast, test type A) above actually suggest that the grid Spec and the remapped file are in agreement.

The mismatches identified by type B) are all "R" on the output plots : i.e. they are present in the
restart but not in the gridSpec. lm4p code has these point values to integer 0; whereas the land_mask_tile<i>.nc files has them as a float value greater than 0. but less than 1.0. (Note, all purely ocean points have the value 0. and purely land points have the value 1.)

Additionally, all the mismatches identified are right on a ocean/land boundary.

Additionally, the mismatches on land_mask_tile1.nc are 36 in total – from a total of about 450 land/ocean boundary points. (There are additional mismatches in the other five tiles, also on land/ocean boundary points, qualitatively similar to that of tile 1)

The grid spec is provided in file mosaic.tar. I've been using as the "grid spec" file land_mask_tile<i>.nc. But I've noticed mosaic.tar has additional files

remap_land code location and status:
The new code is on this repo and branch:
https://github.com/ngs333/FRE-NCTools
branch:  remap_land_newdata

The code used on the test was of the last merge before Dec 5. There is an updated version (not yet merged) that is the result of running static analyzers and Intel inspector on the tests.
The updated code is not changing the results of remapped land.

Remove combine_restarts

Is your feature request related to a problem? Please describe.
Since Bronx-11, FRE's output.stager has combined iceberg restart files using (FRE-NCtool) iceberg_comb.sh. However, combine_restarts has not been similarly updated so is no longer working properly.

Describe the solution you'd like
combine_restarts should be removed.

Describe alternatives you've considered

• Updating combine_restarts to use iceberg_comb.sh and then have output.stager use combine_restarts. This would be a bit dangerous as combine_restarts was designed as an interactive tool and might not work properly in the high-volume output.stager use.
• Updating combine_restarts to use iceberg_comb.sh and leave output.stager as is. This would be reasonable, but we suspect no one is using this tool. If anyone misses this tool, we will bring it back and fix it properly.

Additional context
Add any other context or screenshots about the feature request here.

automation script for stretched grid development

The automation of the stretched grid process for models like LM4/AM4.

The main focus of the script is to automate the process of stretched grid on the basis of the user’s input of required several variables.

This stretched grid process it has been done without the aid the great circle.

The user will be able to input arguments on the auto_grid_stretched_wo_greatcir.csh script.

FREgrid with time-averaged variables

Describe the bug
A clear and concise description of what the bug is

On some model runs it appear that the variable 'nv' is type 'double' compared to normally being type 'float'. This causes an issue with the 'time_bnds' variable that is type 'float'. The type mismatch makes it so fregrid will fail when trying to regrid to a lat/long grid with the following error:
Error from pe 0: mpp_io(mpp_copy_var_att): Error in copying att _FillValue of var time_bnds

To fix this issue, the workaround has been to use ncap2 to convert 'time_bnds' to type 'double'
This command is what was used to convert the variable 'time_bnds':
module load nco
ncap2 -s 'time_bnds=double(time_bnds)' input_file output_file

To Reproduce
Steps to reproduce the behavior

An example runscript can be found here to create the data:

Runscript:
gaea: /ncrc/home1/Lucas.Harris/SHiELD/SHiELD_run/run-uniform/C48_z12

diag_table:
gaea: /ncrc/home1/Lucas.Harris/SHiELD/SHiELD_run/common/diag_table_xshield

When running the fregrid command on the time averaged variables such as, 'MAXREF_max', is when the error arises. All other variables that are not time-averaged have no issues when regridding.

Expected behavior
A clear and concise description of what you expected to happen.

When running fregrid, it is expected to complete the fregrid process and output the variable on the specified lat/lon grid.

System Environment
Describe the system environment, include:

• OS: [e.g. RHEL 7.2]
• Compiler(s): Type and version [e.g. Intel 19.1]
• netCDF Version: For both C and Fortran
• Configure options: Any additional flags, or macros passed to configure

The version of FMS used was what is in Rusty's branch, bensonr/r4_chksum_bugfix

Additional context
Add any other context about the problem. If applicable, include where any files
that help describe, or reproduce the problem exist.

Two file examples can be found here: /ptmp/James.Huff/data/20160801.00Z.C3072.L79z12/history/2016080100
The atmos3d_4xdaily_ave_t_dt_gfdlmp.tile1.nc.0000 is the output from the model run where 'nv' and 'time_bnds' are different types that should be able to be recreated with the above runscripts.
The atmos3d_4xdaily_ave_t_dt_gfdlmp.tile1.nc is after recombining the files and using ncap2 to make both 'nv' and 'time_bnds' be type 'double'.

Large netCDF files and compile outputs in git history

Describe the bug

This repo is nearly 1000 times larger than it was two years ago. A couple years ago a clone of this repo downloaded perhaps 1-2 MB of data. Now it is up nearly 1GB. This seems to be because of

1. netCDF files intentionally checked into version control for tests (e.g. 1982a88)
2. compiled artifacts accidentally checked into version control in prior commits. (a58453f). CI can ensure that this doesn't happen.

This large checkout size makes it difficult to download this code especially in an automated pipeline, and adds friction to working with this code base.

Unfortunately, adding data to git repositories is irreversible without rewriting the history (e.g. using git filter branch). The files remain in the .git folder even if a subsequent commit deletes the files from the working tree.

Would you be open to either

1. Moving these data to a another location (e.g. a git submodule/FTP etc) and rewriting the history to remove them?, or
2. using .gitattributes to remove this datafiles from the tarballs built by github, so that users can download the source quickly w/o the test data.

(2) is a lightweight solution that I have found works well with other repos with large test data checked into version control. This is an example of a .gitattributes that removes a directory of test files:

/path/to/directory/of/test/data export-ignore

To Reproduce

git clone https://github.com/NOAA-GFDL/FRE-NCtools

Add iceberg support to combine_restarts

Is your feature request related to a problem? Please describe.
combine_restarts is an interactive script to manually combine restart files. It uses the same logic in output.stager, calling mppnccombine and combine-ncc.

However, combine_restarts was not updated to combine icebergs, which was added to output.stager a few years ago.

Describe the solution you'd like
combin_restarts to combine icebergs

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

New tool to regrid between two different atmos hybrid coordinate grids

Is your feature request related to a problem? Please describe.
plevel.sh regrids from hybrid vertical coordinate to standard pressure levels. A user has requested a similar tool to regrid to a different hybrid coordinate grid. The proposed use case would be to compare output from reanalysis datasets on an identical vertical grid.

Describe the solution you'd like
Tool would accept as inputs:

• Input file, on a hybrid atmos grid. Containing variables to regrid (e.g. ua) and hybrid coordinate specification (ps, pk, bk)
• Requested output hybrid grid parameters (ps2, pk2, bk2). This would take the form of a second input file
• Requested variables to regrid

Tool would output:

• output file, containing requested variables to regrid on the specified output grid

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
vertical pressure P = pk + bk * ps

iceberg_comb.sh gives incorrect error when ncdump isn't available

Describe the bug
iceberg_comb.sh uses ncdump to determine whether there are icebergs present to combine. However, it doesn't check for ncdump availability, and when it's not available, incorrectly reports that the input files are not valid NetCDF files.

To Reproduce
Use iceberg_comb.sh without ncdump available.

Expected behavior
It should check that ncdump is available and executable before running, as it does for ncrcat. If ncdump isn't available, it should give an error to that effect and exit.

System Environment
Describe the system environment, include:

• OS: [e.g. RHEL 7.2]
• Compiler(s): Type and version [e.g. Intel 19.1]
• netCDF Version: For both C and Fortran
• Configure options: Any additional flags, or macros passed to configure

Additional context
Add any other context about the problem. If applicable, include where any files
that help describe, or reproduce the problem exist.

make_coupler_mosaic tool does not reuse ATM grid for LND when they are the same

Describe the bug
The make_coupler_mosaic tool calculates the LND grid (by clipping it from ATM), rather than reusing the ATM grid itself when the two grids are supposed to be the same.
This unnecessarily complicates the problem and adds to avoidable inaccuracies

It does the right thing (reuse) when using the Great Circle Algorithm. The logic for non-GCA is simply missing.

To Reproduce
Use make_coupler_mosaic for non-GCA identical cubed-sphere grids for ATM and LND

Expected behavior
Reuse the ATM grid for LND rather than calculating (via clipping) to form it.

System Environment
PAN

Additional context

Licensing needs to be implemented throughout the project

Is your feature request related to a problem? Please describe.
The licensing is haphazard and inconsistent. Additionally, we don't have the needed overarching LICENSE.md.

Describe the solution you'd like
All files contain a license header referencing LGPL, in the same manner as FMS, with reference to the project.

Describe alternatives you've considered
N/A

Additional context
This is needed for consistency other NOAA-GFDL projects.

GCC/clang c99 standards compliance

Describe the bug
Some files are not c99 standards compliance. This prohibits these files from compiling with newer gcc version 10 and Apple's clang version 9. There are also several compile time warnings that should be easy to fix.

To Reproduce
Compile with gcc 10 or apple clang 9

Expected behavior
The files with errors should compile without error or warning.

System Environment
Not system related.

Add an editorconfig file to the repository

** Is your feature request related to a problem? Please describe.
No problem, but to add a .editorconfig file to help indicate the specific format to use for the project.

** Describe the solution you'd like
A .editorconfig file following the information at https://editorconfig.org

CubedSphere grid cell area calculated by exchange grid tool has discontinuous values around the two poles

Describe the bug
The cubed sphere grid cell area calculated by the exchange grid tool has bumps around the two poles (see figures below). These bumps are not there in the cell area calculated by make_hgrid itself, the tool that creates the CS grid. Note that make_hgrid uses spherical_excess_area() (Gauss-Bonnet formula area_of_polygon = sum of interior spherical angles - 2pi) , whereas the exchange grid tool uses poly_area() ( area_of_polygon = line_intergal sin(lat) d(lon) ). The question is which one of these formulas are more accurate near the singular poles?

Non-GCA Stretched grids have the same symptoms which results in tiling errors.

To Reproduce
Get a version of frenctools with more verbosity for the exchange grid creation tool:

bash
git clone -b nnz_NONGCA_strechedGridIssues https://gitlab.gfdl.noaa.gov/fre/fre-nctools.git fre-nctools_gitlab
cd fre-nctools_gitlab
git checkout adde039  #adds calculated atm_area to land_mask files
export CONFIG_SITE=site-configs/gfdl-ws/config.site 
source site-configs/gfdl-ws/env.sh 
module load gcc/6.2.0
export PATH=.:$PATH
autoreconf -i
./configure 
make

Now create any coupled mosaic grid with a uniform CS grid, e.g., Test03-grid_coupled_model.sh
Plot the calculated area_atm in land_mask_tile6.nc

Expected behavior
computed area in C48_grid.tile6.nc does not have a bump at the pole.

System Environment
FRE-NCTOOLS build on PAN

Additional context
We already know that the poly_area() formula is not correct when side of polygon crosses a pole.

Figure below shows area of ATM grid cells along the longitude passing through pole (at the peak) calculated by the exchange grid tool (poly_area) for c256_om4p25 grid

Same ATM grid cell area as above calculated by the make_hgrid.c (spherical_excess_area())

Set the run-time search path to include required libraries

Is your feature request related to a problem? Please describe.
Traditionally, executables build for GFDL have set the rpath (run-time search path) to include required libraries. With this build setting, users can run the executables without having to set $LD_LIBRARY_PATH. During some updates to the build settings, this was lost from the build scripts.

Describe the solution you'd like
The best solution would be to set the rpath using native Autotools methods. @underwoo believes the following might do the trick:

"Modify the m4/ax_lib_netcdf.m4 macro to add -Wl,-rpath -Wl,/path/to/netcdf to the NETCDF_LDFLAGS and NETCDF_FCLDFLAGS variables. These can then be added to AM_LDFLAGS in the Makefile.ams."

A shorter-term solution would be to set the LD_RUN_PATH environment variable within the site env.sh files:

setenv LD_RUN_PATH "nc-config --libdir:nf-config --prefix/lib"

This has the disadvantage of requiring users to remember to set this if they aren't using the env.sh scripts and maintaining this in the env.sh files. Also, it won't quite work, as the nc-config program isn't in the PATH until the env.sh script is eval'd. This is roughly equivalent:

setenv LD_RUN_PATH $LD_LIBRARY_PATH

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Remove unneeded combine_blobs

combine_blobs was used in the MOM5-era to combine Lagrangian Blobs but is now unused. It should be removed.

Assemble available documentation

Is your question related to a problem? Please describe.
This collection of tools, FRE-NCtools, has developed over the years to support GFDL's FRE workflow. The tools have command-line usage help, but documentation detail and quality varies after that.

Please collect and organize all available documentation on the FRE-ncotols within this repository.

Need to add `echo "$output"` to test scripts

Describe the bug
When the test scripts fail, and VERBOSE is set, only a cryptic message is printed indicating where the test script failed. However, without the job's output it can be difficult if not impossible to determine the cause. This is especially true when a job in the CI fails. To resolve this, adding echo "$output" as the first command after the bats run command will cause the output to be dumped to the screen if the test fails.

To Reproduce
Run a failed test with make VERBOSE=1 check

Expected behavior
The output of the failed test should be outputted to the screen with make VERBOSE=1 check.

System Environment
All systems

Compiler flags for release 18.1

It was determined that the current baseline( shortly to be tagged the 18.1 release) does not run on AMD
hardware (specifically the DTNs). The compiler flags specified in the config specs should be modified so
the software can also run on the DTNs. Reproduction of results created by the bronx-18 (18.0) release tools
should be considered. Further work beyond this is considered in issue 99 ( #99 )

make_hgrid bad call to get_grid_great_circle_area

Describe the bug
The source make_hgrid/create_lonlat_grid.c uses an implicit declaration to a library function get_grid_great_circle_area which is in the libfrencutils.a library. Compilers when checking for c99 standards will throw a warning. As far as I can tell, the only location get_grid_great_circle_area is only defined in libfrenctools/create_xgrid.c with the following interface:

void get_grid_great_circle_area(const int *nlon, const int *nlat, const double *lon, const double *lat, double *area);

create_lonlat_grid.c calls get_grid_great_circle_area in two locations with the following calls:

get_grid_great_circle_area(&nx, &ny, x_rad, y_rad, area);
get_grid_great_circle_area(&nx, &ny, cart_x, cart_y, cart_z, area);

The second call to get_great_circle_area has an extra passed in parameter cart_z. I'm not sure what compilers do in this case, use cart_z as the area within get_grid_great_circle_area, or something else.

To Reproduce
Compile make_hgrid with c99 standard checking to get the warning. Nothing else to reproduce.

Expected behavior
No warnings with c99 standard checking. The get_grid_great_circle_area is defined via a #include <create_xgrid.h>, which would have caught this error, and not allowed the compile to complete.

System Environment
All systems. Error in the code

Add command-line audit history to all tools

Is your feature request related to a problem? Please describe.
Sooner or later, users end up with a NetCDF file produced by a FRE-nctool and they don't know how it was created. NCO tools and fregrid append this history to the history global attribute, which often comes in handy.

Describe the solution you'd like
All tools to append the command used in the output NetCDF file, such as NCO tools and fregrid.

Describe alternatives you've considered
Better notetaking :)

Additional context
Add any other context or screenshots about the feature request here.

Add Copyright years

It may be beneficial to add the Copyright years to FRE-NCTools files.
Please see this thread from @underwoo #43 (comment).
Also see the discussions in this issue #24 and in this PR #43

Show source git hash used during compilation with --version option

Is your feature request related to a problem? Please describe.
To help debug user issues, it's helpful to know exactly which source was used during the compilation. For cases where the source code is still available, one can simply check the repository. However, if the tools were installed someplace else, it can be harder to determine which source code was used. It would also be helpful to record whether the git repository used was clean or contained uncommited changes.

Describe the solution you'd like
The tools to display this provenance information using the --version option:

If compiled from a clean repository:

<command>: <package_name>(<git_hash>) <commit_date>

If compiled from a uncommited repository:

<command>: <package_name>(Uncommitted) <build_date>

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Some of this information is already captured in the automake-generated config.h file. Each tool would then need to include this file and make it available from the --version option.