FREgrid with time-averaged variables

[huffj]

FREgrid with time-averaged variables

[huffj]

I had posted this here around a month ago and was curious if anyone else has encountered this issue, or if there are any known ways to correct the issue.

Thanks

[ceblanton]

Hello, sorry for the delay. The type mismatch between "time" and "time_bounds" is a bit unusual and I agree causes fregrid trouble, though I'm not sure if it's uncompliant or just unusual. We will report back next week with more details.
Thank you,
Chris

[ceblanton]

Hello again @huffj, thank you for pointers to the example files. Could you also point us to the rest of the prerequisite files to see the problem directly- the grid mosaic files and the other input tiles, please?

[huffj]

Hello @ceblanton , thanks for following up.

The fregrid files used are located here:
--input_mosaic: /work/James.Huff/remap/grid/C3072_mosaic.nc
--remap_file: /work/James.Huff/remap/remap/remap_file_C3072_360x180.nc

Before doing the fregrid, the output files from the model that needed to be converted is here:
/ptmp/James.Huff/data/20160801.00Z.C3072.L79z13/history/2016080100/atmos_8xdaily_ave.tile1.original.nc

After using the ncap2 command this is the same file with the variable type fixed:
/ptmp/James.Huff/data/20160801.00Z.C3072.L79z13/history/2016080100/atmos_8xdaily_ave.tile1.nc

If you need anything else let me know.

[ceblanton]

Hi again, I think I have a solution, but it may not be completely convincing yet. I ran your input tiled file through a CF conventions checker (https://pumatest.nerc.ac.uk/cgi-bin/cf-checker.pl) and it complained that time_bnds should not have missing_value, _FillValue, or units attributes. When I removed those 3 attributes, fregrid ran as expected. (The units attribute didn't matter, though)

The checker reported a violation of CF section 7.1 (https://cfconventions.org/Data/cf-conventions/cf-conventions-1.8/cf-conventions.html#cell-boundaries) but after reading the text it wasn't exactly compelling. Still, I believe the CF checker is correct, and that those extra time_bnds attributes are uncompliant.

Those 3 attributes can be removed with ncatted, but that is not a satisfactory long-term solution: ncatted -a _FillValue,time_bnds,d,, -a missing_value,time_bnds,d,, -a units,time_bnds,d,,

I think the right course of action is to fix the FMS code (diag_manager, etc) to output CF compliant history files, which will then be happily processed by fregrid.

[huffj]

Hello @ceblanton , Ya that's interesting that that could cause the issue. It doesn't happen on every run. Of the 10+ model runs I was processing it only happened on two of them. The ncatted solution seems similar to the ncap2 solution I was using as a temporary fix. For the long term fix, is there anything else on my end that needs to be done, or were you going to put a fix into the FMS code?

[ceblanton]

I would say the ncatted fix is a bit more agreeable than the ncap2 one in that it's only modifying (deleting) attributes, not data, and it makes the input files CF-compliant where they weren't before.

It seems to be a combination issue in that the problem arises when time and time_bnds are different types AND when time_bnds contains those problematic uncompliant attributes. For the 8/10 runs that work, is the situation any different? (Are the time and time_bnds the same type?) It's curious why you are seeing this and presumably not many others. Can you point us to the similar output that fregrid accepts? There be a third factor at play maybe.

Once we convince ourselves a little more of the problem we should file an bug report on the FMS github page (I'm not the best qualified to make the fix itself!) Thanks!

[huffj]

Hello @ceblanton Ya I would agree it could be the type mismatch between 'time' and 'time_bnds'. When I first noticed it, my first thought was the type mismatch between 'nv' and 'time_bnds'. For the runs that fregrid failed, 'time' and 'nv' are both type 'double', while 'time_bnds' is type 'float'. For the successful runs, 'time', 'nv', and 'time_bnds' are all type 'float'. It isn't clear to me why the type changed between the runs.

For example files:
Type mismatch: /ptmp/James.Huff/data/20160801.00Z.C3072.L79z13/history/2016080100/atmos_8xdaily_ave.tile1.original.nc
Fregrid successful: /ptmp/James.Huff/data/20160801.00Z.C3072.L79z16/history/2016080100/atmos_8xdaily_ave.tile1.nc

It appears that 'grid_xt' and 'grid_yt' are also type 'double' on the failed fregrid commands and type 'float' when successful after looking into it more. Not sure if that would affect anything, but just another anomaly.

[ceblanton]

Hello again, I've been in touch with one of the FMS developers (@uramirez8707) who has a fix for both issues: the type mismatch between time and time_bnds, and the noncompliant time_bnds attributes. However, this new code will likely not get into the FMS code release until the 2021.02 release (a month or two-ish). Until then, please continue to use the ncatted (or ncap2) workarounds.

You can follow the FMS update on this tracking issue (NOAA-GFDL/FMS#668). After it is released, you can update your FMS model code and enjoy more seamless error-free fregrid'ing. Thanks for letting us know of this problem!

[huffj]

Hello. Thanks for looking into this for me and getting a code fix implemented into the new FMS code.

[ceblanton]

I'm going to close this now. Please stay tuned for the 2021.02 FMS release which should contain this fix.