ncar / topo Goto Github PK
View Code? Open in Web Editor NEWNCAR Global Model Topography Generation Software for Unstructured Grids
NCAR Global Model Topography Generation Software for Unstructured Grids
February 2023, NCAR. User's guide: https://github.com/NCAR/Topo/wiki/User's-Guide What's new in version 2.0.1: *Command-line execution (e.g.): ./cube_to_target --grid_descriptor_file ne30pg3.nc --intermediate_cs_name gmted2010_modis_bedmachine-ncube3000-220518.nc --output_grid ne30pg3 --smoothing_scale 100.0 --name_email_of_creator 'Peter Hjort Lauritzen, [email protected]' Note: if you get the error message No smoothed topo file specified Required argument not specified: grid_descriptor_file you are probably using Intel compiler. Please use GNU as outlined in the User's guide. *New improved source topography data (ice sheet regions) See #10 *Internal iterative Laplacian smoother with "no leak" option See #39 *Support for SCRIP and ESMF grid descriptor file formats *Seamless support for variable resolution grids (sub-grid-scale variables and smoothing seamlessly adapts to resolution changes) *CESM compliant metadata on netCDF file Please note important bug fix in tag NCAR_Topo_2_0_1 This software is partially documented in Peter H. Lauritzen, Julio T. Bacmeister, Patrick Callaghan and Mark A. Taylor, 2015: NCAR Global Model Topography Generation Software for Unstructured Grids. Geosci. Model Dev., 8, 3975-3986, doi:10.5194/gmd-8-3975-2015 and updated by Julio Bacmeister/Peter Hjort Lauritzen for CESM2.2 support (internal smoothing, ridge finding algorithm, etc.) and variable resolution (Adam Herrington, Rene Wijngaard). Overview of directories ======================= bin_to_cube: ------------ Bin high resolution elevation data (on regular lat-lon grid) onto cubed-sphere grid. Creates intermediate cubed-sphere data for cube_to_target. This is used to separate scales for SGH and SGH30. Note: The user does not need bin_to_cube step unless wanting to change source dataset and/or intermediate cubed-sphere resolution. cube_to_target: --------------- Code to process elevation data and remap to target grid. List namelist options: Execute ./cube_to_target --help Software will produce plot.sh which can be used to produce NCL plots of topography data (source plot.sh) Default interemdiate cubed-sphere data on Cheyenne is located at /glade/p/cgd/amp/pel/topo/cubedata/gmted2010_modis_bedmachine-ncube3000-220518.nc Example: See cubed_to_target/ne30pg3.sh. Execute (on Cheyenne) with: qcmd -l walltime=04:00:00 -- source ne30pg3.sh Fast regression tests: source regr_test1.sh -fast version of ne30pg3.sh but using ncube=540 intermediate cubed-sphere grid data source regr_test2.sh -low resolution variable resolution test using ncube=540 intermediate cubed-sphere grid data source regr_test2_lap.sh -same as regr_test2.sh but using Laplacian smoother source regr_test3.sh low resolution test for lat-lon grid using ncube=540 intermediate cubed-sphere grid data regression-test-data: --------------------- Data used for regression tests in cubed_to_target. See above. make_rll_grid_descriptor_file: ------------------------------ Code to make SCRIP grid descriptor files for regular latitude-longitude grids
Plot below shows the difference between smoothing everywhere and not smoothing when LANDFRAC=0 with ncube=3000 intermediate cubed-sphere grid (~3km):
and ncube=540 (~12km)
There are significant differences inland. Why?
The problem: Binning from ~1km LANDFRAC to 3km cubed-sphere picks up on isolated "inland" ocean points that are not picked up when using ncube=540 intermediate cubed-sphere grid. See this regional plots over Himalayas of LANDFRAC when using ncube=3000:
and ncube=540:
There are sporadic LANDFRAC=0 points (dark blue points) where the Laplacian smoother will not be applied in the ncube=3000 LANDFRAC but close to none in ncube=540 LANDFRAC.
@Katetc found that bin_to_cube
does not compile out of the box:
"However, we still call the bin_to_cube code, and that doesn't build on Derecho in the code I just cloned from github.com/NCAR/topo. I'm getting the error:
NCAR_topo_clone/bin_to_cube> gmake -f Makefile
unknown option: --fc Usage: nc-config [OPTION] Available values for OPTION include: --help display this help message and exit --all (plus many more):"
She later identified the problem:
"Ope, I figured it out. The Makefile in bin_to_cube points to a machine_settings.make file in the root directory and the Makefile in cube_to_target points to a machine_settings.make file in the local directory. And the later one has a ? on the equals sign that fixes all problems.
So, you may need to update the machine_settings.make file in the root directory of NCAR/topo. I'll use the one from cube_to_target in the Icesheet build. I can get everything to build in NCAR/topo now, so I should be able to do the same for the Icesheet code."
lstop_after_smoothing=.true. will not work for refined grids since rrfac is not read in ...
just checking ...
I've merged (on my fork) my latest mods with the up-to-date NCAR/Topo/topowopts. There are some issues that we should discuss:
I'd like to eliminate use_prefilter in favor of just keying off the fine_radius parameter. Code is now logically consistent with that.
I'd like to eliminate separate calls to several subroutines that depend on the value of lregional_refinement flag. I believe we should just let the codes work with rrfac=1 everywhere if refinement is not desired. The case lregional_refinement=.false. should just trigger the creation of an rrfac array that is =1 everywhere.
Before we make any major changes, creating a regression test to ensure we get the same results will be beneficial.
This issue serves as an area to discuss creating this regression test.
For low resolution grids (e.g. ne16, mpas480) source code change is needed:
replace
jmax_segments = MIN( jmax_segments, 10000 )!for low resolution grids this line needs to be uncommented (makes code slow!)
with
jmax_segments = MIN( jmax_segments,100000 )!for low resolution grids, e.g., mpasa480
in cube_to_target.F90.
Note: if jmax_segments is large then high resolution grids make the model run very slow ...!
Maybe jmax_segments could be optional argument?
I just ran topowopts with this command for a ne30pg3 grid:
./cube_to_target --grid_descriptor_file='/glade/work/aherring/grids/uniform-res/ne30np4.pg3/grids/ne30pg3_scrip_170611.nc' --intermediate_cs_name='/glade/p/cgd/amp/pel/topo/cubedata/gmted2010_modis_bedmahcine-ncube3000-220518.nc' --output_grid='ne30pg3' -r -d -m --smoothing_scale=100.0 -u "Adam Herrington, [email protected]"
After almost completing this job (it did get past writing the final file), I got this error:
writing GBXAR data 2.1727097902297268E-004 3.0458649516175928E-004
done writing GBXAR data
close file
At line 973 of file cube_to_target.F90
Fortran runtime error: Attempt to DEALLOCATE unallocated 'target_rrfac'
I suspect the target_rrfac
pointer is not being allocated in read_target_grid
when there is no rrfac on the SCRIP file.
Hi @gold2718
@PeterHjortLauritzen and I talked about merging the TopoCESM2 branch to the head of master. There have been many updates to TopoCESM2 so I think it's time to just bring it in to master.
I'm making some tweaks right now and so will commit a final version of TopoCESM2 in a couple hours, but I just wanted to put out a feeler.
should be 10.5194/gmd-8-3975-2015 (Mijeong Park, ACOM)
What I expected: Since the User's Guide indicates that Intel compilers are an option, I would expect the cube_to_target
executable to work with minimal intervention.
What I saw: Runs to generate bnd_topo files for MPAS-A grids fail unless I use GNU compiler.
What I did on Derecho:
cd
into cube_to_targetmodule purge && module load intel-classic cray-mpich netcdf-mpi
cube_to_target
executable: make
./cube_to_target --grid_descriptor_file "/glade/campaign/cesm/cesmdata/inputdata/share/meshes/mpasa120z32_ESMFmesh_cdf5_c20210120.nc" --intermediate_cs_name "${PATH_TO}/gmted2010_modis_bedmachine-ncube3000-220518.nc" --output_grid mpasa120 --smoothing_scale "100.0" --name_email_of_creator "${YOUR_VALUE_HERE}"
Error text:
# other output...
Writing topo file to
/glade/derecho/scratch/gdicker/2024Apr26_NCARTopoTest/Topo/cube_to_target/mpasa
120_gmted2010_modis_bedmachine_nc3000_Laplace0100_noleak_20240429.nc
# ... multiple blank lines here ...
anticipated number of overlaps jall_anticipated= -198767296
Compute overlap weights:
more weights than anticipated
increase jall
This seems to be due to the logic around line 597 of cube_to_target.F90. GNU builds this seems to be handled appropriately, but my runs with Intel suggest (due to negative jall_anticipated) that the integer overflow check isn't sufficient.
I think we should have topography on both the phsygrid and the GLL grid in topography files for physgrid configurations e.g., ne30pg3
There is no way to tell that a grid is lat-lon from ESMF grid descriptor format. Probably need option in code to tell it is lat-lon so that the final topo file is 2D.
Sample ESMF lat-lon file located here:
/glade/p/cesmdata/inputdata/share/meshes/fv0.9x1.25_141008_ESMFmesh.nc
I have been generating several MPAS topo files that need to be scientifically validated:
Command used for 60-3km grid:
./cube_to_target --grid_descriptor_file='/glade/work/skamaroc/MPAS_meshes/CESM_MPAS/60-3km/West_Pacific/mpas_x20.835586.wpac.1_esmf.nc' --intermediate_cs_name='/glade/p/cgd/amp/pel/topo/cubedata/gmted2010_modis_bedmahcine-ncube3000-220518.nc' --output_grid='mpas_x20.835586.wpac' -u 'Peter Hjort Lauritzen, pel@ucar' -q 'output/' --smoothing_scale=50.0 -r -y 20 -v -m
-m = smoothing over ocean points
-v = rrfac manipulation
-y = refinement factor
Command used for 60-10km grid:
./cube_to_target --grid_descriptor_file='/glade/work/skamaroc/MPAS_meshes/CESM_MPAS/60-10km/mpas_x6.999426.wpac.1_esmf.nc' --intermediate_cs_name='/glade/p/cgd/amp/pel/topo/cubedata/gmted2010_modis_bedmahcine-ncube3000-220518.nc' --output_grid='mpas_x6.999426.wpac' -u 'Peter Hjort Lauritzen, pel@ucar' -q 'output/' --smoothing_scale=50.0 -r -y 6 -v -m
Topo files here:
/glade/p/cgd/amp/pel/topo/grids/mpas-60-3km-WestPacific/mpas_x20.835586.wpac_gmted2010_bedmachine_nc3000_Laplace0050_20220520.nc
/glade/p/cgd/amp/pel/topo/grids/mpas-60-10km-WestPacific/mpas_x6.999426.wpac_gmted2010_bedmachine_nc3000_Laplace0050_20220520.nc
@PeterHjortLauritzen @JulioTBacmeister
I've been using your v1 version of the code to generate topography for GEOS for many years. I've been using an external smoothing (the smoothing utility in UFS_Utils, part of UFS). However, I was checking out the v2 version of the code as I was trying to see how to generate these ridge files that you have produced for us in the past and I've noticed the v2 version of the code is a massive change.
From what I can tell, you can't turn off the smoothing? Is this correct? It was not clear. Is the option to just produce unsmoothed topography available in the version 2 of these codes?
For some reason the command below doesn't result in ldevelopment_diags=.TRUE.
./cube_to_target --grid_descriptor_file='../../regression-test-data/ne30pg3.nc' --intermediate_cs_name='../../regression-test-data/gmted2010_bedmachine-ncube0540.nc' --output_grid=$ogrid --coarse_radius=$Co --fine_radius=$Fi -r -u '[email protected]' -q 'output/' -z
The additional output isn't written.
If I put the '-z' option right after the '-r' option I get the error:
"Required argument not specified: name_email_of_creator"
New topography dataset. It is the old GMTED2010 dataset merged with the GrIS and AIS high resolution bedmachine data, for more accurate representation of ice sheets. Here is what a ne30pg3 topofile looks like using this new dataset minus the old dataset:
Among other changes, the ice sheets have steeper margins, which seems to produce larger subgrid heights. This is interesting b/c if you recall, SGH30 was artificially increased over Greenland in the old dataset to bring the surface winds into better agreement with observations. @whlipscomb @lofverstrom
Once #9 is merged, I can modify the Makefile to point to the new 3km intermediate cubed-sphere dataset.
topo file names should have some indication whether no leak is on/off
Decide on naming convention for PHIS on GLL grid when using physics grid option. This will require changes in dyn_comp.F90 on cam_development (i.e. coordination needed).
See also issue ESCOMP/CAM#552
@adamrher suggested that "-v" should just turn on smoothing of rrfac as well as integer operations on rrfac (needed to spectral-element grids) but that even for MPAS it might be good to bound rrfac between 1 and rrfac_max.
When a refinement factor on the c540/CONUS SCRIP file is regridded to the nc3000 grid in cube_to_target.F90, isolated stripes of ZERO are produced. The field on the SCRIP is fine, i.e., minimum value=1.0.
From standard output:
Final output file :: ./output/°uìÍxHî
jiî@ üp üjiî
ü°
(jiî<<Hî
c;HîjiîPj?Hî``xz2ì
jiîj?Hî
xz2ìjiî
x01ì( üàmiî
jiî\JHî
üji
ü
ü
ümiîP1k÷ë1ì
jiî``ýãì`zsJìXt
@JulioTBacmeister is working to enhance the topography software to provide consistent sub-grid representation of ridges for variable resolution grids. The ridge information is used by the anisotropic orographic gravity wave drag scheme in CAM6, and it is important for this to be done correctly in variable resolution configurations as to not "double count" owing to both resolved and parameterized orographic gravity waves. This is a high priority task due to the need to provide consistent, correct topography boundary conditions for the roll out of our variable resolution grids in the next cesm release (cesm2.2). I am opening up this thread to discuss progress and/or difficulties in completing this task.
[transferring text from issue https://github.com/NCAR/amwg_dev/issues/116 to this repo as it is relevant; the plots in this post are using GMTED2010 elevation data only (no bedmachine)]
We have produced a new topography file where we use a Laplacian smoother (instead of the default distance weighted smoother) and turn off smoothing where the land fraction is 0 (i.e. no smoothing over ocean points; partial ocean points are still smoothed). We are curious if not smoothing over ocean points has an impact in coupled model setup since the temperatures passed to the ocean, where previously the topography was "leaking into the ocean", could be biased cold (assuming temperature decreasing with height on average).
The Figure below shows energy spectra for different topographies:
A couple of observations regarding the energy spectra:
The energy spectra is the same when using a 3km intermediate cubed-sphere field (ncube3000, orange line) compared to using a 16km intermediate cubed-sphere grid (ncube540, red line) when computing energy spectra on 1 degree finite-volume grid. That means, we can assess smoothing using the ncube540 intermediate cubed-sphere grid which is much more computationally efficient to run the smoother on compared to the ncube3000 intermediate cubed-sphere grid.
Below show difference between smoothing everywhere but using the ncube=3000 grid versus ncube =540 grid:
Basically this plot verifies that smoothing on ncube=3000 and ncube=540 using the same physical smoothing scale leads to almost the same result (as one would expect). For comparison distance weighted smoother minus Laplacian smoother (PHIS) below:
[very few differences!]
The next two plots show Greenland area and South America topographies (all mapped to fv 1 degree grid):
The cross section plots (lower right) are though 30S and 65N.
The following observations are made:
Some mathematical details on the smoothing operator:
When interpolating to grids that with 3 km grid spacing, the topo tool produces a height field which is significantly lower resolution than what is expected. I belive this is caused by the double interpolation from the ~1 km GMTED2010 data to the 3 km cube sphere then to the target grid. Then, with ridge analysis the height field is smoothed even further.
Below are two plots of the height field over conus. The first, is the x5.6488066.grid.nc (15-3 km variable resolution mesh) with the 3 km refinement centered over CONUS with the height field that was created with this tool. (I believe the height field outside of the continent lines is due to a lack of landmask).
The second is GMTED2010 dataset interpolated to the x1.65536002.grid.nc (3 km quasi-uniform grid) gird using the MPAS-Model`s init_atmosphere core.
These files can be found on glade at: /glade/scratch/mcurry/cam-mpas-topo-compare
Is there a way interpolate GMTED2010 to a higher resolution cube sphere?
lzero_out_ocean_point_phis=.true. algorithm needs LANDFRAC
Re-introduce LANDFRAC on intermediate cubed-sphere file, in cubed_to_target.F90 but don't write LANDFRAC to final topo file.
Source topo dataset for Mars
/glade/p/cgd/amp/pel/topo/rawdata/mars-rawdata.nc
is missing data North/South of 88.2N/S which trips error checks in bin_to_cube.
ncdump below:
lat = -88.1949168841954, -88.1871353846802, -88.179353885165, ...
./cube_to_target --smoothing_over_ocean --grid_descriptor_file='/glade/p/cesmdata/cseg/inputdata/share/meshes/ne16np4_ESMFmesh_cdf5_c20211018.nc' --intermediate_cs_name='/glade/p/cgd/amp/pel/topo/cubedata/mars-ncube3000.nc' --output_grid='mars_ne16np4' --smoothing_scale=200.0 -u 'Peter Hjort Lauritzen, [email protected]' -q 'output/'
works but when moving --smoothing_over_ocean to the end
./cube_to_target --grid_descriptor_file='/glade/p/cesmdata/cseg/inputdata/share/meshes/ne16np4_ESMFmesh_cdf5_c20211018.nc' --intermediate_cs_name='/glade/p/cgd/amp/pel/topo/cubedata/mars-ncube3000.nc' --output_grid='mars_ne16np4' --smoothing_scale=200.0 -u 'Peter Hjort Lauritzen, [email protected]' -q 'output/' --smoothing_over_ocean
the model just hangs!
An issue to discuss updating the software to match the smoothing of MPAS.
Remove landfrac from topo software as it is not read in by CAM. It is a different landfrac than used in the model, so it's really just a point of confusion and should be removed.
The SCRIP files use the following naming convections (example from MPAS):
netcdf mpas_scrip_60-3_andes {
dimensions:
grid_size = 835586 ;
grid_corners = 7 ;
grid_rank = 1 ;
variables:
double grid_area(grid_size) ;
grid_area:units = "radians^2" ;
grid_area:long_name = "area weights" ;
double rrfac(grid_size) ;
rrfac:units = "dimensionless" ;
rrfac:long_name = "normalized dc" ;
double grid_center_lat(grid_size) ;
grid_center_lat:units = "radians" ;
double grid_center_lon(grid_size) ;
grid_center_lon:units = "radians" ;
double grid_corner_lon(grid_size, grid_corners) ;
grid_corner_lon:units = "radians" ;
double grid_corner_lat(grid_size, grid_corners) ;
grid_corner_lat:units = "radians" ;
int grid_imask(grid_size) ;
int grid_dims(grid_rank) ;
The ESMF version is:
netcdf mpas_esmf_60-3_andes {
dimensions:
nodeCount = 1671168 ;
elementCount = 835586 ;
maxNodePElement = 7 ;
coordDim = 2 ;
variables:
double nodeCoords(nodeCount, coordDim) ;
nodeCoords:units = "degrees" ;
int elementConn(elementCount, maxNodePElement) ;
elementConn:long_name = "Node Indices that define the element connectivity" ;
elementConn:_FillValue = -1 ;
byte numElementConn(elementCount) ;
numElementConn:long_name = "Number of nodes per element" ;
double centerCoords(elementCount, coordDim) ;
centerCoords:units = "degrees" ;
double elementArea(elementCount) ;
elementArea:units = "radians^2" ;
elementArea:long_name = "area weights" ;
double elementRefinementRatio(elementCount) ;
elementRefinementRatio:units = "dimensionless" ;
elementRefinementRatio:long_name = "normalized dc" ;
int elementMask(elementCount) ;
Code for manipulating rr_fac is tailored for SE.
MPAS does not need extra smoothing of rr_fac.
Hence make manipulations optional.
/glade/p/cesmdata/inputdata/share/meshes/ne0ARCTICGRISne30x8_ESMFmesh_c20200730.nc
does not have refinement factor variable!
Note: in ESMF format we have unofficially decided that the refinement variable is named elementRefinementRatio
(this is what is used on the MPAS regional refinement grid descriptor files)
Note that the SCRIP version in the CAM coords directory does have refinement factor variable (rrfac):
/glade/p/cesmdata/inputdata/atm/cam/coords/ne0ARCTICGRISne30x8_scrip_c191209.nc
I noticed a bug in my implementation of the regional refinement enhancements in branch TopoCESM2. In particular, the refinement factor should have the limiter applied to it before being called by the topography smoother. The refinement factor is instead going through the limiter step after the topography smoother is called. This means that the smoother isn't calling the correct refinement factor, which is a bug.
This bug has been fixed in a forked-off branch TopoCESM2.2 in my git repo:
https://github.com/adamrher/Topo/tree/TopoCESM2.2
I also updated the experiment_settings.make file by grouping the three (3) supported variable resolution grids (ARCTIC, ARCTICGRIS & CONUS) together under the sub-header "functionally supported vr-grids." I also had switch the ARCTIC & CONUS settings to default to including anisotropic gravity wave fields, as req'd by CAM6 physics.
The code currently turns on regional refinement if there is an rrfac in the SCRIP file, but some uniform resolution SCRIP files may contain rrfac, causing the software to fail for those grids.
Propose setting rrfac_max = 1 for such a uniform resolution grid, to cause the program to ignore rrfac in the SCRIP file and otherwise carry on as if it were a uniform resolution grid. The code currently will error out if you set this to 1 and rrfac is in the SCRIP file.
Here's a rotated ne30 SCRIP file that contains rrfac, for testing:
/glade/scratch/patc/AMP_REPO/ne0np4.SAMwrf01.ne30x1/grids/SAMwrf01_ne30x1_np4_SCRIP.nc
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.