hafs-community / hafs Goto Github PK
View Code? Open in Web Editor NEWHurricane Analysis and Forecast System
License: Other
Hurricane Analysis and Forecast System
License: Other
For the HAFS application/workflow develop branch, switch to point to the develop branch of the authoritative ufs-weather-model directly for the sorc/hafs_forecast.fd submodule.
Currently, the support/HAFS branch is identical to an early version (11/03/2021) of ufs-weather-model develop branch. When we plan our next round of update of the hafs_forecast.fd submodule inside HAFS, we can just directly point to the ufs-weather-model develop branch.
Note: The HAFS related developments will still be happening in the hafs-community fork of ufs-weather-model (e.g., the feature/hafs_nesting branch for the moving nesting development).
Sync HAFS submodules with their corresponding authoritative branches as of 11/02/2021.
The HAFSv0.3 phase2 configuration in the feature/hafsv0.3_phase2 branch contains the latest HAFS moving nesting, coupling, vortex initialization, data assimilation capabilities, and it points to the latest develop branch (as of 04/12/2022) of ufs-weather-model. Besides, the hafs_utils.fd and hafs_gsi.fd were also synced with their corresponding develop/master branches recently. With that, it would be beneficial to merge this feature branch back to HAFS develop branch.
Merge the feature/hafsv0.3_phase2 branch back to HAFS develop branch.
Directly reference any related issues or PRs in this or other repositories, and describe how they are related. Examples:
Add a script under rocoto to check RT output
Currently, the regional ESG grid is generated out of the HAFS workflow. So, we need to add the capability of generating regional ESG grid in the atm_prep job/task.
Provide a clear and concise description of what the bug is.
Also give a description of what behavior you expected to happen.
The atm_prep task will fail due to the typo if condition (introduced by this commit 7e56220):
if [[ -e ./hafs_sfc_climo_gen.x ]]; then
cp -p $SFCCLIMOEXEC ./hafs_sfc_climo_gen.x
fi
which should be
if [[ ! -e ./hafs_sfc_climo_gen.x ]]; then
cp -p $SFCCLIMOEXEC ./hafs_sfc_climo_gen.x
fi
The atm_prep task will fail for any HAFS application regression tests.
Add any other context about the problem here.
Directly reference any issues or PRs in this or other repositories that this is related to, and describe how they are related. Examples:
Sync HAFS submodules/subcomponents as of 05/12/2021
Add HAFS application/workflow level data assimilation related regression tests in the standard regression tests.
How should the new feature be added? If you have thoughts on the implementation strategy, please share them here.
Add one or two regression tests with the 3DEnVar with GDAS ensembles and/or HAFS ensembles.
Are you (or a colleague) working on adding this feature? If so, please assign yourself (or them) to this issue.
Directly reference any issues or PRs in this or other repositories that this is related to, and describe how they are related. Example:
Enable running HAFS on WCOSS2
For submodules of hafs_utils.fd (UFS_UTILS), hafs_gsi.fd, hafs_forecast.fd, and hafs_post.fd, wait for the porting from their own authoritative branches. For other HAFS submodules (hafs_tools.fd, hafs_hycom_utils.fd, hafs_graphics.fd, hafs_vortextracker.fd) as well as the application/workflow scripting system, will need to port them from the HAFS side.
Fixed by PR #136
The global offset predictor coefficient does not update when online_satbias config is set to yes. This is because in file parm/analysis/gsi/gsiparm.anl.tmp: upd_pred(1)
equals 0
when it should equal _UPD_PRED_
.
This bug can be seen by examining satbias_out files from model runs in which online_satbias=yes, and seeing that the first predictor coefficient remains constant.
See the following commit for additional context on the purpose of the online_satbias option and the significance of the upd_pred variables:
b6172ee
Incorporate wave-coupling in the HAFS application/workflow by establishing the HAFS FV3-HYCOM-WW3 coupled system.
Sync HAFS submodules (hafs_forecast.fd, hafs_gsi.fd, hafs_utils.fd) with their corresponding authoritative branches.
Provide a clear and concise description of the requested feature/capability.
Upgrade to the new UGWP scheme (which can also reproduce the existing old ugwp scheme when setting namelist option properly).
How should the new feature/capability be added? If you have thoughts on the implementation strategy, please share them here.
Workflow script changes are needed to generate GSL orography files.
Developments are needed to support the HAFS moving-nesting configuration properly.
Do you (or a colleague) plan to work on adding this feature?
Directly reference any related issues or PRs in this or other repositories, and describe how they are related. Examples:
Provide a clear and concise description of the requested feature/capability.
How should the new feature/capability be added? If you have thoughts on the implementation strategy, please share them here.
The upoff namelist option in FV3dycore has been added during a working session. Workflow level config option will need to be added.
Do you (or a colleague) plan to work on adding this feature?
Directly reference any related issues or PRs in this or other repositories, and describe how they are related. Examples:
Add the fms2_io_nml namelist section with netcdf_default_format of netcdf4 in input*.nml
&fms2_io_nml
netcdf_default_format="netcdf4"
/
Otherwise, the default was the NetCDF "64-bit offset" version, which has issues to support large files.
Note: This information is from the RRFS application team (Eric Rogers) with the suggestion from @bensonr.
Add the above-mentioned namelist section in the input*.nml files for the forecast job (ufs-weather-model).
Directly reference any related issues or PRs in this or other repositories, and describe how they are related. Examples:
Switch to use /bin/sh instead of /bin/ksh for Shell based scripts under jobs, scripts, and ush.
Please branch off the feautre/hafs_ensda branch for this work.
Running Thompson microphysics in an automative way using the HAFS workflow.
G2 (GRIB2 encoder/decoder) from hpc-stack is not reading grib2 files. Specifically, the routine getgb2 from G2, signaling that there is an error (code 96) in reading GRIB2 files. This is happening on all platforms(Jet/Hera/Orion/Wcoss-Cray).
I have an example(on Jet) that G2(not from hpc-stack, g2/v3.1.0) successfully decoding a GRIB2 file:
/lfs1/HFIP/hwrfv3/Biju.Thomas/Test_G2_getgb2/G2_getgb2
However, when I use G2(from hpc-stack, g2/3.4.1 ) to read the same file, it throws an error(error code 96).
/lfs1/HFIP/hwrfv3/Biju.Thomas/Test_G2_getgb2/G2_getgb2_hpc-stack
Provide a clear and concise description of the requested feature/capability.
Sync HAFS submodules (hafs_forecast.fd, hafs_post.fd, hafs_utils.fd) with their latest authoritative branches as of 202201.
How should the new feature/capability be added? If you have thoughts on the implementation strategy, please share them here.
Directly reference any related issues or PRs in this or other repositories, and describe how they are related. Examples:
"atm_post" fails occasionally on Orion and Hera with the message:
**** FATAL ERROR: rd_grib2_msg_seq_file, missing end section ('7777') ****"
Normally this error is not reproducible, rewinding the relevant tasks will succeed the "atm_post".
It appears that the error occurs when the grib2 records were written, probably due to a memory issue and grib2 file is incomplete. Once the bad grib2 file is created, the error message can be duplicated by wgrib2 (grib2file)
.
In a grib file, there is a header that provides information about the length of the record, then the record, and a "7777" signals the end of the record. The error message "missing end section ('7777)" probably indicates that the grib2 file is incomplete.
On Hera, if you run the tasks with the "bigmem" partition (large memory available), such an error is not encountered so far.
During the testing of the inline post capability in HAFS, we updated the HAFS post control files, which will need to be included in the HAFS workflow.
Due to two bugs #63 in the g2 library(prior to the latest 3.4.3 version), hafs subcomponents (hycom_utils and vortextracker) fail with g2 from hpc-stack on certain platforms. Currently, the workaround is, use g2 from HWRF build in the vortextracker component, and point to the specific g2/3.4.1 version for hycom_utils build. The two bugs in g2 were fixed in the 3.4.3 release. Accordingly, the HAFS script and modulefile files were updated to point g2/3.4.2 in hpc-stack in the PR (#77)
Enable using the restart_interval config option to write out more FV3ATM restart files
forecast.restart_interval='"0.025 3. 6. 9."'
Provide a clear and concise description of the requested feature/capability.
Sync HAFS submodules (hafs_forecast.fd, hafs_post.fd, hafs_utils.fd, hafs_gsi.fd, etc.) with their latest authoritative branches as of 202210.
How should the new feature/capability be added? If you have thoughts on the implementation strategy, please share them here.
Sync HAFS submodules and conduct HAFS application level regression tests.
Do you (or a colleague) plan to work on adding this feature?
Directly reference any related issues or PRs in this or other repositories, and describe how they are related. Examples:
Import HAFS vortextracker component including source code and build system.
Update README.md as well as the issue and pull request templates.
Discussed with code managers on making some further refinements for the issue and pull request templates.
Jobs keep failing on Orion when using GDAS ensemble in 3DEnVar. The error messages occur in random places due to out of memory related info.
If someone unfamiliar with the bug follows these steps:
Potentially relevant details could include:
-It is likely due to the recent changes in the NOAA RDHPCS system.
slurmstepd: error: Detected 1 oom-kill event(s) in step 3495064.0 cgroup. Some of your processes may have been killed by the cgroup out-of-memory handler.
srun: error: Orion-01-01: task 4: Out Of Memory
srun: Terminating job step 3495064.0
slurmstepd: error: *** STEP 3495064.0 ON orion-01-01 CANCELLED AT 2021-10-27T19:35:36 ***
srun: error: Orion-18-69: task 289: Bus error
srun: Terminating job step 3495450.0
slurmstepd: error: *** STEP 3495450.0 ON orion-01-04 CANCELLED AT 2021-10-27T20:22:22 ***
Bin Liu helped resolve the issue by changing the rocoto/sites/orion.ent file
from
into:
--partition=orion --exclusive">Provide a clear and concise description of the requested feature/capability.
Convert modulefiles to .lua format on WCOSS2, hera, jet, orion, including for both HAFS application level and related submodule components (e.g., hafs_graphics.fd, hafs_tracker.fd).
How should the new feature/capability be added? If you have thoughts on the implementation strategy, please share them here.
See examples from ufs-community/ufs-weather-model/pull/1399
Do you (or a colleague) plan to work on adding this feature?
Directly reference any related issues or PRs in this or other repositories, and describe how they are related. Examples:
Issue:
On WCOSS, the 2020 tcvitals file under operational location:
/gpfs/dell1/nco/ops/com/gfs/prod/syndat
is incomplete. This will cause issues to run those storms on wcoss_cray and wcoss_dell_p3.
Solution:
Fixes in ush/tcutil_generate_vitals.py and ush/tcutil_multistorm_sort.py have been made and tested on wcoss_cray, so that they will use the SYNDAThafs environment variable when exists/defined by the HAFS workflow (for example, in parm/system.conf).
Sync HAFS submodules with their latest authoritative branches as of 202107.
Sync HAFS submodules with their corresponding authoritative branches as of 09/09/2021.
The HAFSv0.2 configuration has been finalized, and the HFIP HAFSV0.2 series of real-time experiments are running on Jet and other platforms. So, it's time to merge hafsv0.2 configuration back to the HAFS develop branch.
I'm trying to
git clone --recursive https://github.com/hafs-community/HAFS
But I ran into following error, I wonder if someone could give me access to this repo, if this is the problem, thanks:
Cloning into '/work/noaa/marine/lliu/runs/deb/HAFS/sorc/hafs_gsi.fd/fix'...
ssh: Could not resolve hostname gerrit: Name or service not known
fatal: Could not read from remote repository.
Updating satinfo file to turn off metop-A IASI channels
Currently, in HAFS, fhcyc is set as 24 in input.nml. Propose to change it to 0, since HAFS focuses on hurricane weather forecasting.
Change fhcyc in input.nml from 24 to 0.
HAFS FV3ATM-HYCOM coupling forecast fails when there are sea ice grid points in the regional HAFS model domain. In the current ufs-weather-model forecast, when cplflx is set .true., it assumes there will be an active sea-ice model component, which is the case for the S2S application (FV3ATM-MOM6-CICE6 coupling). But, HAFS currently only has HYCOM coupling without a sea-ice model component.
Currently, a temporary fix (workaround) has been committed in the support/HAFS branch of https://github.com/hafs-community/ccpp-physics to deal with sea ice points properly in HAFS FV3-HYCOM coupling without the CICE6 component, following the discussions/suggestions among @junwang-noaa , @SMoorthi-emc, @ChunxiZhang-NOAA @danrosen25, @BinLiu-NOAA, etal. Detailed source code changes can be seen in this commit.
However, a better solution would be to separate the cplflx namelist option into cplocn and cplice (like the existing cplwav) options, and use these namelist options to control the related ocean and sea-ice coupling situations. This probably will need a unified/coordinated solution among all the related coupled UFS applications (e.g., S2S, HAFS).
Add support to run global uniform or stretched configuration without a nest. Also the tile 6 can be configured to focus a TC basin, a specific storm, or any user specified lat-lon location.
Based on @TimMarchok-NOAA's recent research on the GFDL tracker, he made a suggestion to run TC tracker with the two thickness parameters turned off. Specifically, in the tracker namelist, change "y" to "n" for the following parameters:
user_wants_to_track_thick500850 = "n",
user_wants_to_track_thick200850 = "n",
The differences (w/ and w/o the thickness parameters) are small except for the TC locations at the initial time.
This issue is created based on an email from @ZhanZhang-NOAA.
Provide a clear and concise description of the requested feature/capability.
Add HAFS NHC ASCII products like:
How should the new feature/capability be added? If you have thoughts on the implementation strategy, please share them here.
Follow HMON's method to generate these files based on the atcfunix track file.
Do you (or a colleague) plan to work on adding this feature?
@LinZhu-NOAA will explore to add this capability in HAFS application.
Directly reference any related issues or PRs in this or other repositories, and describe how they are related. Examples:
As we now have the HAFSv0.3 configurations finalized for 2022 hurricane season real-time experiments, we would like to bring the HAFSv0.3 series developments back to HAFS develop branch.
Merge the feature/hafs.v0.3.0 branch back to the develop branch.
Directly reference any related issues or PRs in this or other repositories, and describe how they are related. Examples:
Sync to use the same field_table as in ufs-weather-model HAFS regression tests:
sorc/hafs_forecast.fd/tests/parm/field_table/field_table_hafs
Copy or link sorc/hafs_forecast.fd/tests/parm/field_table/field_table_hafs into the corresponding parm/forecast/[regional|globnest]/field_table files.
Do you (or a colleague) plan to work on adding this feature?
Provide a clear and concise description of the requested feature/capability.
As WCOSS2 replaced WCOSS Cray and WCOSS Dell_p3, need to remove the HAFS application support for WCOSS Cray and WCOSS Dell_p3.
How should the new feature/capability be added? If you have thoughts on the implementation strategy, please share them here.
Update/remove the related module files and scripts to disable the support for WCOSS Cray and Dell_p3 (including for the related submodules).
Do you (or a colleague) plan to work on adding this feature?
Directly reference any related issues or PRs in this or other repositories, and describe how they are related. Examples:
Currently, all HAFS related changes/developments for UPP have been merged back to UPP develop after this PR (NOAA-EMC/UPP/pull/418). So HAFS can switch to point UPP develop branch directly.
When we plan our next round of update of the hafs_post.fd submodule inside HAFS, we can just directly point to the UPP develop branch. Beside, UPP is also becoming a submodule of ufs-weather-model/FV3/upp (see NOAA-EMC/fv3atm/pull/421), so we can also consider to just symbolically link upp from sorc/hafs_forecast.fd/FV3/upp into sorc/hafs_post.fd.
Explore using the hybens_info file to specify the vertical profile of horizontal and vertical localization length scales for HAFS EnVar and EnKF analysis.
Check and compare HAFS grib2 output fields with those from HWRF as well as those from GFS and FV3CAM. Since HAFS is FV3 based, we may or may not be able to output every field as from HWRF. Also, we may try to follow GFS for those fields common to GFS and HAFS.
Currently, HAFS does not have satellite post products yet. Add satellite post capability and products in HAFS application workflow.
When choosing the disk archive option (e.g., on Orion), write out a done file after the tar file is successfully created. This is useful for easily detecting and transferring the tar files across platforms (saying from Orion to Niagara, and then to HPSS tape).
Turn on the pressure tendency diagnostic, which is useful to monitor the model instability.
Add print_diff_pgr = .true. to the &gfs_physics_nml section of the UFSATM namelist input.nml.
The inline post capability is already available in ufs-weather-model's (hafs_forecast.fd) write grid component. We need add the inline post option in the HAFS workflow as well.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.