Add METplus metatasks to develop

[noaa-emc]

Add METplus metatasks to develop

[KateFriedman-NOAA]

Created new branch: feature/metplus

[KateFriedman-NOAA]

Pulled config.metp into feature/metplus branch. Adjusted config.vrfy to remove METplus entries. Saw adjustment in setup_expt.py for default config and liked it so pulled it in.

Pulled in METplus changes for setup_workflow_fcstonly.py. @malloryprow would you mind pulling the setup_workflow.py changes for METplus from feature/gfsv16b into the copy in the feature/metplus branch? I saw other changes in feature/gfsv16b's setup_workflow.py script for v16 work (e.g. GLDAS) so I need your help picking out the METplus changes. Thanks!

@malloryprow What other METplus files/changes need to get brought into feature/metplus from feature/gfsv16b?

[malloryprow]

Sure, changes were made for the metatask too in setup_workflow_fcstonly.py. Going off the top of my head here, I have a nice piece of paper of the files and scripts adjusted for the metatask....jobs/rocoto/metp.sh will need pulled in...config.resources had changes in there as well. I can do the changes. I'll be at AMS next week, so I can look at it the following week if that is okay. Is there an immediate deadline coming up?

[KateFriedman-NOAA]

The week after AMS is fine, thanks! No rush, just wanted to get this started and logged in GitHub.

[malloryprow]

Updates to feature/metplus...

• Added jobs/rocoto/metp.sh.

• Updated ush/rocoto/setup_workflow.py to bring in gfsmetp metask and its tasks

• Update sorc/checkout.sh check out EMC_verif-global tag to 1.4.0

• Update config.resources for resources for gfsmetp

• Update files in env for various supported machines for gfsmetp

[malloryprow]

After inital testing some changes were made and committed to feature/metplus...

1. in feature/gfsv16b, the online archive pgb files are saved as grib2, so they have ".grib2" at the end. It looks like in feature/metplus (branched off of develop?) still has them saved as grib1 with no suffix at the end of the pbg files. This is a simple change in conf.metp to tell the code what file format to look for, just something that will be needed to be updated later if/when the files are saved with ".grib2" at the end like in version 16.

2. For the grid-to-obs verification, the prepbufr files needed for the dates of my testing were not located online on Hera, so it tried to get them from HPSS but a mismatching of variables in the mediator script between the global workflow and EMC_verif-global for the transfer queue caused it to use the default "dev_transfer", and so it failed because that doesn't exist on Hera! I have fixed this and since the mediator script resides in EMC_verif-global I cut a new tag for EMC_verif-global (1.4.1) and updated the checkout in sorc/checkout.sh .

[malloryprow]

One more commit to feature/metplus...

• Related to 1. in above comment, forgot to update in a few other places in config.metp

[malloryprow]

Single cycle free-forecast testing on Hera successful!

[malloryprow]

Updated tase for EMC_verif_global checkout to verif_global_v1.5.0. This tag is mostly things that wouldn't affect the workflow like plotting, but one thing that may be important for the workflow are some soon-to-be-old Phase 1/2 paths to some production data were replaced with their new locations on WCOSS, particularly for the observation data used for the precipitation verification, but this doesn't affect things on Hera. This is the tag that is being used in the latest branch for GFSv16 testing, feature/gfsv16b_paranetcdf.

[KateFriedman-NOAA]

Pull request #26 completed and changes committed to develop branch at 1b359db. Thanks Mallory! Closing issue now.

[KateFriedman-NOAA]

Realized we never set up a switch to turn on/off the MET+ metatask. Will add, test setup scripts, and commit.

[KateFriedman-NOAA]

Created feature/metplus2 branch to make switch changes.

[KateFriedman-NOAA]

PR #28 initiated. Requested review from Mallory before commit.

[KateFriedman-NOAA]

PR submitted. Closing ticket.

[KateFriedman-NOAA]

Deleted feature/metplus and feature/metplus2 branches.