usepa/cmaq

Code for U.S. EPA’s Community Multiscale Air Quality Model (CMAQ) which helps in conducting air quality model simulations

CMAQv5.4

US EPA Community Multiscale Air Quality Model (CMAQ) Website: https://www.epa.gov/cmaq

CMAQ is an active open-source development project of the U.S. EPA's Office of Research and Development that consists of a suite of programs for conducting air quality model simulations. CMAQ is supported by the CMAS Center: http://www.cmascenter.org

CMAQ combines current knowledge in atmospheric science and air quality modeling with multi-processor computing techniques in an open-source framework to deliver fast, technically sound estimates of ozone, particulates, toxics, and acid deposition.

CMAQ version 5.4 Overview:

• Frequently asked questions for upgrading to the latest CMAQ version) - Updated for v5.4 release.
• CMAQv5.4 Release Notes - Release Notes are available on the CMAQ GitHub wiki.
• Tutorials for setting up and running CMAQ - Tutorials on running CMAQ, CMAQ-ISAM, CMAQ-DDM, and WRF-CMAQ test cases.

New features in CMAQ version 5.4 include:

• Updated chemistry for ozone (O3) and particulate matter (PM) formation from global-to-local scales
• Introduction of the Community Regional Atmospheric Chemistry Multiphase Model (CRACMM)
• Biogenic emissions algorithm options have been expanded
• Revised algorithms for modeling the dry deposition of particles from the atmosphere (M3DRY and STAGE updates)
• Streamlined building of the coupled WRF-CMAQ system compatible with WRFv4.4+
• Improved efficiency, accuracy, and user experience for CMAQ instrumented model extensions CMAQ-DDM-3D and CMAQ-ISAM
• Expansion of emissions diagnostic output features
• Introduction of a domain-wide Budget Reporting tool
• Online integration of common pollutant post-processing tasks (i.e. output total PM2.5 mass and more directly!)
• Community Contribution: Incorporation of the Two-Dimensional Volatility Bases Set (2D-VBS) chemical mechanism
• See the full list of CMAQv5.4 updates on our new CMAQ Wiki page. CMAQv5.4 Updates

Important update for WRF-CMAQ users

A bug was identified within the CMAQ to WRF coupling routine (twoway_feedback.F90) where aerosol feedback information is transferred from CMAQ to WRF. In doing so, it was found that WRF was not receiving the correct aerosol feedback information due to a looping error relating to the number of layers set to 1 in some cases. The bug impacts the WRF-CMAQ coupled system in the CMAQv5.3 release series (v5.3, v5.3.1, v5.3.2, v5.3.3) when running with short wave radiative feedback. The bug was not present in prior WRF-CMAQ versions. The bugfix in CMAQv5.4 now correctly captures the variations in the aerosol optical properties and consequently the direct feedback effects through all layers. Users of WRF-CMAQ are strongly encouraged to update to CMAQv5.4.

Getting the CMAQ Repository

This CMAQ Git archive is organized with each official public release stored as a branch on the main USEPA/CMAQ repository. The most recently released version of the the model will always be on the branch called 'main'. To clone code from the CMAQ Git archive, specify the branch (i.e. version number) and issue the following command from within a working directory on your server:

git clone -b main https://github.com/USEPA/CMAQ.git CMAQ_REPO

CMAQ Repository Guide

Source code and scripts are organized as follows:

• CCTM (CMAQ Chemical Transport Model): code and scripts for running the 3D-CTM at the heart of CMAQ.
• DOCS: CMAQ User's Guide, developers guidance, and short tutorials.
• PREP: Data preprocessing tools for important input files like initial and boundary conditions, meteorology, etc.
• POST: Data postprocessing tools for aggregating and evaluating CMAQ output products (e.g. Combine, Site-Compare, etc)
• PYTOOLS: Python pre- and postprocessing tools (currently this includes the DMSCHLO preprocessor)
• UTIL: Utilities for generating code and using CMAQ (e.g. chemical mechanism generation)

Documentation

Code documentation is included within this repository (they are version-controlled along with the code itself).

• FAQ for upgrading to the latest CMAQ version
• CMAQv5.4 Release Notes
• Tutorials
• CMAQ User's Guide
• Developers' Guide

CMAQ Test Cases

Benchmark/tutorial data for the CMAQv5.4 release are available from the CMAS Data Warehouse. The input and output files are stored on Google Drive and on Amazon Web Services (AWS) Open Data Registry. CMAQv5.4 comes with new input and output benchmark data for July 1-2, 2018 over the Northeast US (links provided below). Tutorials are provided for using the benchmark data to test running of the base CMAQ model, WRF-CMAQ, CMAQ-ISAM, and CMAQ-DDM-3D. The input datasets include a grid mask file for the United States (GRIDMASK_STATES_12SE1.nc). The grid mask file is used for running the new ISAM and DDM-3D test cases, or to test out regional emissions scaling with DESID. The input datasets also include an ocean file with new variables needed to use the cb6r5_ae7 and cb6r5m_ae7 mechanisms. See the Ocean File tutorial for more information on changes to the required ocean file input in v5.4.

In addition, a full set of inputs for 2018 are provided for the 12US1 domain (299 column x 459 row x 35 layer, 12-km horizontal grid spacing) on AWS, including emissions compatible with both the CB6r5 and CRACMM chemical mechanisms. Note that the 12US1 inputs are netCDF-4/HDF5 compressed files to substantially reduce file sizes. Through testing at the EPA, we’ve noticed that certain domains encounter model crashes from reading in large amounts of compressed netCDF data. A work around for those cases is uncompressing the data manually via nccopy 1 or m3cple (compiled with HDF5) before running the CMAQ simulation.

CMAQ Version	Data Type (Size)	Domain	Simulation Dates	Data Access
v5.4 CB6	Input (10.3 Gb)	Northeast US	July 1 - 2, 2018	Metadata, DOI, and download instructions Google Drive Link AWS Link
v5.4 CB6	Output (13.9 Gb)	Northeast US	July 1 - 2, 2018	Metadata, DOI, and download instructions Google Drive Link AWS Link
v5.4 CB6	Input	12US1	Jan 1 - Dec 31, 2018	Metadata, DOI, and links to data on AWS
v5.4 CRACMM	Input	12US1	Jan 1 - Dec 31, 2018	Metadata, DOI, and links to data on AWS

User Support

• Frequent CMAQ Questions are available on our website.
• The CMAS User Forum is available for users and developers to discuss issues related to using the CMAQ system. Please read and follow these steps prior to submitting new questions to the User Forum.

EPA Disclaimer

The United States Environmental Protection Agency (EPA) GitHub project code is provided on an "as is" basis and the user assumes responsibility for its use. EPA has relinquished control of the information and no longer has responsibility to protect the integrity, confidentiality, or availability of the information. Any reference to specific commercial products, processes, or services by service mark, trademark, manufacturer, or otherwise, does not constitute or imply their endorsement, recommendation or favoring by EPA. The EPA seal and logo shall not be used in any manner to imply endorsement of any commercial product or activity by EPA or the United States Government.

• Open source license

cmaq's Issues

Night AOD or aerosol extinction is zero from CCTM_PHOTDIAG3 file simulated by CMAQv5.3.1?

Daytime

Nightime

If so, any recommendation how to fix this problem?

DMS and Chlorophyll Notebook issue with Chlorophyll

Description
When running the Jupyter Notebook in PYTOOLS/dmschlo/CMAQ_DMS_ChlorA.ipynb, the notebook fails when finding climatology files.

Scope and Impact

This causes the notebook to fail and the DMS/CHLO variables cannot be created.

Solution

The error is caused by a server reorganization of the files on the server at the OPBG DAAC.

1. the directory structure has changed from Julian day (%j) of year to month-day (%m%d).
2. Naming structure also changed
   • old> A%Y%j%Y%j.L3m_MC_CHL_chlor_a_9km.nc
   • new> AQUA_MODIS.%Y%m%d_%Y%m%d.L3m.MC.CHL.chlor_a.9km.nc
3. The climatology files previously used 2003-08-01 as the start of climatology data, but now they are starting with 2002-07-01.

This requires two changes to the notebook. Both are in the cell that starts with “if getlatestchlo”.

First, change dates in the for loop

<    for prefix in ['2003/0801', '2003/0901', '2003/1001', '2003/1101', '2003/1201', '2004/0101', '2004/0201', '2004/0301', '2004/0401', '2004/0501', '2004/0601', '2004/0701']:
>    for prefix in ['2002/0701', '2002/0801', '2002/0901', '2002/1001', '2002/1101', '2002/1201', '2003/0101', '2003/0201', '2003/0301', '2003/0401', '2003/0501', '2003/0601']:

Second, change the regular expression that finds the files

<        mostrecent = sorted(re.compile('(?<=>).+L3m_MC_CHL_chlor_a_9km.nc(?=</)').findall(htmltxt))[-1]
>        mostrecent = sorted(re.compile('(?<=>).+.L3m.MC.CHL.chlor_a.9km.nc(?=</)').findall(htmltxt))[-1]

I have successfully run for a new domain with the latest climatology files.

Additional context

A PR will be forthcoming that also changes the documentation

This type of update is inherent in including download as part of the process.

1. We could move download out of the notebook, but the problem of reorganization will continue -- just outside the notebook.
2. We could avoid this by using the CMR to dynamically query, but the CMR is not always up-to-date.
3. Open to other proposals.

Two-Way Model Download

The instructions for the WRF-CMAQ model on the release notes page are not correct:

https://github.com/USEPA/CMAQ/blob/5.2/CCTM/docs/Release_Notes/Two_Way_Coupled_WRF-CMAQ.md

Download coupled model tarball from the CMAS Center Software Clearinghouse. From http://www.cmascenter.org, select Download -> Software -> CMAQ and choose version 5.2 to download the file WRFv3.8_CMAQv5.2_TwoWay_Model.tar.gz. Unzip the tarball and then move the twoway directory inside WRFV38 as well.

There is no tar.gz file on the CMAS page. Further, we should put the contents of this file up on GitHub so that it downloads with the repo.

MCIP error when processing WRF V4.0.3 output

Dear all,

I am trying to run MCIP v4.5 with WRF V4.0.3 output but I am getting this error:

Mon Apr 15 13:57:50 EDT 2019
rm: No match.

     This program uses the EPA-AREAL/MCNC-EnvPgms/BAMS Models-3
     I/O Applications Programming Interface, [I/O API] which is
     built on top of the netCDF I/O library (Copyright 1993, 1996
     University Corporation for Atmospheric Research/Unidata
     Program) and the PVM parallel-programming library (from
     Oak Ridge National Laboratory).
     Copyright (C) 1992-2002 MCNC,
     (C) 1992-2013 Carlie J. Coats, Jr.,
     (C) 2003-2012 Baron Advanced Meteorological Systems, LLC, and
     (C) 2014-2016 UNC Institute for the Environment.
     Released under the GNU LGPL  License, version 2.1.  See URL

         https://www.gnu.org/licenses/old-licenses/lgpl-2.1.html

     for conditions of use.

     ioapi-3.2: $Id: init3.F90 98 2018-04-05 14:35:07Z coats $
     Version with PARMS3.EXT/PARAMETER::MXVARS3= 2048
     netCDF version 4.1.3 of Jun 19 2018 15:50:22 $


     EXECUTION_ID: mcip

 ==============================================================================

                  US EPA COMMUNITY MULTISCALE AIR QUALITY MODEL
                    METEOROLOGY-CHEMISTRY INTERFACE PROCESSOR

                           MCIP V4.5 FROZEN 03/16/2018

 ==============================================================================




 **********************************************************************
 *** SUBROUTINE: SETUP
 ***   UNKNOWN OR UNSUPPORTED WRF OUTPUT VERSION
 ***   VERSION =  OUTPUT FROM WRF V4.0.3 MODEL
 **********************************************************************

     *** ERROR ABORT in subroutine SETUP
     ABNORMAL TERMINATION IN SETUP
     Date and time  0:00:00   July 22, 2015   (2015203:000000)

Error running mcip

My WRF V4.0.3 is with terrain following sigma levels and not with hybrid vertical levels.

Is there a way to fix or I just need to add the "V4" label in the "ifs" that are checking the model version in /src/setup.f90 and /src/setup_wrfem.f90 ?

I changed the MCIP code at /src/setup.f90 and /src/setup_wrfem.f90 to also accept the V4 version:

in /src/setup.f90 at line 131
from:
IF (wrfversion(18:19) == "V3" ) THEN
to:
IF ( (wrfversion(18:19) == "V3") .OR. (wrfversion(18:19) == "V4") ) THEN

in /src/setup_wrfem.f90 at line 918
from:
IF ( (wrfversion(18:21) == "V2.2") .OR. (wrfversion(18:19) >= "V3") ) THEN

to:

IF ( (wrfversion(18:21) == "V2.2") .OR. (wrfversion(18:19) >= "V3") &
                                   .OR. (wrfversion(18:19) >= "V4")) THEN

If I do this changes, MCIP will keep the consistency or I'll have problems with it?

Thank you so much for your help

MPI message loss issue in HPC Cluster

Hi, All

I have some issues related loss of message passing in hpc cluster.
Here is my problem:
(I don't know the version of CMAQ that I used.)
In y_ppm & y_yamo subroutines, the code use SWAP2D & SWAP3D(in swap_sandia_routines.f).
But there are some loss of message passing with between procs.

I have check that the CMAQ currently being deployed does not have swap_sandia, nor does y_yamo. It would be nice to get a new version and test it, but I want to solve the problem in the code that I have.
Could you tell me if there was an issue related to it ?

Actually, I thought it was MPI problem and I asked intel and there are more details in the posting.
If you want, check the address below or I'll get the contents.

https://software.intel.com/en-us/forums/intel-clusters-and-hpc-technology/topic/851369#comment-1955679

Additional Bugfix for Representative Day 2-D & 3-D Surface Gridded Emissions Files

CMAQv5.3.1-i4
Date: 2020-03-09
Contact: David Wong ([email protected])

Description: Using a 2-D and/or 3-D Gridded Emission File with representative day format specified via runscript with the environmental variable GR_EM_SYM_DATE_XXX set to T will cause a model crash when reading time-dependent initial conditions (ICs) due to a memory issue. This issue stems from the choice to psuedo-interpolate the ICs instead of extracting them directly, which is problematic if an emissions file is a 2-D or 3-D Emissions file of representative type. This is because to linearly interpolate temporally, two points are required stored as the head and tail. The head was stored correctly, but the tail was not being stored correctly and was picking up from whatever was in memory last, which usually is the 2-D/3-D Gridded Emissions file. This resulted in an error when trying to extract the data at the second point as the IC file only has data at the head.

If no emissions are present, the interpolation would pick the point from whatever file was read last whether that be a MET file, bioseason file, lightning file or IC file.

It should be noted, this issue would have not been seen if the IC file were time independent as IOAPI ignores the time input when trying to extract data from time independent files.

Scope and Impact: The model will terminate execution with a time retrieval error for the initial conditions. No model results will change.

Solution: Replace CCTM/src/cio/centralized_io_module.F file in repository with the version located in the folder DOCS/Known_Issues/CMAQv5.3.1-i2. Directions on how to replace this file in your repository can be found in Appendix F of the CMAQ User's Guide. This code fix will also be included with the next minor CMAQ release.

/usr/bin/time: Command not found in CMAQ v.5.3.3

Good day,

I am trying to apply use the standalone CMAQ v. 5.3.3 but need help to resolve the above error that seems like the runscript not starting mpirun (There are no rsl.* files). I have it attached here, together with the job*.err.txt and job*.out.txt files. I think user-specific syntax is needed for mpirun, but unsure how to go about it.
job.12760.err.txt
job.12760.out.txt
run_cctm_Bench_2016_12SE1.txt

CTM_WVEL must be set to Y in all runscripts

Description

CTM_WVEL is a run time science option to write out the vertical velocity component to the concentration file. The default setting, currently, is listed as Y in all runscripts within the repository. If the CTM_WVEL science option is set to N, the model immediately crashes . This is because the array that stores the vertical velocity component for writing to the concentration file is never allocated and is being used to calculate the average vertical velocity to be written to the average concentration file.

Scope and Impact

The model will terminate execution with a segmentation fault.

Solution

Users should leave CTM_WVEL set to Y in all runscripts. A code fix will be included with the next minor CMAQ release.

request to change your tutorial

In order to keep current with respect to netcdf-c and netcdf-fortran
installation, I would suggest you modify your tutorial page
(https://github.com/USEPA/CMAQ/blob/5.2.1/DOCS/Tutorials/CMAQ_GettingStarted.md)
to refer to the following pages at unidata:

1. Install netcdf-c : https://www.unidata.ucar.edu/software/netcdf/docs/getting_and_building_netcdf.html
2. Install netcdf-fortran : https://www.unidata.ucar.edu/software/netcdf/docs/building_netcdf_fortran.html

Bugfix for Representative Day 2-D & 3-D Surface Gridded Emissions Files

CMAQv5.3.1-i2:
Date: 2019-12-31
Contact: David Wong ([email protected])

Description : Using a 2-D and/or 3-D Gridded Emission File with representative day format specified via runscript with the environmental variable GR_EM_SYM_DATE_XXX. set to T, will prompt the centralized i/o module to store the start date of the file. However, this information was never passed on to the function used to extract the data from the netCDF file causing an error, as the time is not available on file. The information is now properly passed on to the variable required to extract the data from the netCDF file.

Scope and Impact: The model will terminate execution with a time retrievel error. No model results will change.

Solution: Replace CCTM/src/cio/centralized_io_module.F file in repository with the version located in the folder DOCS/Known_Issues/CMAQv5.3.1-i2. This code fix will also be included with the next minor CMAQ release.

Sulfur Tracking issues in CMAQv5.4

Description
A few issues with the implementation of the Sulfur Tracking Method (STM) in CMAQv5.4 have been identified. These are as follows: (1) Not all chemical mechanisms that predict organo-sulfate species have been included in the list of mechanisms for which STM should also track “OSO4” species. (2) The model makes erroneously low ASO4GAS tag predictions. (3) Due to the aerosol-to-aqueous (AE2AQ) surrogate chosen for ABR, there will be differences between STM and non-STM model results for the cb6r5m_ae7_aq mechanism.

Scope and Impact
If employing the STM in CMAQv5.4, your SO4 tags will be incorrect due to the severe underprediction of the ASO4GAS tag. If running STM with a chemical mechanism that has organo-sulfate species but is not included in the list of mechanisms for which STM should calculate “OSO4” tags (i.e., cb6r5_ae7* and cracmm* mechanisms), said “OSO4” tags will not be included in the model output. Predictions with STM for the cb6r5m_ae7_aq mechanism will be incorrect and will differ from the base model results.

Solution
The solution to the first issue is to add the cb6r5_ and cracmm mechanisms to the STM-related lists regarding OSO4 tagged species in the following files: AERO_DATA.F, CGRID_SPCS.F, and convcld_acm.F.

The solution to the second issue is to swap the order of the calls of PA_UPDATE_AERO and STM_WRAP_AE after the AERO call in sciproc.F.

Until a fix has been implemented to update this and other aerosol-to-aqueous surrogates, it is not recommended to use STM with the cb6r5m_ae7_aq mechanism at this time.

Additional context
A pull request has been prepared and is under review to address these issues.

error - related to I/O API MXFILE3 = 64 open file limit when testing CMAQ_DDM

Obtained this error when testing a run of CMAQ v5.2 DDM at UNC
Compiler: gcc (on UNC systems used command module load openmpi_gcc/4.8.1)
I/O API version: 3.1

 >>--->> WARNING in subroutine OPEN3     Could not open S_CGRID         Maximum number of files already have been opened.
     *** ERROR ABORT in subroutine WR_CGRID on PE 000
     Could not open S_CGRID file
     Date and time 0:00:00   July 2, 2011   (2011183:000000)
--------------------------------------------------------------------------
MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
with errorcode 32767.

Work around:
Reduced the number of files opened by changing the following settings in the run script.
(These settings also matched the run script run.ddm.cmas provided by Sergei)

setenv CTM_ABFLUX N
setenv CTM_GRAV_SETL N

Proposed fix:
Recompile I/O API with a larger MXFILE3 setting

Error in SpecDef_cb6r3_ae6_aq.txt definition of VOC in CMAQ v5.3.2 op and dev branches

The formula has a redundant occurrence of the term
2.0*ETHA[1]. This was confirmed as an error by Christian Hogrefe when checked against the 5.2.1 release code base (https://github.com/USEPA/CMAQ/blob/5.2.1/CCTM/src/MECHS/cb6r3_ae6_aq/SpecDef_cb6r3_ae6_aq.txt). The formula is commented out but could be uncommented and used, and needs to be corrected.

SNOWH variable missing from wrf sample input files

Using the following command, it is determined that SNOWH is missing from the wrf sample input data for MCIP.

ncdump -h subset_wrfout_d01_2011-07-01_00:00:00 | grep SNOWH $1

This caused the following error to be generated when running mcipv5.4, see following comment.
https://gist.github.com/lizadams/4d5a3c76d28621f30485f398e491c7ff#gistcomment-2886966

Tanya Spero will be providing new subset_wrfout files.
Will need them for both the single day and multi day benchmark cases.
Currently have the following files for single day

SE52BENCH/single_day/cctm_input/met/wrf
subset_wrfout_d01_2011-07-01_00:00:00  
subset_wrfout_d01_2011-07-02_00:00:00

And the following file for the multi-day benchmark case

subset_wrfout_d01_2011-07-01_00:00:00
subset_wrfout_d01_2011-07-02_00:00:00
subset_wrfout_d01_2011-07-03_00:00:00
subset_wrfout_d01_2011-07-04_00:00:00
subset_wrfout_d01_2011-07-05_00:00:00
subset_wrfout_d01_2011-07-06_00:00:00
subset_wrfout_d01_2011-07-07_00:00:00
subset_wrfout_d01_2011-07-08_00:00:00
subset_wrfout_d01_2011-07-09_00:00:00
subset_wrfout_d01_2011-07-10_00:00:00
subset_wrfout_d01_2011-07-11_00:00:00
subset_wrfout_d01_2011-07-12_00:00:00
subset_wrfout_d01_2011-07-13_00:00:00
subset_wrfout_d01_2011-07-14_00:00:00
subset_wrfout_d01_2011-07-15_00:00:00

Liz

Add Example Plots

Add a directory of all example plots to the user's guide. Reference these examples in the text of the guide.

CMAQ will not work on macOS

Description
The existence of the following files permits building or even doing any git work on macOS:

UTIL/create_ebi/src_RXNSU/junit.f
UTIL/create_ebi/src_RXNSU/junit.F
UTIL/inline_phot_preproc/src/CSQY_DATA.f
UTIL/inline_phot_preproc/src/CSQY_DATA.F

It also breaks any build system that attempts to automatically update branches from the authoritative repositories, because there are always changes to these files that result in unresolvable conflicts. This is because macOS uses a pseudo case-sensitive filesystem and doesn't recognize that UTIL/create_ebi/src_RXNSU/junit.f and UTIL/create_ebi/src_RXNSU/junit.F are different files.

If the .f versions of the two files above are preprocessed from the .F files and were mistakenly added to the repository, then these should be removed.

Scope and Impact
See above

Solution
Remove the two .f files and do preprocessing on the fly (if a two-step approach = preprocessing+compiling is preferred, then the preprocessor must write to .tmp.f instead of .f, or another problem will be created).

timing log from UNC's dogwood machine on 64 processors

Timing log for CMAQv5.3beta using mvapich2.
This uses the following srun command line options
srun -n $NPROCS --cpu_bind=cores --cpu_bind=verbose --mpi=pmi2 $BLD/$EXEC |& tee buff_${EXECUTION_ID}.txt

Total Time = 399.20
slurm-670265.out.CCTM_v53_mvapich2_2.3rc1_intel_17.2_intel_SE52BENCH.txt

Much slower times were obtained with these command line options:
srun -n $NPROCS --mpi=pmi2 $BLD/$EXEC |& tee buff_${EXECUTION_ID}.txt

Num Day Wall Time
01 2011-07-01 1763.8

slurm-614125.out.CCTM_v53_mvapich2_2.3rc1_intel_17.2_intel_SE52BENCH.no_cpu_bind.txt

OMP

Hi, Can CMAQ support openmp + mpi ?

WRF-CMAQ: Average ELMO Output File Gives Erroneous Results

Description
It has been discovered that the ELMO output file gives erroneously low results (about 90% low) for all variables when the WRF-CMAQ model is used. This issue does not impact the standard offline CMAQ model, which applies to the vast majority of use cases.

Scope and Impact
If you are using WRF-CMAQ with CMAQv5.4, you should not trust the output data on the ELMO files for now.

Solution
The temporary solution is to continue using the output variables from CONC and ACONC if you are running WRF-CMAQ. Please also continue to use the SpecDef files with the combine utility from CMAQv5.3.3.

We will solve the bug in the WRF-CMAQ implementation of ELMO and issue the fix as a Pull Request to this repository.

Additional context
Again, this only impacts WRF-CMAQ with CMAQv5.4 for output from the ELMO processing module.

Typo for ioapi include path for Intel compiler in config_cmaq.csh

Code is:

setenv IOAPI_INCL_DIR iopai_inc_intel #> I/O API include header files

Note "iopai"

How to modify the dry deposition velocity for a specific area?

I need to modify the dry deposition velocity for a specific area.

So I change the file: /project/yoj/arc/CCTM/src/aero/aero5/aero_depv.F
After the line 277:
IF (C .GT. 10 .AND. C .LT. 20 .AND R .GT. 10 .AND. R .LT. 20) THEN
VDEP_AE(V, C, R) = 100.0
ENDIF

But there were no difference between modified dry deposition velocity and unmodify, only some noisy.

Can someone help solve this problem?

Can we make a docker image for CMAQ to simplify the installation process?

A containers (Docker, Singularity, etc.) with working CMAQ setups will be a really helpful and save a lot of time while deploying this tools.

undefined references, missing libraries for pnetcdf and openmp

Obtained the following type of message when trying to compile CMAQ
/home/lizadams/CMAQ_REPO/linuxbrew_mpif90_gcc/lib/x86_64/gcc/ioapi/lib/libioapi.a(wrtflag.o): In function wrtflag_': wrtflag.f:(.text+0x75): undefined reference to GOMP_critical_name_start'
wrtflag.f:(.text+0xab): undefined reference to GOMP_critical_name_end' wrtflag.f:(.text+0x104): undefined reference to GOMP_critical_name_start'
wrtflag.f:(.text+0x125): undefined reference to GOMP_critical_name_end' wrtflag.f:(.text+0x396): undefined reference to GOMP_critical_name_start'
wrtflag.f:(.text+0x870): undefined reference to GOMP_critical_name_end' /home/lizadams/CMAQ_REPO/linuxbrew_mpif90_gcc/lib/x86_64/gcc/ioapi/lib/libioapi.a(modatts3.o): In function modatts3_MOD_pn_getcmaq':
modatts3.F90:(.text+0x139): undefined reference to nfmpi_get_att_int_' modatts3.F90:(.text+0x171): undefined reference to nfmpi_get_att_text'
modatts3.F90:(.text+0x19f): undefined reference to nfmpi_get_att_text_' modatts3.F90:(.text+0x1cd): undefined reference to nfmpi_get_att_text'
modatts3.F90:(.text+0x1fb): undefined reference to nfmpi_get_att_text_' modatts3.F90:(.text+0x22c): undefined reference to nfmpi_get_att_text_'

When I added the following to the Makefile: -pnetcdf -fopenmp, then the undefined references were satisfied.
NETCDF = -L$(LIB)/netcdf/lib -lnetcdff -lnetcdf -lpnetcdf -fopenmp

Note, to obtain netcdf and pnetcdf, I used the following method:
Install Linuxbrew on a linux machine following instructions on this link: http://linuxbrew.sh/
Install netcdf using linuxbrew command:
brew install netcdf
Install pnetcdf using linuxbrew command from this link: MPAS-Dev/MPAS-Model#34
brew install pwolfram/mpas/parallel-netcdf
All of the libraries, executables, and include files are available under:
/home/linuxbrew/.linuxbrew
Download ioapi, using the github clone command
git clone https://github.com/cjcoats/ioapi-3.2
Install csh on ubuntu
sudo apt-get install csh
Edit the Makefile to specify the Makeinclude that you will be using
BIN = Linux4_x86_64gfortmpi

cd ioapi
cp Makeinclude.Linux2_x86_64gfortmpi Makeinclude.Linux4_x86_64gfortmpi
I made two modifications to the ioapi-3.2/ioapi/Makeinclude.Linux4_x86_64gfortmpi
Changed:
CXX = mpicxx
to remove the path to the default system libraries-L/usr/lib64
ARCHLIB = -dynamic -lm -lpthread –lc

Edit the ioapi-3.2/Makefile to add the pnetcdf library
NCFLIBS = -lnetcdf -lnetcdff -lpnetcdf
Run make to build the ioapi library and m3tools
make
Next step was to download the 5.3beta2 version of CMAQ

git clone -b 5.3.b2 https://github.com/USEPA/CMAQ.git CMAQ_REPO

cd CMAQ_REPO
edit bldit_project.csh to set CMAQ_HOME
set CMAQ_HOME = /home/lizadams/linuxbrew_mpi_gcc
./bldit_project.csh
cd /home/lizadams/linuxbrew_mpi_gcc
I made a modification to the config_cmaq.csh under the section:
case gcc:

    #> I/O API, netCDF, and MPI library locations
    setenv IOAPI_MOD_DIR   /home/lizadams/ioapi-3.2/Linux4_x86_64gfortmpi #> I/O API precompiled modules
    setenv IOAPI_INCL_DIR  /home/lizadams/ioapi-3.2/ioapi/fixed_src  #> I/O API include header files
    setenv IOAPI_LIB_DIR   /home/lizadams/ioapi-3.2/Linux4_x86_64gfortmpi  #> I/O API libraries
    setenv NETCDF_LIB_DIR  /home/linuxbrew/.linuxbrew/lib #> netCDF directory path
    setenv NETCDF_INCL_DIR /home/linuxbrew/.linuxbrew/include  #> netCDF directory path
    setenv MPI_LIB_DIR     /home/linuxbrew/.linuxbrew/lib    #> MPI directory path

    setenv netcdf_lib "-lnetcdff -lnetcdf -lpnetcdf"
    setenv myLINK_FLAG "-fopenmp"

Built CMAQ using the following:

cd CCTM/scripts
./bldit_cctm.csh gcc |& tee ./bldit_cctm.log

Successful build with no undefined references.

Issue with EMIS_SYM_DATE flag

CMAQv5.3.1-i5
Date: 2020-03-09
Contact: David Wong ([email protected])

Description: The inability to run with the EMIS_SYM_DATE flag due inconsistent implementation between cio and the emissions modules.

Scope and Impact: The model will terminate execution with a time retrieval error for the emission stream.

Solution: Do not use the environmental variable EMIS_SYM_DATE. Instead, if you wish to use representative day emissions for some or all of your emission streams, individually set the GR_EM_SYM_DATE_### and/or STK_EM_SYM_DATE_### environment variables to T for the desired streams. A code fix will be included with the next minor CMAQ release.

BCON Operator does not support equatorial mercator

From Barron Henderson:

Add the code below to lat_lon.F (before ELSE ! Unsupported Coordinates) to enable support.

      ELSE IF ( GDTYP .EQ. EQMGRD3 ) THEN   ! Equatorial mercator Coordinates
         print *,P_ALP,P_BET,P_GAM,XCENT,YCENT
         IF ( .NOT. SETEQM( SNGL( 1. ),        !  first, initialize
     &                      SNGL( P_BET ),        !  for EQM2LL()
     &                      SNGL( P_GAM ),
     &                      SNGL( XCENT ),
     &                      SNGL( YCENT ) ) ) THEN
            MSG = 'Equatorial merc proj setup error for CTM CONC file'
            CALL M3EXIT( PNAME, 0, 0, MSG, 2 )
         END IF

         X = X0 + FLOAT( COL ) * SNGL( XCELL )
         Y = Y0 + FLOAT( ROW ) * SNGL( YCELL )
         IF ( .NOT. EQM2LL( X, Y, LON, LAT ) ) THEN
            MSG = 'Equatorial merc conversion error for CTM CONC file'
            CALL M3EXIT( PNAME, 0, 0, MSG, 2 )
         END IF

CMAQ version release note does not work

Description
The links of releases note in this page does not work
https://github.com/USEPA/CMAQ/releases

Bugfix for ISAMv5.3.1

CMAQv5.3.1-i3:
Date: 2020-01-23
Contact: Sergey Napelenok ([email protected])

Description
CMAQ version 5.3.1 release included an inadvertently modified version of EMIS_DEFN.F causing crashes during compiling.

Scope and Impact
The model did not compile with the -Disam option with errors pointing to EMIS_DEFN.F

Solution

1. Option 1: Replace CCTM/src/emis/emis/EMIS_DEFN.F in repository with the version located in the folder DOCS/Known_Issues/CMAQv5.3.1-i3. Directions on how to replace this file in your repository can be found in Appendix F of the CMAQ User's Guide.
2. Option 2: The code fix to EMIS_DEFN.F was merged into the master branch on 1/23/2020. You can download the updated v5.3.1 source code from this link: https://github.com/USEPA/CMAQ/archive/master.zip, or use git commands to update your current repository.

Two-Way Model Package Updates/Bug Fixes

The twoway/reconfigure script is looking for a CCTM config file called cfg*. The name of this file in the new CMAQv5.2 release is now *.cfg.

Change line 3 for twoway/reconfigure to:
set cfg_file = ls -1 cmaq/*.cfg

How to activate VBS in CMAQ v5.2?

Hi,

I would like to use the VBS mechanism for SOA simulation in CMAQ v5.2, but it seems no explicit way to activate it. I found a piece of code in CCTM/src/aero/aero6/AERO_DATA.F as follows:

C Primary Organic Aerosol Volatility Distributions
      Integer, Parameter :: n_vbs_bin = 5
      Character( 10 ) :: poa_name( n_vbs_bin ) = (/ 'LVPO1', 'SVPO1', 'SVPO2', 'SVPO3', 'IVPO1' /)
      Real, Parameter :: poa_op_vf( n_vbs_bin ) = (/ 0.09,  0.09,  0.14,  0.18,  0.5 /)  ! Aggregated

! The Following Volatility Distributions are alternative options
! but at this point can only be implemented indivdually for the
! entire POA suite of compounds.
!     Real, Parameter :: poa_gv_vf(n_vbs_bin) = (/0.27,  0.15,   0.26,   0.15,   0.17/) ! Gasoline
!     Real, Parameter :: poa_dv_vf(n_vbs_bin) = (/0.03,  0.25,   0.37,   0.24,   0.11/) ! Diesel
!     Real, Parameter :: poa_bb_vf(n_vbs_bin) = (/0.2,   0.1,    0.1,    0.2,    0.4/)  ! Biomass Burning
!     Real, Parameter :: poa_nv_vf(n_vbs_bin) = (/1.0,   0.0,    0.0,    0.0,    0.0/)  ! Nonvolatile
!     Real, Parameter :: poa_mc_vf(n_vbs_bin) = (/0.35,  0.35,   0.1,    0.1,    0.1/)  ! Meat Cooking

! POA_AMF is the fraction of total emissions in the particle phase.
! This parameter is for distributing the total emissions between
! gas and particle BEFORE the aerosol size distribution is applied.
! This helps prevent numerical issues with shrinking the particles
! instantaneously.
      Real, Parameter :: poa_amf( n_vbs_bin ) = (/ 1.0, 0.5, 0.0, 0.0, 0.0 /)

C number of lognormal modes in windblown dust aerosol = n_mode
C - but only accumulation and coarse modes used
      Type emis_table
         Character( 16 ) :: description
         Character( 16 ) :: name  ( n_mode )
         Real            :: spcfac( n_mode )
      End type emis_table

However, I am not sure whether it implements the VBS mechanism if I uncomment the piece of code. Moreover, is there anything I should do with the emission inventories if using the VBS mechanism?

Operational Guidance: Chapter 7 Figures

Need to update program schematics to be consistent with CMAQv5.2. Current figures are referring to old mechanisms INCLUDE (*.EXT) files; replace with nml files.

When i use csh assemble command to build the wrf-cmaq i got error

cp: cannot stat ?.rfcmaq_twoway_coupler/Makefile?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/clean?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/configure?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/Registry/Registry.EM?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/Registry/registry.em_shared_collection?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/arch/Config.pl?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/dyn_em/module_first_rk_step_part1.F?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/dyn_em/solve_em.F?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/dyn_nmm/module_PHYSICS_CALLS.F?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/main/Makefile?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/main/depend.common?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/phys/Makefile?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/phys/module_ra_rrtmg_sw.F?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/phys/module_radiation_driver.F?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/phys/module_sf_noahdrv.F?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/external/makefile?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/Registry/registry.WRF-CMAQ-twoway?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/phys/complex_number_module.F?. No such file or directory
cp: cannot stat ?.rfcmaq_twoway_coupler/phys/module_twoway_rrtmg_aero_optical_util.F?. No such file or directory

I can be sure that these files exist, I am using bash, is it a problem with the script?

I/O API OPEN3 File Types

Consider changing as many of the output files as possible from the FSNEW3 type to FSRDWR3 in the I/O API OPEN3 statement. Several of the output files are already opened as FSRDWR3, but some are still FSNEW3. The problem with the FSNEW3 type is that if the file is already on disk when someone tries to run the model (like for a restart after a failed run), the program will abort with a nebulous message about not being able to write to the offending file.

Here are the files that I'm seeing are opened as FSNEW3 for the CMAQ benchmark case

grep OPEN3 *.F | grep FSNEW

returns:

BIDI_MOD.F: If ( .Not. OPEN3( MEDIA_CONC, FSNEW3, PNAME ) ) Then
hrno.F: IF ( .NOT. OPEN3( SOILOUT, FSNEW3, PNAME ) ) THEN
MGEMIS.F: IF ( .NOT. OPEN3( CTM_MGEM_1, FSNEW3, PNAME ) ) THEN
opaconc.F: IF ( .NOT. OPEN3( A_CONC_1, FSNEW3, PNAME ) ) THEN
opapmdiag.F: IF ( .NOT. OPEN3( CTM_APMDIAG_1, FSNEW3, PNAME ) ) THEN
opavis.F: IF ( .NOT. OPEN3( CTM_AVIS_1, FSNEW3, PNAME ) ) THEN
opconc.F: IF ( .NOT. OPEN3( CTM_CONC_1, FSNEW3, PNAME ) ) THEN
opddep.F: IF ( .NOT. OPEN3( CTM_DRY_DEP_1, FSNEW3, PNAME ) ) THEN
opddep_fst.F: IF ( .NOT. OPEN3( CTM_DRY_DEP_FST, FSNEW3, PNAME ) ) THEN
opddep_mos.F: IF ( .NOT. OPEN3( CTM_DRY_DEP_MOS, FSNEW3, PNAME ) ) THEN
opdepv_diag.F: IF ( .NOT. OPEN3( CTM_DEPV_DIAG, FSNEW3, PNAME ) ) THEN
opdepv_fst.F: IF ( .NOT. OPEN3( CTM_DEPV_FST, FSNEW3, PNAME ) ) THEN
opdepv_mos.F: IF ( .NOT. OPEN3( CTM_DEPV_MOS, FSNEW3, PNAME ) ) THEN
openlayout.F: IF ( .NOT. OPEN3( LNAME( N ), FSNEW3, UPNAM3D ) ) THEN
opphot.F: IF ( .NOT. OPEN3( CTM_RJ_1, FSNEW3, PNAME ) ) THEN
opphot.F: IF ( .NOT. OPEN3( CTM_RJ_2, FSNEW3, PNAME ) ) THEN
oppmdiag.F: IF ( .NOT. OPEN3( CTM_PMDIAG_1, FSNEW3, PNAME ) ) THEN
oppt3d_diag.F: IF ( .NOT. OPEN3( PT3DNAME, FSNEW3, PNAME ) ) THEN
opvis.F: IF ( .NOT. OPEN3( CTM_VIS_1, FSNEW3, PNAME ) ) THEN
opwdep.F: IF ( .NOT. OPEN3( CTM_WET_DEP_1, FSNEW3, PNAME ) ) THEN
opwdep.F: IF ( .NOT. OPEN3( CTM_WET_DEP_2, FSNEW3, PNAME ) ) THEN
pa_init.F: IF ( OPEN3( OUTFNAME, FSNEW3, PNAME ) ) THEN ! open new
pa_init.F: IF ( OPEN3( OUTFNAME, FSNEW3, PNAME ) ) THEN ! open new
SSEMIS.F: IF ( .NOT. OPEN3( CTM_SSEMIS_1, FSNEW3, PNAME ) ) THEN
tmpbeis.F: IF ( .NOT. OPEN3( SNAME, FSNEW3, PNAME ) ) THEN
wr_cgrid.F: IF ( .NOT. OPEN3( S_CGRID, FSNEW3, PNAME ) ) THEN

BELD4_LU undefined if CTM_ABFLUX == N

In the run script, if a user selects CTM_WB_DUST Y, and CTM_ABFLUX N, then the BELD4_LU is undefined. Need to add the following within the if then block for CTM_WB_DUST = Y.

  # Input variables for BELD4 Landuse option
  setenv BELD4_LU $LUpath/beld4_12US1_459X299_output_tot_bench.nc

https://github.com/USEPA/CMAQ/blob/5.2/CCTM/scripts/run_cctm.csh

Actually, there seems to be two different BELD4 files in the distribution, do you use one for the CTM_WB_DUST option, and the other for the CTM_ABFLUX option?

Regridding MODIS FPAR and LAI data

I (Dale) am attempting to run CMAQv5.2 and need gridded FPAR and LAI data.

I am attempting to Regrid MODIS FPAR and LAI Data following the tutorial:

https://github.com/USEPA/CMAQ/blob/5.2/DOCS/Tutorials/CMAQ_DustInput.md

However, I am getting the following error message:

discover35/discover/nobackup/djallen4/wrf/WPS> ./geogrid.exe
Parsed 26 entries in GEOGRID.TBL
Processing domain 1 of 1
ERROR: Could not open /discover/nobackup/djallen4/wrf/WPS/data/WPS_GEOG/modis_landuse_20class_30s/index

I do see a similar file in a subdirectory that ends with ... with_lakes
discover32/discover/nobackup/djallen4/wrf/WPS/data/WPS_GEOG/modis_landuse_20class_30s_with_lakes> ls -la index
-rw-r--r-- 1 djallen4 s1090 358 Apr 5 2017 index

Can this be fixed by modifying GEOGRID.TBL?

Here is the namelist I used.
&share
wrf_core = 'ARW',
max_dom = 1,
start_date = '2013-08-18_00:00:00',
end_date = '2013-08-20_00:00:00',
interval_seconds = 21600
io_form_geogrid = 2,
opt_output_from_geogrid_path = '/discover/nobackup/djallen4/CMAQ-5.2/PREP/dust/daqtx/',
debug_level = 0
/

&geogrid
parent_id = 1,
parent_grid_ratio = 1,
i_parent_start = 1,
j_parent_start = 1,
e_we = 148,
e_sn = 112,
!
!!!!!!!!!!!!!!!!!!!!!!!!!!!! IMPORTANT NOTE !!!!!!!!!!!!!!!!!!!!!!!!!!!!
! The default datasets used to produce the HGT_M, GREENFRAC,
! and LU_INDEX/LANDUSEF fields have changed in WPS v3.8. The HGT_M field
! is now interpolated from 30-arc-second USGS GMTED2010, the GREENFRAC
! field is interpolated from MODIS FPAR, and the LU_INDEX/LANDUSEF fields
! are interpolated from 21-class MODIS.
!
! To match the output given by the default namelist.wps in WPS v3.7.1,
! the following setting for geog_data_res may be used:
!
! geog_data_res = 'gtopo_10m+usgs_10m+nesdis_greenfrac+10m','gtopo_2m+usgs_2m+nesdis_greenfrac+2m',
!
!!!!!!!!!!!!!!!!!!!!!!!!!!!! IMPORTANT NOTE !!!!!!!!!!!!!!!!!!!!!!!!!!!!
!
geog_data_res = 'modis_30s+5m',
dx = 36000,
dy = 36000,
map_proj = 'lambert',
ref_lat = 40,
ref_lon = -97,
truelat1 = 33.0,
truelat2 = 45.0,
stand_lon = -97.0,
opt_geogrid_tbl_path = '/discover/nobackup/djallen4/wrf/WPS/geogrid/'
geog_data_path = '/discover/nobackup/djallen4/wrf/WPS/data/WPS_GEOG/'
/

Error to run the assemble command with WRF4.1.1_CMAQ5.3.2_twoway option

Hi Dr. Foley,
I'm a new user of WRF-CMAQ model, currently I have been compiling the this model with WRF4.1.1_CMAQ5.3.2_twoway option.
However, I got the problem at the 9 stage of the tutorials, which is displayed in the below:
[weather@igplogin WRF-4.1.1]$ ./twoway/assemble cp: cannot stat ‘cmaq/complex_number_module.F90’: No such file or directory grep: Makefile: No such file or directory grep: Makefile: No such file or directory grep: Makefile: No such file or directory grep: Makefile: No such file or directory grep: Makefile: No such file or directory grep: Makefile: No such file or directory wc: Makefile: No such file or directory cpp_begin_line: Subscript out of range. Please help me to address this issue.
In addition, I didn't find the org directory in the twoway/misc path.
Thank you very much!
Bests,
-Manh

Mercator projection not supported in nested simulations

Hi,

CMAQ docs says:

CMAQ can use any of the four map projections defined for WRF. The four map projection coordinate systems are regular latitude-longitude geographic, Lambert conformal conic, Mercator, and Polar stereographic

but I noticed that mercator projection is actually not supported by modules ICON and BCON with the m3conc option.

In order to implement this projection:

1. IOAPI v3.2 is required (because 3.1 has a bug that affects directly to this matter)
2. Add these lines to files PREP/bcon/src/common/lat_lon.F and PREP/icon/src/common/lat_lon.F at line 115:

      ELSE IF ( GDTYP .EQ. EQMGRD3 ) THEN  ! Equatorial Mercator

         IF ( .NOT. SETEQM( SNGL( P_ALP ),        !  first, initialize
     &                      SNGL( P_BET ),        !  for EQM2LL()
     &                      SNGL( P_GAM ),
     &                      SNGL( XCENT ),
     &                      SNGL( YCENT ) ) ) THEN
            MSG = 'Eq Mercator projection setup error for CTM CONC file'
            CALL M3EXIT( PNAME, 0, 0, MSG, 2 )
         END IF

         X = X0 + FLOAT( COL ) * SNGL( XCELL )
         Y = Y0 + FLOAT( ROW ) * SNGL( YCELL )
         IF ( .NOT. EQM2LL( X, Y, LON, LAT ) ) THEN
            MSG = 'Eq Mercator conversion error for CTM CONC file'
            CALL M3EXIT( PNAME, 0, 0, MSG, 2 )
         END IF

Thanks

Pedro

./combine_v531.exe: free(): invalid next size (normal)

I want to get all layer of the CMAQ output，this is my SpecDef

[bedrock@node1 combine]$ vim SpecDef_cb6r3_ae6_aq.txt 

!#start   YYYYJJJ  010000
!#end     YYYYJJJ  000000
!#layer         1 

I get this error

Copying Variables at time:2019205:140000
*** glibc detected *** ./combine_v531.exe: free(): invalid next size (normal): 0x0000000005bbca00 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3be2875e5e]
/lib64/libc.so.6[0x3be2878cf0]
/lib64/libc.so.6(__open_catalog+0x1b3)[0x3be2831113]
/lib64/libc.so.6(catopen+0x5d)[0x3be2830d5d]
./combine_v531.exe(for__issue_diagnostic+0x116)[0x53a4c6]
./combine_v531.exe(for__signal_handler+0x583)[0x5428c3]
/lib64/libpthread.so.0[0x3be340f7e0]
/public/home/bedrock/envs/v1.0/netcdf/4.7.0/lib/libnetcdf.so.15(ncx_getn_float_float+0x4a)[0x2ba4f70c1bca]
/public/home/bedrock/envs/v1.0/netcdf/4.7.0/lib/libnetcdf.so.15(+0x47765)[0x2ba4f7082765]
/public/home/bedrock/envs/v1.0/netcdf/4.7.0/lib/libnetcdf.so.15(NC3_get_vara+0x6cc)[0x2ba4f707cb0c]
/public/home/bedrock/envs/v1.0/netcdf/4.7.0/lib/libnetcdf.so.15(NC_get_vara+0x5f)[0x2ba4f706112f]
/public/home/bedrock/envs/v1.0/netcdf/4.7.0/lib/libnetcdff.so.6(nf_get_vara_real_+0x63d)[0x2ba4f6a9a3c1]
./combine_v531.exe[0x441ce7]
./combine_v531.exe[0x4f3540]
./combine_v531.exe[0x437354]
./combine_v531.exe[0x411e25]
./combine_v531.exe[0x41f8f0]
./combine_v531.exe[0x419777]
./combine_v531.exe[0x41769f]
./combine_v531.exe[0x415463]
./combine_v531.exe[0x414dcd]
./combine_v531.exe[0x42761a]
./combine_v531.exe[0x40e1a2]
/lib64/libc.so.6(__libc_start_main+0x100)[0x3be281ed20]
./combine_v531.exe[0x40e0a9]

this is my run script

#!/bin/sh
set -e # stop the shell on first error
set -u # fail when using an undefined variable
set -x # echo script lines as they are executed
set -o pipefail

 DOMAIN=1
 TASK_TIME=2019072412
 MECH=cb6r3_ae6_aq

# 执行ID
 export RUNID=combine # 

# 网格
 export IOAPI_ISPH=20 # WRF

# 输出输出
 export INFILE1=../cctm/D${DOMAIN}_CCTM_ACONC.nc  #> CCTM Output Directory
 export INFILE2=../mcip/D${DOMAIN}_METCRO3D_${TASK_TIME}.nc  #> CCTM Output Directory
 export INFILE3=../cctm/D${DOMAIN}_CCTM_APMDIAG.nc  #> 
 export INFILE4=../mcip/D${DOMAIN}_METCRO2D_${TASK_TIME}.nc  #> CCTM Output Directory
 export OUTFILE=./D${DOMAIN}_CCTM_ACONC.nc

# 化学机制: 变量映射关系
 export SPECIES_DEF=./SpecDef_${MECH}.txt

#> Use GENSPEC switch to generate a new specdef file (does not generate output file).
export GENSPEC=N

./combine_v531.exe

Please help to fix this problem, thanks

Benchmark model run gives only WRF output; no ACONC output

Description
WRF-CMAQ functions like WRF model.

Scope and Impact
Downloaded the WRF-CMAQ source code and input files for the two-day benchmark case. All the executables were successfully built as described at https://github.com/USEPA/CMAQ/blob/main/DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_WRF-CMAQ_build_gcc.md
The model runs but gives only WRF output, regardless of 'wrf_cmaq_option' setting. Is this normal?

Solution
Either the runscript or the code itself is missing something??

Additional context
Another on CMAQ user forum has reported same issue. Kindly assist.

config.cmaq.csh

A few changes to make this script work for the general user. The diffs below are in this form: diff cmas.config.csh epa.config.csh

Add a link flag for Intel compilers. This is a generic flag that the I/O API uses and it needs to be in there for the programs that use Intel-compiled I/O API libraries.

94c94
<         setenv myLINK_FLAG "-openmp"
---
>         setenv myLINK_FLAG "

You are linking the netcdf library directly into the NETCDF_DIR. This doesn't work because bldmake is looking in NETCDF_DIR/lib for the library. Either fix this in bldmake, or just make the change that I suggest here to the config.cmaq.csh script.

190,191c189
<  if ( ! -d $NETCDF_DIR )  mkdir $NETCDF_DIR
<     ln -s $NETCDF_LIB_DIR $NETCDF_DIR/lib
---
>  if ( ! -d $NETCDF_DIR )  ln -s $NETCDF_LIB_DIR $NETCDF_DIR

We can get rid of the check for the NetCDF includes, as they're not needed anymore:

204,207c202,205
< # if ( ! -e $NETCDF_DIR/include/netcdf.h ) then 
< #    echo "ERROR: $NETCDF_DIR/include/netcdf.h does not exist in your CMAQ_LIB directory !!! Check your installation before proceeding with CMAQ build."
< #    exit
< # endif
---
>  if ( ! -e $NETCDF_DIR/include/netcdf.h ) then 
>     echo "ERROR: $NETCDF_DIR/include/netcdf.h does not exist in your CMAQ_LIB directory !!! Check your installation before proceeding with CMAQ build."
>     exit
>  endif

MEGAN preprocessor

HI,

In the Documentation (Chapter 4) when describing MEGAN input files, it mentioned a "MEGAN preprocessor", could you share where to find it?

Thanks

Problems with Mercator Projection in MCIP

Hi,

I'm trying to run CMAQ with a grid in mercator projection.

I am preparing the input data and I have reprojected latlon (epsg:4326) data to my mercator projection with PROJ library.
Documentation/Users_Guide says I should use the following srs definition in order to reproject properly my data:
Mercator: "+proj=merc +a=6370000.0 +b=6370000.0 +lat_ts=$P_ALP +lon_0=$P_GAM"

With $P_ALP and $P_GAM given in the GRIDDESC file.

But when I do that it doesn't exactly match the grid generated by MCIP nor the grid of the wrfout file.

I have looked the code and I find two curious things:

(1) in ll2xy_merc.f90 it uses USE const, ONLY: rearth but in the const_mod.f90 rearth isn't defined. I dont know where it extracts the rearth.

(2) the ll2xy_merc (phi, lambda, lambda0, xx, yy) subroutine only only has lambda0 (center longitude [deg]) as projection argument, but in the documentation that I cited above i have $P_ALP and $P_GAM that represents the latitude of true scale and longitude of the central meridian respectively (according the ioapi documentation ).

Any help would be appreciated

I leave here mi griddesc file and the PROJ srs definition that I am trying to use:
GRIDDESC:

' '
'PAPILA'
  7         0.000         0.000       -61.000       -61.000         0.000
' '
'PAPILAGRID'
'PAPILA'  -3510000.000  -7631603.000     27000.000     27000.000  260  337    1
' '

SRS of Input Grids: epsg:4326
SRS of Output Grids: +proj=merc +lat_ts=0.000 +lon_0=-61.000 +a=6370000.0 +b=6370000.0 +units=m

Errors with Hg Bidirectional Flux Simulations

Description

CMAQ simulations for mercury with the CTM_HGBIDI option set to T will fail due to two issues in the current CMAQ code except for when using the STAGE dry deposition option for a single day simulation. The first issue affects only the M3Dry dry deposition option and causes a model crash if CTM_HGBIDI is set to T but CTM_ABFLUX (bidirectional flux option for NH3) is set to F. The second issue affects both the M3Dry and STAGE dry deposition options and leads to a model crash on the second day of the simulation when using a MEDIA_CONC restart file.

Scope and Impact

The current CMAQ code with not complete successfully for mercury simulations with the CTM_HGBIDI option set to T for multi-day simulations for both the M3Dry and STAGE dry deposition options. For M3Dry, the code will not complete successfully even on the first day of the simulation unless the CTM_ABFLUX is also set to T.

Solution

The first issue is caused by the land use fraction array needed by the Hg bidirectional calculations not being allocated and assigned a value in the M3Dry version of ASX_DATA_MOD.F. To fix this issue, change these statements on lines 557 and 638 from

   If ( ABFLUX) Then

To

   If ( ABFLUX .or. HGBIDI ) Then 

The second issue is caused by subroutine INIT_BIDI in BIDI_MOD.F being called twice in both M3Dry and STAGE, leading to a duplicate allocation error. To fix this issue, the beginning of that subroutine should be modified as follows:

Current code (lines 75 – 80 in the M3Dry version of BIDI_MOD.F):

     Integer         :: STATUS
     
    C--------------------------------------------------------------------------
     ! Set Mercury BiDi Processing Flag equal to false if there is
     ! no mercury gas-phase species.
     If ( INDEX1( 'HG', N_GC_DDEP, GC_DDEP ) .EQ. 0 ) HGBIDI = .FALSE.

Updated code:

     Integer         :: STATUS
     Logical, SAVE   :: INITIALIZED = .FALSE.

     C--------------------------------------------------------------------------
     C Prevent initilializing the code twice 
     If ( INITIALIZED ) Return 

     INITIALIZED = .TRUE.

     ! Set Mercury BiDi Processing Flag equal to false if there is
     ! no mercury gas-phase species.
     If ( INDEX1( 'HG', N_GC_DDEP, GC_DDEP ) .EQ. 0 ) HGBIDI = .FALSE.

For STAGE, corresponding changes would have to be made in lines 73 – 76 of the STAGE version of BIDI_MOD.F

Additional context

These issues will be fixed in the next release of CMAQ.

Remove MPI library paths

Remove MPI library paths from scripts

Excessive/Redundant Logging from the CCTM

When CMAQ is run in MPI mode, the standard output and error receive logging from all processors simultaneously. While the CTM_LOG files give logging information from each processor individually, the standard output/error gets a jumbled dump of information that is difficult to decipher when there are more than a couple of processors being used. Runs with a large number of processors produce very large and somewhat unreadable logs. Suggested fix is to either add a flag to disable logging to the standard error/output or to somehow limit to this output to only a single processor, like maybe the master node.

The CTM also outputs a lot of redundant logging information between the CTM_LOG files and the standard output. I also wonder how much the run times could be improved by giving the option to either disable or limit the information written to the logs.

Inconsistent IOAPI module directory requirement vs error message in config_cmaq.csh

The check for m3utilio.mod Fortran module file is:

 if ( ! -e $IOAPI_DIR/modules/m3utilio.mod ) then 
    echo "ERROR: $IOAPI_DIR/include/m3utilio.mod does not exist in your CMAQ_LIB directory!!! Check your installation before proceeding with CMAQ build."
    exit
 endif

The "if" statement checks for $IOAPI_DIR/modules/m3utilio.mod while the error message mentions $IOAPI_DIR/include/m3utilio.mod

And earlier in the file:

 if ( ! -d $IOAPI_DIR ) then 
    mkdir $IOAPI_DIR
    ln -s $IOAPI_MOD_DIR  $IOAPI_DIR/modules
    ln -s $IOAPI_INCL_DIR $IOAPI_DIR/include_files
    ln -s $IOAPI_LIB_DIR  $IOAPI_DIR/lib
 endif

it sets the IOAPI_INCL_DIR to yet a different path: $IOAPI_DIR/include_files

bldit_project.csh Updates

Resolve location of a couple of PREP directories in this script:

Fix the ICON, BCON CMAQ_REPO locations. The directories CMAQ_REPO/PREP/ic/ICON and CMAQ_REPO/PREP/bc/BCON do not exist.

Replace: CMAQ_REPO/PREP/ic/ICON with CMAQ_REPO/PREP/icon
Replace: CMAQ_REPO/PREP/ic/BCON with CMAQ_REPO/PREP/bcon

Remove this whole section: #> Copy LTNG scripts as there are no longer lightning processing scripts needed in the CMAQ package. Regridding of these files is done outside of CMAQ.

Naming of NH3 flux components when post-processing with the combine utility

CMAQv5.3.1-i6
Date: 2020-03-09
Contact: Christian Hogrefe ([email protected])

Description: In CMAQv5.3.1, there is an inconsistency in the naming convention of the bi-directional NH3 flux components contained in the CCTM_DRYDEP file between the STAGE and M3DRY dry deposition options. When running CMAQ with the NH3 bi-directional flux option enabled (CTM_ABFLUX set to Y in the run script), the model calculates and stores three NH3 flux components in the CCTM_DRYDEP file: the downward deposition flux, the upward emissions flux, and the net flux representing the difference between the deposition and emissions fluxes. The table below shows how these flux components are named in the CCTM_DRYDEP file depending on whether the STAGE or M3DRY dry deposition option is used.

	STAGE	M3DRY
Downward Deposition Flux (always positive)	NH3	NH3_DDEP
Upward Emissions Flux (always positive)	NH3_Emis	NH3_EMIS
Net Flux (positive if downward and negative if upward)	NH3_Flux	NH3

When using the STAGE dry deposition option with bi-directional NH3 flux enabled, the CCTM_DRYDEP file contains additional diagnostic deposition values (NH3_Stom, NH3_Cut, NH3_Soil, NH3_Ag, NH3_Nat, and NH3_Wat) that are described in the CMAQv5.3.1 Release Notes.

Note that when the model is run without the bi-directional NH3 flux option enabled (CTM_ABFLUX set to N in the run script), the variable NH3 in the CCTM_DRYDEP file represents the unidirectional NH3 dry deposition flux in both STAGE and M3DRY.

Scope and Impact: These differences in naming conventions need to be considered when performing post-processing of CCTM_DRYDEP outputs. For example, the “combine” example species definition file for deposition fields utilizes the CCTM_DRYDEP variable “NH3” when defining variables “DDEP_NH3” and “DDEP_NHX” (see for example see the CB6r3_AE7 species definition file under CCTM/src/MECHS/cb6r3_ae7_aq).

Solution: If the intent of these derived fields is to represent downward deposition fluxes, no changes are needed if CMAQ was run without the bi-directional NH3 flux option with either STAGE or M3DRY, or if CMAQ was run with the bi-directional NH3 flux option and the STAGE dry deposition option. However, if CMAQ was run with the bi-directional NH3 flux option and the M3DRY dry deposition option, these definitions in the species definition file should be changed from “NH3” to “NH3_DDEP”.

CCTM crashes for non-hourly BNDY_CONC_1 files

Description

The current implementation of the Centralized Input/Output Module only supports BNDY_CONC_1 files that have a time step of 1 hour. Previous versions of CMAQ were able to process boundary condition files with time steps greater than 1 hour.

Scope and Impact

If providing a boundary condition file with a time step greater than 1 hour, the model will terminate execution with an error message that the BNDY_CONC_1 cannot be read for the requested time step.

Solution

Replace CCTM/src/cio/centralized_io_module.F file in repository with the version located in the folder DOCS/Known_Issues/CMAQv5.3-i2. This code fix will also be included with the next minor CMAQ release.

ISAM predictions of secondary gas species

CMAQv5.3.1-i1:
Date: 12/19/2019 Contact: Sergey Napelenok ([email protected])

Description
CMAQ-ISAM overcontributes apportionment of secondary gaseous pollutants to the ICON and BCON tags. As the result, contributions to other tracked sources is underestimated.

Scope and Impact
All secondary gas-phase species (O3, NOx, etc.) in CMAQ-ISAM v5.3 and v5.3.1.

Solution
A modification to the ISAM chemistry algorithms is currenlty in testing. It will be released in the near future.

Instructions to install IO API are incorrect

The instructions for IO API on this page
https://github.com/USEPA/CMAQ/blob/master/DOCS/Users_Guide/Tutorials/CMAQ_UG_tutorial_build_library_gcc.md
are just plain wrong.

I don't know what they are supposed to be, but if you
mkdir $BIN
and then
cd $BIN

you won't have any Makefile in that directory to run "make" on...

Nor are there any Makefile in the "ioapi" directory either.

There are tons of Makefile... but none called "Makefile", which is the only default target for the "make" command.