usepa/amet

Code base for the U.S. EPA’s Atmospheric Model Evaluation Tool (AMET)

amet's Issues

AMET on HPC cluster with no MySQL

Hi,
Is there an option to run AMET without having a MySQL connection, maybe by somehow using SQLite when compiling?
If not, any plans in the future to implement this?
Thanks,
Lisa

Update AQ_Soccerplot.R

Minor change to AQ_Soccerplot.R to work with the new credentials:

Change root_login and root_pass to amet_login and amet_pass in the following two lines:

mysql <- list(login=root_login, passwd=root_pass, server=mysql_server, dbase=dbase, maxrec=1000000)
con             <- dbConnect(MySQL(),user=root_login,password=root_pass,dbname=dbase,host=mysql_server)

**Critical** Error in AQ database setup that prevents metadata from loading properly

There is an error (since fixed) in the setup of the AQ site_metadata table that will prevent the site metadata for each network from loading properly. There is a missing column definition, namely "co_network", that when missing causes the MySQL queries to fail, and hence no metadata being loaded in the site_metadata table. This will then prevent database queries from functioning correctly. This error does not affect results when not using the database for AQ analysis.

This issue was fixed by making the simple change below (in bold) to the AQ_setup_dbase.R script.

aq_new_3 <- "create table site_metadata (stat_id varchar(25) UNIQUE KEY, num_stat_id int(10), stat_name varchar(100), network varchar(15), co_network varchar(200), state varchar(4), city varchar(50), start_date varchar(20), end_date varchar(20), lat double, lon double, elevation int(10), landuse varchar(50), loc_setting varchar(50), county varchar(100), country varchar(100), near_road varchar(10), timezone varchar(20), GMT_Offset double, NLCD2011_Imperv_Surf_Frac double,NLCD2011_Imperv_Surf_Loc_Setting varchar(100),NLCD2006_Imperv_Surf_Frac double, NLCD2006_Imperv_Surf_Loc_Setting varchar(100), NLCD2001_Imperv_Surf_Frac double, NLCD2001_Imperv_Surf_Loc_Setting varchar(100))"

I've fixed the code with the above change. If you downloaded the AMET code prior to 11/6/2019, you should either download the full AMET code again, download and replace your existing AQ_setup_dbase.R script in the R_db_code/ directory of your AMET installation, or edit your existing AQ_setup_dbase.R script with the change noted above.

Note that if you've already run the AMET setup, you'll need to either manually delete the "site_metadata" table from your AMET database so that it can be recreated without the column missing, or delete your entire AMET setup and start fresh using the script provided with the AMET code. I would recommend that if you've already done analysis and loaded data into the database that you attempt to manually delete "site_metadata" table and have it recreated automatically when you run the aqExample_post_only.csh script.

A third option would be to simply edit your existing "site_metadata" table in the MySQL database to add the missing column manually. This can be accomplished by logging into MySQL with your username and password, switching to your AMET database, and then editing the existing table. An example is given below.

mysql -h mysql_host_name -u user_name -p
enter password
use AMET_DATABASE;
alter table site_metadata add column co_network varchar(200) after network;
exit

You should then re-run your aqExample_post_only.csh script with the reload_metadata flag set to T. That should then properly load your site metadata.

Wyat

Apparent discrepency between code and documentation for fractional bias and fractional error

The documentation states that the numerator and denominator should be summed separately to calculate fractional bias and error.

However, the code for DomainStats in AQ_Misc_Functions.R appears to calculate these metrics as the sum after the fraction is calculated for each observation/model pair. The code also multiplies the result by a 100 to report as a percentage rather than a fraction.

https://github.com/USEPA/AMET/blob/1.5/R_analysis_code/AQ_Misc_Functions.R#L652

fb <- signif(c(fb,(sum((data.df$mod_val-data.df$ob_val)/((data.df$mod_val+data.df$ob_val)/2)))/length(data.df$stat_id))*100,3)
fe <- signif(c(fe,(sum(abs(data.df$mod_val-data.df$ob_val)/((data.df$mod_val+data.df$ob_val/2)))/length(data.df$stat_id))*100,3)

Am I correct that this is a discrepancy, and can this be updated to be consistent with the correct approach?

Issue with AQ_Stacked_Barplot_AE6.R

Currently there is an issue with the AQ_Stacked_Barplot_AE6.R script that results in incorrect bars being plotted. This is likely due to an error in the computation of PM_other in the script and the aggregation function being applied. Users should not use this script until the issue is resolved.

Users are advised to use the other stacked bar plotting scripts available, namely the AQ_Stacked_Barplot_AE6_ggplot.R and AQ_Stacked_Barplot_AE6_plotly.R scripts. These scripts appear to be functioning correctly.

This issue will be updated when the problem with the script is fixed.

Units Update

The update mode for the AQ database scripts doesn't appear to work as expected. Once a project has been created, the units table does not get updated correctly. It actually looks like it's being clobbered (networks deleted) when you try to update the table.

When I grep on the log from the aqProject.csh script, run in update mode, I see this message (check to see which networks are being loaded into the db:

Finshed populating database for network  IMPROVE 
Finshed populating database for network  CSN 
Finshed populating database for network  CASTNET 
Finshed populating database for network  CASTNET_Hourly 
Finshed populating database for network  CASTNET_Daily 
Finshed populating database for network  CASTNET_Drydep 
Finshed populating database for network  NADP 
Finshed populating database for network  MDN 
Finshed populating database for network  AMON 

When I query the database, I'm only seeing that the last network processed exists in the units table:

mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| amet_example       |
| amet_iwdw          |
| mysql              |
| performance_schema |
+--------------------+
5 rows in set (0.00 sec)

mysql> use amet_iwdw  ;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+----------------------+
| Tables_in_amet_iwdw  |
+----------------------+
| WAQS_Base11b         |
| WAQS_Base11b_CAMx12k |
| WAQS_Base11b_CAMx4k  |
| aq_project_log       |
| project_units        |
| site_metadata        |
+----------------------+
6 rows in set (0.00 sec)

mysql> select network from project_units;
+---------+
| network |
+---------+
| AMON    |
+---------+
1 row in set (0.00 sec)

This is a problem because the units are used in the plotting scripts for various conditionals. If the units are NULL in the database, the plotting scripts don't work.

Cannot build bldoverlay, sitecmp, and sitecmp_dailyo3.

According to the AMET_Install_Guide_v14b under the section Install AMET Source Code and Tier 3 Software there should be a src directory containing the programs bldoverlay, sitecmp, and sitecmp_dailyo3. The src directory appears to be missing in the 14b release. There is a directory called tools_src which seems to have the source for these programs and the config.amet file. However, they are missing any make files to build bldoverlay, sitecmp, and sitecmp_dailyo3.

Bug in Network.input

The setting of network_names to NULL in the script Network.input (line 41) breaks the batch scripts. The Network.input script sets the network names based on if (inc_improve=="y") type conditionals. These types of conditionals don't get set in the batch mode of the scripts and so initializing these vectors to NULL breaks the batch scripts because they never get set back to actual network names after the Network.input script is sourced. The offending "source" command that unsets these variables happens at line 1174 of R_analysis_code/AQ_Misc_Functions.R.

Suggested fix is to wrap these lines in Network.input in a conditional to ignore them if the scripts are in batch mode:

if ( batch != 'y' ){
network_names <- NULL
network_label <- NULL
}

Enhancement Request: Species <-> Networks and Error Handling

AMET result in a nebulous error when you try to plot a species from a network that doesn't include said species. The error may look something like this:

Error in `$<-.data.frame`(`*tmp*`, "good_ob", value = 0) :
  replacement has 1 row, data has 0
Calls: Average -> $<- -> $<-.data.frame
Execution halted

The system would be much more user friendly if (1) there was a sanity check on the species (AMET_AQSPECIES) and network (e.g. AMET_CSN) pairs; and (2) a table (maybe a new Appendix in the UG) that shows what species are available for each network. When the check is written for (1), there would be an informative error that says something like "AMET_AQSPECIES O3_8hrmax is not available in the network AMET_CSN; please refer to Table XXX in the AMET user's guide to see the list of species available for each network in the AMET database."

AMET Example Plots

Add a directory of all example plots to the user's guide. Reference these examples in the text of the guide.

2015/16 Obs Data

Need to add Obs data for 2015/16 to the CMAS data clearinghouse. Currently we have data for 2001-2014: https://drive.google.com/drive/folders/0B2kjxCwKICxUZ3ZyNEVld2F6bHc

AQ Batch Scripts

There are settings missing from AMET_batch.input that result in failure of the scatter density plots. Add these fields to the release script:

density_zlim<-NULL
num_dens_bins<-NULL
bin_by_mod<-""

The new run_AMET_batch.csh script is looking for:
setenv AMETRINPUT $AMETBASE/scripts_analysis/aqExample/input_files/AMET_batch.R

I'm only seeing AMET_batch.input in the input_files directory. It seems to work to use the *.input file. Is this just a type in the repo distribution?

Met Tutorial Data and MADIS obs

• Add WRF and MPAS tutorial data to the official release
• Either add an auto download script for MADIS data, or include instructions in the user's guide for how to download the MADIS obs

Documentation

• Update User's Guide TOC once the content is near final
• Clean up headings in the user's guide to render correctly in Git Hub
• Add AMET Met instructions in user's guide
• Add Installation Guide from UNC

Login/Pass

New unified met/aq scripts use a single database script to create projects (if needed) and load data. There is a way to include your login and password as variables in the script, otherwise, if these variables aren't set the database load script asks for your login info upon execution.

should we include login and password env vars as examples/optional in the script (e.x., aqProject.csh) to bypass the interactive mode?

how do these variables connect to the amet-config.R scripts? Is there a way to reference that centralized configuration script from these database loading scripts?

Version 1.3 Final Testing

The documentation for v1.3 (Section 6.2) refers to a combine subdirectory under $AMETBASE/scripts_db/aqExample. Is this directory missing in the installation or should I omit this reference in the documentation?

site_metadata_files

Add $AMETBASE/obs/site_metadata_files to the distribution rep.