atmoschem/eixport

Export Emissions to Atmospheric Models

eixport

Exporting emissions to atmospheric models, eixport: 0.6.0

Emissions are mass that affects atmosphere in complex ways, not only physical, but also, in the health of humans, ecosystems, economically, etc.

There are several models whose inputs are emissions, such as R-Line or WRF-Chem. This R-Package provide functions to read emissions from VEIN and from other models in different formats and export the emissions into the appropriate format suitable to other models.

Install

To install the CRAN version:

install.packages("eixport")

To install the development version:

devtools::install_github("atmoschem/eixport")

Some functions:

• get_edgar: Download EDGAR emissions data.
• to_rline: Export emissions to other formats
• to_wrf: Combine total/spatial/temporal/split and write emission to file
• to_brams_spm: inputs for SPM BRAMS
• wrf_profile: Create spatial profile for WRF-Chem
• wrf_create: Create emission files to the WRF-Chem
• wrf_plot: simple but useful plot
• wrf_get: Read variables
• wrf_put: Write variables
• to_as4wrf: Create WRF-Chem inputs using NCL scrip AS4WRF.ncl.
• to_munich: To generate inputs for MUNICH model.

Summary

library(eixport)
#> The legacy packages maptools, rgdal, and rgeos, underpinning the sp package,
#> which was just loaded, will retire in October 2023.
#> Please refer to R-spatial evolution reports for details, especially
#> https://r-spatial.org/r/2023/05/15/evolution4.html.
#> It may be desirable to make the sf package available;
#> package maintainers should consider adding sf to Suggests:.
#> The sp package is now running under evolution status 2
#>      (status 2 uses the sf package in place of rgdal)
file = paste0(system.file("extdata", package = "eixport"),"/wrfinput_d02")
wrf_summary(file = file)
#>   |                                                                              |                                                                      |   0%  |                                                                              |=======================                                               |  33%  |                                                                              |===============================================                       |  67%  |                                                                              |======================================================================| 100%
#>                Min.       1st Qu.        Median          Mean       3rd Qu.
#> Times  1.312178e+09  1.312178e+09  1.312178e+09  1.312178e+09  1.312178e+09
#> XLAT  -2.438538e+01 -2.405025e+01 -2.370471e+01 -2.370379e+01 -2.335773e+01
#> XLONG -4.742899e+01 -4.696930e+01 -4.650305e+01 -4.650304e+01 -4.603427e+01
#>                Max.        sum
#> Times  1.312178e+09         NA
#> XLAT  -2.301877e+01  -76160.28
#> XLONG -4.558643e+01 -149414.28

Attributes as data.frame

file = paste0(system.file("extdata", package = "eixport"),"/wrfinput_d02")
f <- wrf_meta(file)
names(f)
#> [1] "global" "vars"
head(f$global)
#>                          att                                       vars
#> 1                      TITLE  OUTPUT FROM REAL_EM V3.9.1.1 PREPROCESSOR
#> 2                 START_DATE                        2011-08-01_00:00:00
#> 3      SIMULATION_START_DATE                        2011-08-01_00:00:00
#> 4   WEST-EAST_GRID_DIMENSION                                         64
#> 5 SOUTH-NORTH_GRID_DIMENSION                                         52
#> 6  BOTTOM-TOP_GRID_DIMENSION                                         35
head(f$vars)
#>    vars MemoryOrder                 description        units stagger FieldType
#> 1  XLAT          XY LATITUDE, SOUTH IS NEGATIVE degree north               104
#> 2 XLONG          XY LONGITUDE, WEST IS NEGATIVE  degree east               104

Paper on Journal of Open Source Software (JOSS)

https://doi.org/10.21105/joss.00607

@article{eixport,
    title = {eixport: An R package to export emissions to atmospheric models},
    journal = {The Journal of Open Source Software},
    author = {Sergio Ibarra-Espinosa and Daniel Schuch and Edmilson {Dias de Freitas}},
    year = {2018},
    doi = {10.21105/joss.00607},
    url = {http://joss.theoj.org/papers/10.21105/joss.00607},
  }

<script async src="https://badge.dimensions.ai/badge.js" charset="utf-8"></script>

Contributing

Please, read this guide. Contributions of all sorts are welcome, issues and pull requests are the preferred ways of sharing them. When contributing pull requests, please follow the Google’s R Style Guide. This project is released with a Contributor Code of Conduct. By participating in this project you agree to abide by its terms.

eixport's Issues

wrf_create should automatically determine frames_per_auxinput5

add to_smoke.R

Nonexistent Emissions

Some sectors do not emit (or have not been calculated) emissions for certain pollutants, I found this list of exceptions. I found this list of combinations that returned some error.

for v432_AP, .nc format for 2012:
'NH3' && 'FFF'
'NH3' && 'FOO_PAP'
'NH3' && 'IRO'
'NH3' && 'PRO'
'NOx' && 'NMM'
'OC' && 'CHE'
'OC' && 'FOO_PAP'
'OC' && 'IRO'
'OC' && 'NMM'
'PM2.5_bio' && 'CHE'
'PM2.5_bio' && 'FFF'
'PM2.5_bio' && 'FOO_PAP'
'PM2.5_bio' && 'IRO'

for v432_VOC_spec, .nc format for 2012:
'voc1' && 'TNR_Ship'
'voc17' && 'TNR_Ship'
'voc18' && 'TNR_Ship'
'voc19' && 'TNR_Ship'
'voc18' && 'REF'
'voc19' && 'REF'
'voc20' && 'REF'
'voc21' && 'REF'
'voc22' && 'REF'
'voc24' && 'REF'
'PM2.5_bio' && 'NMM'
'PM2.5_bio' && 'PRO'
'PM2.5_fossil' && 'AWB'
'SO2' && 'SWD_INC'

Finding and programming these exceptions in a script takes a long time, it would be interesting to add an option that skips missing links and add an option that does not download files that have already been downloaded.

C-LINE?

It is possible to add sources into Cline cmas. I'm gonna make some tests to create a function that reads morning rush hour traffic siimulation and extrapolate them to 24 hours to produces Annual Average Daily Traffic (AADT) per link. The format of the resulting data-frame must be:

road name	road class	start longitude	start latitude	end longitude	end latitude	state FIPS code	county FIPS code	AADT	MPH	gas car multiplier	gas truck multiplier	diesel car multiplier	diesel truck multiplier
231	4	−9E+11	3.0E+10	−9E+11	3.0E+11	12	5	7700	24	1	1	1	1
231	4	−9E+11	3.0E+11	−9E+11	3.0E+11	12	5	18100	24	1	1	1	1
231	4	−9E+11	3.0E+11	−9E+11	3.0E+11	12	5	18100	24	1	1	1	1

add option in wrf_grid to create grid grom geo_d*.nc

Currently, wrf_grid generates a multipolygon or matrix based on a wrfinput_d*.nc file. It would be nice to add an option to generates this grid based on 'wrfinput', 'geo' or other wrf file.

wrf_put should be able to read spatial grids, list of arrays and other type of spatial information

Classes for emissions Arrays

vein::EmissionsArray is class of Array with dimensions streets x number of vehicles x hours x days.

When working with emissions inputs for air quality models, a required format is an Array with dimensions:
LAT x LONG x VERTICAL LEVEL x TIMES.

It would be good create a class that converts vein::EmissionsArray and other type of objects into this new class, which could be EmissionsModel or EmissionsFlux in order to accomplish the required dimensions.

This class could be SC3 or eventually SC4. It would be a good ideia to take a look into Spatiotemporal tidy arrays for R stars

Check the package with R CMD check --as-cran

@Schuch666 Try:
system("R CMD check --as-cran eixport.tar.gz")

Or in RStudio build configuration, add "--as-cran" in options: Check figure:
https://imgur.com/a/mshHj

New metadata for WRF 4.0 emission files

WRF Error: trying to read metadata
You can try 1) ensure that the input file was created with WRF v4 pre-processors (in progress),
2) use force_use_old_data=T in the time_control record of the namelist.input file (the procedure for now)

add st_to_raster

Add support to CSVY

CSVY is a file format consisting in a YAML header followed by the CSV data. The header would be benefical for exporting data with columns with different units. For instance, exporting a file to_as4wrf in CSVY format would include the units and type of file for each column

Add summary in to_as4wrf (emis_wrf)

atmoschem/vein#34

add to_nicamchem

ncview not reading E_CO from wrf_emissions example

wrfchemis not being read by ncview despite color scale are read:

dir.create("EMISS")
wrf_create(wrfinput_dir = system.file("extdata", package = "eixport"),
           wrfchemi_dir = "EMISS",
           frames_per_auxinput5 = 24)
# get the name of created file
files <- list.files(path = "EMISS",pattern = "wrfchemi",full.names = T)

# open, put some numbers and write
CO <- wrf_get(file = files[1],name = "E_CO")
CO <- CO[,,1]
CO[] = rnorm(length(CO))

perfil <- c(0.010760058, 0.005280596, 0.002883553, 0.002666932,
            0.005781312, 0.018412838, 0.051900411, 0.077834636,
            0.067919758, 0.060831614, 0.055852868, 0.052468599,
            0.050938043, 0.051921718, 0.052756244, 0.052820165,
            0.058388406, 0.072855890, 0.075267137, 0.063246412,
            0.042713523, 0.029108975, 0.022091855, 0.015298458)

wrf_emission(CO,files[1],profile = perfil,names = "E_CO")
}

And now reading wrchemis with ncview

ncview wrfchemi_d01_2011-08-01_00\:00\:00
Error

Add ini_epsg and end_epsg in wrf_grid

add option to derive geometric atributes in to_rline

Currently, attributes in to_rline are entered explicitly. However, it would a good idea to derive them from a spatial object.

add gains

https://www.iiasa.ac.at/web/home/research/researchPrograms/air/ECLIPSEv5a.html

Add function to return number of hours from wrfinput

add to_geojson (or to_json)

The idea would be to read a different type of emissions data and export to JSON efficiently

add to_munich Model of Urban Network of Intersecting Canyons and Highways (MUNICH)

We need eixport::to_munich. Besides, it would be nice to make run an example with raw data and compare with to_rline.

add to_geoschem

Fix get_edgar

As there are many dataset that need to be checked, I'm relating all the issues here #46 #47 and #48

• Check v50_AP
• Check v432_AP
• Check v432
• Check v432_VOC_spec
• Check v432_v4tox2
• Check v432_htav_v2_2

relates #46 #47 #48

speed up wrf_create

add wrf_times

Danie,tu poderia adicionar uma fun;cao para extraer uma serie de tempo POSIXct de um wrfinput ou wrfout por favor??

wrf_times <- function(x, ...)

obrigado

add to_cmaq.R

wrf_plot only plot nivel 1

add to_NHM_Chem

include units in emisco, to_munich and to_rline

Different download paths

I found that some files has different patterns depending on the combination of sector and pollutant like:

downloading BC from NFE sector (v432_AP and .nc for 2012)
do get_edgar:'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/EDGAR/datasets/v432_AP/BC/NFE/v432_BC_2012_IPCC_2C3_2C4_2C5.0.1x0.1.zip'
do site: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/EDGAR/datasets/v432_AP/BC/NFE/v432_NFE_BC_2012_IPCC_2C3_2C4_2C5.0.1x0.1.zip'

downloading BC from TNR_Aviation_LTO sector (v432_AP and .nc for 2012)
do get_edgar:'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/EDGAR/datasets/v432_AP/BC/TNR_Aviation_LTO/v432_BC_2012_IPCC_1A3a_CRS.0.1x0.1.zip'
do site: 'https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/EDGAR/datasets/v432_AP/BC/TNR_Aviation_LTO/v432_BC_2012_IPCC_1A3a_LTO.0.1x.01.zip'

downloading voc1 from the TRF sector (v432_AP and .nc for 2012)
do get_edgar: https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/EDGAR/datasets/v432_VOC_spec/voc1/TRF/v432_VOC_spec_voc1_2012_IPCC_1A1c_1A5b1_1B1b_1B2a6_1B2b5_2C1b.zip
so site: https://jeodpp.jrc.ec.europa.eu/ftp/jrc-opendata/EDGAR/datasets/v432_VOC_spec/voc1/TRF/v432_VOC_spec_voc1_2012_1A1c_1A5b1_1B1b_1B2a6_1B2b5_2C1b.0.1x0.1.zip

I also found some problems downloading the:
'NFE','TNR_Aviation_LTO','TNR_Other' sectors and the 'CO2' variable of the v432_AP dataset
and on the
'TRF' 'TNR_Aviation_LTO' sectors from the v432_VOC_spec dataset

include to_brams_spm

spm_brams needs a function to generate a file with the following characteristic:

daily_emissions, long, lat

One file per pollutant. It could include a list of file and save them with some loop.

It has to include the function to represent the temporal distribution of the daily emissions.

"+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs" or 4326?

Então Daniel, tenho uma duvida, que tentei resolver aqui sem muito éxito:

According to the WRF Manual , page 3-10: "As a general guideline,

• the polar stereographic projection is best suited for high-latitude WRF domains,
• the Lambert conformal projection is well suited for mid-latitude domains, and
• the Mercator projection is good for low-latitude domains or domains with predominantly west-east extent."

Qual é o Sistema de Referencia exacto para cada um? Eu to achando que assumir "+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs" or 4326 para cada um dos casos é um erro.

remove MULTILINESTRING from to_munich

and include a Warning

add to_calpuff

add get_edgar

add function to automatic download edgar emissions

allow different colour pallets in wrf_plot

sería possivel usar outros cores no wrf_plot?

add message with number of columns and rows in wrf_grid

Also, wrf_grid with matrix = T does not return proper matrix

wrf_grid returns a MULTIPOLYGON

wrf_grid returns a MULTIPOLYGON which is not good when intersecting. It needs st_cast to "POLYGON". Also, it needs a field "id" with the first to the last grid cell.

Do we need merging raster functions to include EDGAR?

Spatial distribution with street density

We need to include a spatial distribution with street density. I'm writing this here to remember and then close this issue.

The ideia is one line:

function(emis, network, params, ...)

Suggested edits from Hollister review for JOSS

• R Dependency to 3.3.0
• Check on Authorship - Different author list on paper vs. software.
• Fix example on emis_opt
• Check example on wrf_plot to make sure the warning is not a problem
• Add a separate section to README for installation instructions. A good example is https://github.com/ropensci/lawn
• Add tests. Should have coverage for at least the main functions.
• Add details on how to contribute in a CONTRIBUTING.md
• Provide better justification for the package. In particular, provide more detail on why an R package is needed on top of the already available lower level implementation in Fortran and C.
• Add DOIs to references.
• Review PR #21 for suggested edits
• As a future enhancement, add a vignette showing typical use case

add function to process EDGAR and aggregate it by chemical mechanism

to_aas4wrf needed?

vein::emis_wrf is a function that produces an data-frame formatted to input AS4WRF pre-processorfor WRF-Chem. This functions could be integrated into eixport as to_as4wrf. Reasons:

eixport and EmissV functions creates emissions inputs for WRF-Chem for the with the same size as the domain of the wrfinput.
aas4wrf assigns an emissions grid into the domain.
eixport and EmissV needs solely the wrfinput. Hence, can run in any OS.
aas4wrf needs wrfinput and ncl installed and configurated.

This means that both set of functions achieve the same objective with different approaches. Therefore, both should be here to provide flexibility to the user.

The references of eixport and EmissV are in development.
The reference of aas4wrf is:
Vara-Vela, A., Andrade, M. F., Kumar, P., Ynoue, R. Y., & Muñoz, A. G. (2016). Impact of vehicular emissions on the formation of fine particles in the Sao Paulo Metropolitan Area: a numerical study with the WRF-Chem model. Atmospheric Chemistry and Physics, 16(2), 777-797.

What do you think @Schuch666 ?

Function to process EDGAR, RCP, and other global emissions inventories

we have EmissV::read to read NetCDF Emissions. Perhaps, importing EmissV::read into a new function process this emissions would be a good idea.

Inputs:
path where the emissions are
type of emissions: The category of emissions (for instance, all but Transport)
Pollutant:
grid: for resampling where necessary
profile:. This would imply the length in hours of the output

Output: vein::GriddedEmissionsArray (robject)
specials: if RADM2/CBMZ, it return a several vein::GriddedEmissionsArray written in the disk. This will have some assumptions. This function would have to allow the user to change this assumptions, or return each object and change it manually.

wrf_get, as_raster = T não funciona e variavel tem 4 dimensões

Missing https

I found some errors in the form of

downloading BC from the TNR_Other sector (v432_AP and .nc for 2012)

Error in `$<-.data.frame`(`*tmp*`, "links", value = NA) :
  replacement has 1 row, data has 0
Calls: get_edgar -> $<- -> $<-.data.frame
Execution halted

downloading SO2 from the SWD_INC sector (v432_AP and .nc for 2012)

Error in `$<-.data.frame`(`*tmp*`, "links", value = NA) :
  replacement has 1 row, data has 0
Calls: get_edgar -> $<- -> $<-.data.frame
Execution halted

Test new forms to generate NetCDF files

NetCDF can be generated with ncgen -o sample.nc sample.cdl, with the data in CDL format.
There is also the library C NetCDF and NetCDF Fortran and NetCDF C++. It would be interesting trying to generate and manipulate NetCDF files with a SC3 class on R.

We need a regridding function

This function would allow us to input any gridded inventory and output to our Target grid.

I believe this function have high priority

Add appveyor

need classes!

we need classes!

Add travis

Travis allows to check that package is working and it is very useful