cfranken/gridding

Generic gridding routine for satellite data with corner coordinates supplied. in the Scientific Development Phase, use at your own risk and let authors know about issues (quick&dirty was the way!)

gridding's Issues

Gridding TROPOMI multi-band data returns nan

Hello, thank you very much for developing this tool, it helps me gridding very well!
But some problems occurred when I gridded the TOA_RFL field in TROPOMI. TOA_RFL is a 7-band data with a dimension of 73092691. When I gridded the field, the results returned were all nan, but When processing a single-band array, such as 13092691, the correct grid can be achieved. My command is as follows: julia ./gridL2_Dates.jl --dLat 0.05 --dLon 0.05 --dDays 1 --startDate 2020-01- 01 --stopDate 2020-01-31 -o E:\SIF\06_data_process\SIF\TROPOMI_L2_TOA_REF_gridding\2020\TROPOMI_TOA_REF_L2_gridded_202001.nc
What should I do to correctly grid the 7-band data? I look forward to your reply, thank you!

Small readme mistake

Great module! Just starting to use.

I think there's a typo and:
alias julia /Applications/Julia-1.3.app/Contents/Resources/julia/bin/julia
should be
alias julia=/Applications/Julia-1.3.app/Contents/Resources/julia/bin/julia

:)

Output all NAN values

Hi, I'm getting all the output values in NAN. Everything runs fine - it reads the files and output the product with all NAN values...

json file:

{
"basic":{
"lat": "PRODUCT/latitude",
"lon": "PRODUCT/longitude",
"lat_bnd": "PRODUCT/SUPPORT_DATA/GEOLOCATIONS/latitude_bounds",
"lon_bnd": "PRODUCT/SUPPORT_DATA/GEOLOCATIONS/longitude_bounds",
"time": "PRODUCT/time_utc"
},
"grid":{
"ch4": "PRODUCT/methane_mixing_ratio_bias_corrected",
"altitude": "PRODUCT/SUPPORT_DATA/INPUT_DATA/surface_altitude"
},
"filter_gt":{
"PRODUCT/qa_value": 0.3,
"PRODUCT/methane_mixing_ratio_bias_corrected": 1600
},
"filter_lt":{
"PRODUCT/methane_mixing_ratio_bias_corrected": 2200
},
"filePattern": "S5P__L2__CH4____YYYYMMDD.zip",
"folder": "/net/fluo/data2/projects/TROPOMI_GASES/ch4/"
}

command:

julia ./gridL2_Dates.jl --dLat 0.05 --dLon 0.05 --latMin 25 --latMax 45 --lonMin -125 --lonMax -60 --dDays 1 --startDate 2018-05-01 --stopDate 2018-05-03 --Dict ./tropomi_sif.json -o /local/workdir/lzq/NSF_SIF/Data/TROPOMI/tropomi_daily_0.05deg_test.nc

Thanks if someone can help me! :D

defVar error

The line 342 in gridL2_Dates.jl:
dN = defVar(dsOut,"n",Float32,("time","lon","lat"),deflatelevel=4, fillvalue=-999, units=" ", long_name="Number of pixels in average") #,

The error appears as:
ERROR: LoadError: MethodError: no method matching defVar(::NCDataset{Nothing, Missing}, ::String, ::Type{Float32}, ::Tuple{String, String, String}; deflatelevel=4, fillvalue=-999, units=" ", long_name="Number of pixels in average")
Closest candidates are:
defVar(::NCDataset, ::Union{AbstractString, Symbol}, ::DataType, ::Any; chunksizes, shuffle, deflatelevel, checksum, fillvalue, nofill, typename, attrib) at /home/zl725/.julia/packages/NCDatasets/ivzVX/src/cfvariable.jl:99 got unsupported keyword arguments "units", "long_name"
defVar(::CommonDataModel.AbstractDataset, ::Union{AbstractString, Symbol}, ::DataType, ::Any) at /home/zl725/.julia/packages/CommonDataModel/GGvMn/src/variable.jl:33 got unsupported keyword arguments "deflatelevel", "fillvalue", "units", "long_name"
defVar(::CommonDataModel.AbstractDataset, ::Union{AbstractString, Symbol}, ::Any, ::Any; kwargs...) at /home/zl725/.julia/packages/CommonDataModel/GGvMn/src/variable.jl:108

And it works out when I just remove the units=" ", long_name="Number of pixels in average" in the function

Zhenqi

Specific formats needed for lat/long/time

Hi, I'm getting some errors thrown that I have difficulty troubleshooting.

e = NCDatasets.NetCDFError(-43, "NetCDF: Attribute not found")
Couldn't write attrib valid_range
e = NCDatasets.NetCDFError(-43, "NetCDF: Attribute not found")
Couldn't write attrib description
e = NCDatasets.NetCDFError(-43, "NetCDF: Attribute not found")
Couldn't write attrib unit
e = NCDatasets.NetCDFError(-43, "NetCDF: Attribute not found")
Couldn't write attrib longname

After some trouble shooting, I think this may have to do with the formatting of lat/long/time variables in the basic grid.
Could you tell me if a specific data format or unit is needed for these 3 variables? My suspicion is that it may not working with the date (time stamp format) or time (units in seconds since 1970-01-01 00:00:00) format of my OCO3 ncf4 files.

If possible could I obtain the json file, command line, and a data file that works with the routine as it's hard to trouble shoot without a working example at hand?

Thanks! :D

Running for TROPOMI data returns empty arrays

Hello, very nice lib! It will definitely help me a lot in my research.

However, I am facing some issues for creating nc file for tropomi data. I used the following command:

julia ./gridL2_Dates.jl --dLat 1 --dLon 1 --startDate 2019-01-01 --stopDate 2020-01-05 --Dict jsonFiles/tropomi_all.json -o test.nc

The test.nc files were generated sucessfully but it has only 55k and when I try to open it with python xarray's it seems to be empty:

ds.sif[0]

<xarray.DataArray 'sif' (lon: 360, time: 47)>
array([[nan, nan, nan, ..., nan, nan, nan],
       [nan, nan, nan, ..., nan, nan, nan],
       [nan, nan, nan, ..., nan, nan, nan],
       ...,
       [nan, nan, nan, ..., nan, nan, nan],
       [nan, nan, nan, ..., nan, nan, nan],
       [nan, nan, nan, ..., nan, nan, nan]], dtype=float32)
Coordinates:
    lat      float32 -89.5
  * lon      (lon) float32 -179.5 -178.5 -177.5 -176.5 ... 177.5 178.5 179.5
  * time     (time) datetime64[ns] 2019-01-01 2019-01-09 ... 2020-01-04

Did I do something wrong?