Inconsistency between GUI and non-interactive behaviour in AOI specification

[ropensci]

Inconsistency between GUI and non-interactive behaviour in AOI specification

[lbusett]

Thanks for reporting this. I think that the problem may be that you are specifying the bbox in lat/lon coordinates, while keeping the output projection in sinusoidal. Is that the case ?

If so, could you try specifying the bbox in sinusoidal projection and see if it works ? Or also, specifying the bbox as is, and changing the output projection to lat/lon ?

If that solves the issue, I recon however that we should emphasize more that the bbox should be provided in coordinates of the output projection.

[joelfiddes]

thanks for quick response! I dont think it solves it though, both are specified as lat/long, options.json here as an example. Note tile coords and bbox dont intersect:

{
"sel_prod": "Vegetation Indexes_16Days_250m (M*D13Q1)",
"sensor": "Terra",
"prod_version": "6",
"start_date":"2016-08-12",
"end_date":"2016-08-12",
"bandsel": [ 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
"indexes_bandsel": [ 0, 0, 0, 0 ],
"quality_bandsel": [ 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
"start_x": 15,
"end_x": 15,
"start_y": 4,
"end_y": 4,
"user": "xxxxx",
"password": "xxxxxx",
"use_aria": false,
"download_server": "http",
"download_range": "full",
"proj": "Latlon WGS84",
"user_proj4": "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0",
"out_res_sel": "Native",
"out_res": "native",
"full_ext": "Resized",
"resampling": "near",
"out_format": "GTiff",
"ts_format": "None",
"rts": "Yes",
"compress": {
"None": "None"
},
"nodata_change": "No",
"scale_val": "No",
"delete_hdf": "No",
"reprocess": "Yes",
"bbox": [7.9999, 46.125, 8.6249, 46.875],
"out_folder":"/home/joel/sim/test2/grid1/MODIS/NDVI",
"out_folder_mod":"/home/joel/sim/test2/grid1/MODIS/NDVI",
"MODIStspVersion": "1.3.2",
"custom_indexes": []
}

[joelfiddes]

Apologies, the options above don't seem to be valid for that date. Below is a reproducible example (with pwd/user/outDirPaths) the downloaded HDF will be MODIS tile H19/V4 and the processed tif will have a footprint of bbox but with no data as bbox and x/y parameters don't intersect. If I do the same in the GUI it will automatically download H18V4 which contains my bbox and therefore give valid data in the processed tif.

{
"sel_prod": "Vegetation Indexes_16Days_250m (M*D13Q1)",
"sensor": "Terra",
"prod_version": "6",
"start_date": "2008-08-12",
"end_date": "2008-08-12",
"bandsel": [ 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
"indexes_bandsel": [ 0, 0, 0, 0 ],
"quality_bandsel": [ 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
"start_x": 19,
"end_x": 19,
"start_y": 4,
"end_y": 4,
"user": "xxxx",
"password": "xxxx",
"use_aria": false,
"download_server": "http",
"download_range": "full",
"proj": "Latlon WGS84",
"user_proj4": "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0",
"out_res_sel": "Native",
"out_res": "native",
"full_ext": "Resized",
"resampling": "near",
"out_format": "GTiff",
"ts_format": "None",
"rts": "Yes",
"compress": {
"None": "None"
},
"nodata_change": "No",
"scale_val": "No",
"delete_hdf": "No",
"reprocess": "Yes",
"bbox": [7.9999, 46.125, 8.6249, 46.875],
"out_folder":"xxxx",
"out_folder_mod":"xxxx",
"MODIStspVersion": "1.3.2",
"custom_indexes": []
}

[lbusett]

Ok. Thanks for providing the JSON: it should speed-up checking the issue. We'll try tomorrow to run MODIStsp with that json and see what's happening.

Lorenzo

[lbusett]

One quick question: the json you provided was saved from the GUI after inputting the "non intersecting" coordinates and clicking "Retrieve Tiles from Bounding box" ?

[joelfiddes]

Thanks again for looking into this! No, I have a template json (which I originally saved from the gui) then I use some grep/sed to set the lon/lat coords on each call to MODIStsp(). I hadn't noticed this button to be honest - and have never activated it. However, I just did an experiment: (1) loaded my template json (above) (2) clicked the "Retrieve Tiles from Bounding box" button and resaved this json. Results below of this new json, compare to template above, no difference I think, I don't see a keyword that reflects this button:

{
"sel_prod": "Vegetation Indexes_16Days_250m (M*D13Q1)",
"sensor": "Terra",
"prod_version": "6",
"start_date": "2008-08-12",
"end_date": "2008-08-12",
"bandsel": [ 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
"indexes_bandsel": [ 0, 0, 0, 0 ],
"quality_bandsel": [ 0, 0, 0, 0, 0, 0, 0, 0, 0 ],
"start_x": 18,
"end_x": 18,
"start_y": 4,
"end_y": 4,
"user": "xxx",
"password": "xxxx",
"use_aria": false,
"download_server": "http",
"download_range": "full",
"proj": "Latlon WGS84",
"user_proj4": "+init=epsg:4326 +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0",
"out_res_sel": "Native",
"out_res": "native",
"full_ext": "Resized",
"resampling": "near",
"out_format": "GTiff",
"ts_format": "None",
"rts": "Yes",
"compress": {
"None": "None"
},
"nodata_change": "No",
"scale_val": "No",
"delete_hdf": "No",
"reprocess": "Yes",
"bbox": [ 7.9999, 46.125, 8.6249, 46.875 ],
"out_folder": "xxxx",
"out_folder_mod": "xxxx",
"MODIStspVersion": "1.3.2",
"custom_indexes": []
}

[lbusett]

a couple of things:

1. I tried on my machine, and if I specify a non-overlapping bbox, cloick on "Retreive tiles from bounding box" and then save the JSON, the saved json has the correct tiles. The same happens if I save the file without clicking the button: I am asked if to correct the tiles, and if I press "Yes", then the tiles are correctly updated in the JSON . Could you confirm this behaviour ?

2. Besides this, what you are trying to do (modifying the bbox by rewriting only the bbox of the json) is not supported. If you want to proceed like that, you should modify both the bbox AND the start_x, end_x, etcetera. We have helper functions in MODIStsp that do that automatically before saving the json, but they are not currently exported. I could maybe "extract" them from the code base and send them to you if you want.

3. As an allternative, you could pass your extent to MODIStsp using a "spatial file" with the desired extent (e.g., a shapefile). See:

http://lbusett.github.io/MODIStsp/articles/noninteractive_execution.html#looping-on-different-spatial-extents

Just create a SpatialPolygons object of your extent, save it as shapefile with writeOGR and pass the filename in the call to MODIStsp:

MODIStsp(gui = FALSE, options_file = options_file, spatial_file_path = my_shape_filename )

4. Notice that you are probably using an old versin. Current MODIStsp version is 1.3.3.1

Lorenzo

[joelfiddes]

1. Yes I can confirm that works fine
2. If that's no trouble would be great - be a cleaner way of updating other parameters like start/ end date compared to my sed/grep hacks.
3. I'm having a gdal problem there, although its a gdal problem and separate issue (some details below mainly for my reference).

I think we can close this issue now - Thanks for all your help!

===============================================================
GDAL version in use:1.11.3
ERROR 4: `/home/joel/e.shp' not recognized as a supported file format.
gdalinfo failed - unable to open '/home/joel/e.shp'.

While my shape opens fine in QGIS and imports to R and plots fine also:

Thats how i generate the shape:

e <- as(raster::extent(rst), "SpatialPolygons")

e
class : SpatialPolygons
features : 1
extent : 9.375139, 10.12486, 46.12514, 46.87486 (xmin, xmax, ymin, ymax)
coord. ref. : NA
proj4string(e) <- "+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs"
e
class : SpatialPolygons
features : 1
extent : 9.375139, 10.12486, 46.12514, 46.87486 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs +towgs84=0,0,0

shapefile(e, "~/e.shp", overwrite=T)

[lbusett]

Wild shot: try making a SpatialPolygonsDataFrame instead than a simple SpatialPolygons.

[lbusett]

I'll try to send you the "extracted" routines next week. Closing this now.