rtcApp.py is incomplete

[isce-framework]

rtcApp.py is incomplete

[sgk0]

Hi! Glad to hear that it is working.

The "too many points failed to transform" warnings are related to GDAL. I also get the same error, I'm not sure what to do about it either - it may be a limitation of the geolocation array approach. But I haven't seen any trouble in the resulting tiff files, so I think you can ignore it.

As to the "free disk error", I'm also not surprised - because you said you want to use a geocode spacing = 1, which will take up about 100x more disk space than the raw image.

Please keep in mind: posting should generally be smaller or the same as the geocode spacing, but not greater. By setting posting to 100, the image is multi-looked (here, 10x in azimuth and 10x in range) to go from the 10 m x 10 m to 100 m x 100 m. The purpose of this step is to reduce the disk space and computing time needed for geocoding the image.

[hbparache]

@piyushrpt, yes, if you could retire #119 that would be great,. Thanks!

[piyushrpt]

That is expected. The output at 10x10 is 100 times larger than 100x100. A lot more interpolations need to happen - 4 times is better than 100 times which is the size of the output.

[sgk0]

We haven't added the geocode step as part of the rtcApp.py workflow yet. We've been using a geolocation array approach outside of the workflow, as in this example: https://github.com/isce-framework/isce2-docs/blob/master/Notebooks/GDAL/03_RasterProjection.ipynb. I'll try to add this step in the near future

I'm not sure about errors in the input file, will look into it.

[piyushrpt]

@HBaldwin3 - #128 has been merged. Could you let us know if this suffices?
@sgk0 - thanks for looking into this promptly.

If everything looks ok, I will retire #119

[shobhitagarwal1612]

Thank you so much for the fix! I tried the latest changes and it runs geocoding step as expected.

When I tried it with spacing 100, I didn't notice any errors/warnings.

But doing the same with spacing 10 generated the following warnings.
(although the geocoded output was produced)

****Geocoding file 1 out of 2: gamma_VV.10alks_10rlks.img.vrt****
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute output bounds.
Warning 1: Unable to compute source region for output window 0,0,3564,2676, skipping.
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute output bounds.
Warning 1: Unable to compute source region for output window 3564,0,3564,2676, skipping.
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute output bounds.
Warning 1: Unable to compute source region for output window 17821,18732,3564,2676, skipping.
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute output bounds.
Warning 1: Unable to compute source region for output window 21385,18732,3564,2676, skipping.
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute output bounds.
Warning 1: Unable to compute source region for output window 24949,18732,3565,2676, skipping.
****Geocoding file 2 out of 2: gamma_VH.10alks_10rlks.img.vrt****
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute output bounds.
Warning 1: Unable to compute source region for output window 0,0,3564,2676, skipping.
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute output bounds.
Warning 1: Unable to compute source region for output window 3564,0,3564,2676, skipping.
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute output bounds.
Warning 1: Unable to compute source region for output window 17821,18732,3564,2676, skipping.
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute output bounds.
Warning 1: Unable to compute source region for output window 21385,18732,3564,2676, skipping.
ERROR 1: Too many points (529 out of 529) failed to transform, unable to compute output bounds.
Warning 1: Unable to compute source region for output window 24949,18732,3565,2676, skipping.
2020-05-07 15:02:06,908 - isce.grdsar - INFO - Total Time: 431 seconds

And setting spacing as 1 crashes the geocoding step because of the huge space requirements

****Geocoding file 1 out of 2: gamma_VV.10alks_10rlks.img.vrt****
ERROR 3: Free disk space available is 151300382720 bytes, whereas 244175362812 are at least necessary. You can disable this check by defining the CHECK_DISK_FREE_SPACE configuration option to FALSE.
****Geocoding file 2 out of 2: gamma_VH.10alks_10rlks.img.vrt****
ERROR 3: Free disk space available is 151300382720 bytes, whereas 244175362812 are at least necessary. You can disable this check by defining the CHECK_DISK_FREE_SPACE configuration option to FALSE.
2020-05-07 18:48:24,795 - isce.grdsar - INFO - Total Time: 54 seconds

Is this expected?

Config:

<rtcApp>
  <constant name="dir">/usr/local/google/home/ashobhit/isce2</constant>
  <component name="rtcApp">
    <property name="sensor name">sentinel1</property>
    <property name="posting">100</property>
    <property name="polarizations">[VV, VH]</property>
    <property name="epsg id">32645</property>
    <property name="geocode spacing">10</property>
    <property name="geocode interpolation method">bilinear</property>
    <component name="master">
      <property name="safe">$dir$/S1A_IW_GRDH_1SDV_20200320T121408_20200320T121433_031759_03A9BC_4532.zip</property>
      <property name="orbit directory">$dir$/orbits</property>
      <property name="output directory">$dir$/sentinel_output</property>
    </component>
  </component>
</rtcApp>

[piyushrpt]

The radar swath is not perfectly oriented N-S/ W-E. The warnings are just GDAL letting you know that on the geocoded grid which is oriented N-S/W-E it did not find any data that could be used to fill up and that it will fill that part with nodata.

[shobhitagarwal1612]

Please keep in mind: posting should generally be smaller or the same as the geocode spacing, but not greater. By setting posting to 100, the image is multi-looked (here, 10x in azimuth and 10x in range) to go from the 10 m x 10 m to 100 m x 100 m. The purpose of this step is to reduce the disk space and computing time needed for geocoding the image.

@sgk0
With spacing 10 and posting 10, it took 1811 seconds.
But with spacing 10 and posting 100, it took 440 seconds.

It was 4 times slower when posting and spacing were same.

[piyushrpt]

Closed by #128