brownag/rsamgeo

R 'reticulate' wrapper for `segment-geospatial` 'Python' package for segmenting geospatial data with the ‘Segment Anything Model’ (SAM).

{rsamgeo}

The goal of {rsamgeo} is to provide a basic R wrapper around the segment-geospatial ‘Python’ package by Dr. Qiusheng Wu. Uses R {reticulate} package to call tools for segmenting geospatial data with the Meta ‘Segment Anything Model’ (SAM). The ‘segment-geospatial’ package draws its inspiration from ‘segment-anything-eo’ repository authored by Aliaksandr Hancharenka.

Installation

You can install the development version of {rsamgeo} like so:

remotes::install_github("brownag/rsamgeo")
rsamgeo::sg_install()

Example

After installing the package and the ‘Python’ dependencies, we can download some sample data using tms_to_geotiff()

library(rsamgeo)

samgeo()$tms_to_geotiff(
  output = "satellite.tif",
  bbox = c(-120.3704, 37.6762, -120.368, 37.6775),
  zoom = 20L,
  source = 'Satellite',
  overwrite = TRUE
)

The SAM model_type specifies the SAM model you wish to use. Trained model data used for segmentation are downloaded if the file checkpoint is not found. Downloading this for the first time may take a while. Create an instance of your desired model with sg_samgeo()

out_dir <- path.expand(file.path('~', 'Downloads'))
checkpoint <- file.path(out_dir, 'sam_vit_h_4b8939.pth')

sam <- sg_samgeo(
  model_type = 'vit_h',
  checkpoint = checkpoint,
  erosion_kernel = c(3L, 3L),
  mask_multiplier = 255L,
  sam_kwargs = NULL
)

Finally, generate a segmented image from the input, processing the input in chunks as needed with batch=TRUE. Note that you want this processing to run on the GPU with CUDA enabled. For the Google Colab Notebook example remember to set the notebook runtime to ‘GPU’.

sg_generate(sam, 
            "satellite.tif", 
            "segment.tif", 
            batch = TRUE)

Now that we have processed the input data, we can convert the segmented image to vectors and write them out as a layer in a GeoPackage for subsequent use.

sam$tiff_to_gpkg("segment.tif",
                 "segment.gpkg",
                 simplify_tolerance=NULL)

It is then fairly easy to overlay our segment polygons on the original satellite image with {terra}:

library(terra)
r <- rast("satellite.tif")
v <- vect("segment.gpkg")
v$ID <- seq_len(nrow(v))
plotRGB(r)
plot(v, col = v$ID, alpha = 0.25, add = TRUE)