Giter Site home page Giter Site logo

rwfs's Introduction

rwfs

rOG-badge Project Status: Active - The project has reached a stable, usable state and is being actively developed. R-CMD-check codecov

WFS client for R

Overview

This R package provides a client to access a Web Feature Service (WFS). Its core functionality is build on top of sf package.

Usage

Installation

rwfs can be installed from GitHub using devtools:

install.packages("devtools")
library("devtools")
devtools::install_github("ropengov/rwfs")

and loaded with

library(rwfs)

Request and client classes

The package consists of request and client classes which are of type R6. The request classes are used to construct a reference to a data source and provide access methods to the data. The client classes dispatch a request to obtain and possibly manipulate the data.

The request classes currently implemented are

  • WFSStreamingRequest for streaming data directly from a WFS,
  • WFSCachingRequest for downloading data from a WFS and caching it to disk and
  • GMLFile for reading data from a GML file.

All request classes are abstract with the exception of GMLFile meaning that the user of rwfs must provide a subclass for each relevant class and implement abstract methods in the classes. The user may provide additional methods for accessing the data for convenience.

The following client classes are currently implemented

  • WFSStreamingClient for streaming requests and
  • WFSCachingClient for downloading and caching requests and for local file access requests.

The client classes are available to be used directly. However, additional methods for manipulating data for user convenience can be provided by inheriting the classes.

In the following sections, we illustrate the use of the rwfs package by following two examples taken from the packages gisfin and fmi.

Inheritance of request classes

For streaming, the WFSStreamingRequest abstract class is required to be inherited to a subclass that implements the abstract method getDataSource(), which provides a data access reference. For example (taken from gisfin), the following class provides a URL to access data with the private method getURL(), which is called from getDataSource():

GeoStatFiWFSRequest <- R6::R6Class(
  "GeoStatFiWFSRequest",
  inherit = rwfs::WFSStreamingRequest,
  private = list(
    getURL = function() {
      url <- paste0("http://geo.stat.fi/geoserver/", private$getPathString(), "/wfs?", private$getParametersString())
      return(url)
    }
  ),
  public = list(
    getDataSource = function() private$getURL(),
    
    getGeoStatFiLayers = function(path) {
      if (missing(path))
        stop("Required argument 'path' missing.")      
      return(self$setPath(path)$getCapabilities())
    },

    getGeoStatFiLayer = function(path, layer) {
      if (missing(path))
        stop("Required argument 'path' missing.")      
      if (missing(layer))
        stop("Required argument 'layer' missing.")
      return(self$setPath(path)$getFeature(typeName=layer))
    },
    
    getPopulationLayers = function() self$getGeoStatFiLayers("vaestoruutu"),
    getPopulation = function(layer) self$getGeoStatFiLayer("vaestoruutu", layer),
    getProductionAndIndustrialFacilitiesLayers = function() self$getGeoStatFiLayers("ttlaitokset/ttlaitokset:toimipaikat"),
    getProductionAndIndustrialFacilities = function(layer="ttlaitokset:toimipaikat") self$getGeoStatFiLayer("ttlaitokset/ttlaitokset:toimipaikat", layer),
    getEducationalInstitutionsLayers = function() self$getGeoStatFiLayers("oppilaitokset/oppilaitokset:oppilaitokset"),
    getEducationalInstitutions = function(layer="oppilaitokset:oppilaitokset") self$getGeoStatFiLayer("oppilaitokset/oppilaitokset:oppilaitokset", layer),
    getRoadAccidentsLayers = function() self$getGeoStatFiLayers("tieliikenne"),
    getRoadAccidents = function(layer) self$getGeoStatFiLayer("tieliikenne", layer),
    getPostalCodeAreaLayers = function() self$getGeoStatFiLayers("postialue"),
    getPostalCodeArea = function(layer) self$getGeoStatFiLayer("postialue", layer)
  )
)

The method getGeoStatFiLayers(path) lists available layers by setting path, which refers to the data set behind path in the service, and the method getGeoStatFiLayer(path, layers) which obtains the layer layer from the path data set. The rest of the methods are for convenience purpose for the user that cover the available data sets in the service.

Similar to WFSStreamingRequest, WFSCachingRequest must implement the getURL abstract method (example taken from fmi):

FMIWFSRequest <- R6::R6Class(
  "FMIWFSRequest",
  inherit = rwfs::WFSCachingRequest,
  private = list(
    apiKey = NA,
    
    getURL = function() {
      url <- paste0("http://data.fmi.fi/fmi-apikey/", private$apiKey, "/wfs?", private$getParametersString())
      return(url)
    }
  ),
  public = list(
    initialize = function(apiKey) {
      if (missing(apiKey))
        stop("Must specify the 'apiKey' parameter.")
      private$apiKey <- apiKey
    }    
  )
)

Here the class provides also a mechnism for storing an API key, which is required to access the service.

Inheritance of client classes

Continuing with the example from gisfin, there is no need to inherit WFSStreamingClient. However, for consistency the package provides the class GeoStatFiWFSClient inheriting WFSStreamingClient, which is exactly the same class but with a different name.

The FMIWFSClient class inheriting WFSCachingClient in the fmi example, sets the service access parameters and returns data after formatting it. For example, the getMonthlyWeatherRaster calls the setParameters() method in the request object and the getRaster() in the superclass to obtain a raster object.

FMIWFSClient <- R6::R6Class(
  "FMIWFSClient",
  inherit = rwfs::WFSCachingClient,
  private = list(
    processParameters = function(startDateTime=NULL, endDateTime=NULL, bbox=NULL, fmisid=NULL) {
      [...]
    },
    
    getRasterURL = function(parameters) {
      [...]
    }
  ),
  public = list(
    getDailyWeather = function(variables=c("rrday","snow","tday","tmin","tmax"), startDateTime, endDateTime, bbox=NULL, fmisid=NULL) {      
      [...]
    },
    
    getMonthlyWeatherRaster = function(startDateTime, endDateTime) {
      if (inherits(private$request, "FMIWFSRequest")) {
        if (missing(startDateTime) | missing(endDateTime))
          stop("Arguments 'startDateTime' and 'endDateTime' must be provided.")
        
        p <- private$processParameters(startDateTime=startDateTime, endDateTime=endDateTime)
        private$request$setParameters(request="getFeature",
                                      storedquery_id="fmi::observations::weather::monthly::grid",
                                      starttime=p$startDateTime,
                                      endtime=p$endDateTime)
      }
      
      response <- self$getRaster(parameters=list(splitListFields=TRUE))
      if (is.character(response)) return(character())
      NAvalue(response) <- 9999
      names(response) <- getRasterLayerNames(startDateTime=startDateTime,
                                             endDateTime=endDateTime,
                                             by="month",
                                             variables=c("MeanTemperature", "Precipitation"))
      return(response)
    }
  )
)

The getMonthlyWeatherRaster() method first checks that request object has been given and it is of the typeFMIWFSRequest. The method also does some data manipulation such as setting layer names for the raster object.

Accessing WFS

Once we have defined appropriate subclasses, we can build a request object and access WFS to obtain data with a client object. For examples, please look at the vignettes in the gisfin and in the fmi packages.

File access

An example to access a local GML file:

library(rwfs)
fileName <- tempfile()
download.file("http://geo.stat.fi/geoserver/vaestoalue/wfs?service=WFS&version=1.0.0&request=GetFeature&typeName=vaestoalue:suuralue_vaki2014", fileName)
request <- rwfs::GMLFile$new(fileName)
client <- rwfs::WFSCachingClient$new(request)
layer <- client$getLayer("suuralue_vaki2014")
print(layer@data)
plot(layer)
unlink(fileName)

Contact

You are welcome to:

Acknowledgements

Roger Bivand for helping with getting the WFS driver working on Windows.

rwfs's People

Contributors

jlehtoma avatar statguy avatar antagomir avatar dieghernan avatar ouzor avatar gitter-badger avatar

Stargazers

HarryZhu avatar Santiago Mota avatar obrl_soil avatar Andrew Gene Brown avatar Nicole Hemming-Schroeder avatar Stephanie Hazlitt avatar Andy Teucher avatar Petr Bouchal avatar Benjamin Louis avatar Patrick Hausmann avatar Robin Lovelace avatar Sam Boysel avatar Sam Albers avatar Vassily Trubetskoy avatar Sebastian Jeworutzki avatar Andreas Petutschnig avatar Jemma Stachelek avatar Michael Henry avatar

Watchers

 avatar James Cloos avatar  avatar  avatar Markus Kainu avatar Måns Magnusson avatar Emanuele Cordano avatar Przemysław Biecek avatar  avatar

Forkers

gitter-badger

rwfs's Issues

Check if rgdal supports streaming data directly from data source

rwfs already has class WFSStreamingClient() which reads the data directly from the source. However, there seems to a lot of issues with geometry types and there may be no way of doing ogr2ogr -explodecollections with readOGR(). Will need to look in more detail to this.

Improve error reporting in WFSClient$getLayer or deal with empty / missing layers in some other fashion

For original context see fmi/issues/22.

When trying to download weather data from FMI for a period or station where they don't exist, the FMIWFSClient$getDailyWeather get fails with an error in convertOGR. It would be preferable to have more descriptive error message or an empty dataset to be returned. The relevant rwfs code is in the WFSClient$getLayer method.

> client$getDailyWeather(startDateTime="2012-05-15", endDateTime="2012-07-17", fmisid = 100929)
ogr2ogr -f GML  -splitlistfields /tmp/Rtmp9R6Bi7/file470e214c4ed /tmp/Rtmp9R6Bi7/file470e50047a05 PointTimeSeriesObservation
ERROR 1: Couldn't fetch requested layer 'PointTimeSeriesObservation'!
Error in convertOGR(sourceFile = private$cachedResponseFile, layer = layer,  : 
  Conversion failed.

On the rwfs develop branch the error is still not very informative.

Error in ogrInfo(dsn = dsn, layer = layer, encoding = encoding, use_iconv = use_iconv,  : 
  Cannot open layer
Error in private$.getLayer(dataSource = sourceFile, layer = layer, crs = crs,  : 
  Fatal error.

A shortened version of the empty response:

<?xml version="1.0" encoding="UTF-8"?>
<wfs:FeatureCollection
    timeStamp="2016-11-28T10:54:03Z"
    numberMatched="0"
    numberReturned="0"
           xmlns:wfs="...">
   
</wfs:FeatureCollection>

Implement getDataSource() in base class WFSStreamingRequest

Since the following case is very common:

getDataSource = function() {
    private$getURL()
}

This could be implemented in WFSStreamingRequest making it a non-abstract method. This WFSStreaming* and WFSCaching* classes could be used directly (for simple use cases) without the need of implementing subclasses first.

Add an example/vignette?

Hi!

I've tried to use the package, but I can't understand how to use it. Can you put up some example code?

Remove need for ogr2ogr translation

Currently, there are two ways getting the data from the data source:

  1. Create a class inheriting from rwfs::WFSStreamingRequest, implement method getDataSource() and read data directly from data source using rgdal::readOGR() (actually twice: first for reading the layers, then the actual data). No caching is possible.
  2. Create a class inheriting from rwfs::WFSCachingRequest or instantiate it directly. Read (download) data only once from the data source and save to a temp directory. Use ogr2ogr -explodecollections to locally translate the (GML) data. Read the translated data in using rgdal::readOGR(). Within the lifetime of an instance (object), the responses are cached.

Original problem

The original reason for having the extra ogr2ogr-step is the following:

Due to limited support for WFS 2.x in the rgdal package, which the rwfs package depends on, data in > WFS 2.x is accessible only via downloading the data first and thus no streaming can be used (at least > in some services).

This has (hopefully) changed with more recent (>= 2.0.0) versions of GDAL, meaning that the ogr2ogr-step could in theory be removed. Many users, particularly on Windows, have had trouble in getting ogr2ogr to work.

Making ogr2ogr optional

If the ogr2ogr step is not required (now in rwfs::WFSCachingClient$getLayer()), then effectively modifying rwfs::WFSCachingClient should be enough. Instead of completely removing the step, the functionality should be retained for cases where it is still needed. For caching, it is always necessary to download the data. Downloading the data also removes the need the get the data twice as in streaming.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.