Comments (7)
Use WFS2.0 as data source for pygeoapi
from pygeoapi.
+1 great feature! Considerations:
- we can use OWSLib’s WFS functionality to interact with WFS 1 or 2 sources
- we can also use OWSLib’s Filter XML functionality to convert WFS 3 property filters into WFS 1/2 Filter XML. This may get tricky depending on the underlying WFS 1/2 implementation
- the devil will be in the WFS 1/2 GetFeature responses. OWSLib simply returns string/XML object representations which would need to be converted into GeoJSON. Suggest we start with common WFS 1/2 implementations (MapServer, GeoServer, others?)
from pygeoapi.
- Need dockers with WFS services for testing on travis (https://docs.travis-ci.com/user/docker)
- Starting with mapserver WFS2.0 support and then WFS2.0 in Geoserver
from pygeoapi.
Think this can be solved in a compact way with a single OGRProvider
: a PyGeoAPI Provider that uses the GDAL/OGR Python bindings. Did some tests with both GeoServer and MapServer WFS v2 backends like from PDOK (Dutch National SDI). Many of the WFS-FES capabilities can be supported directly (if backend supports WFS v2):
- result type
hits
: feature count - paging: translates to WFS paging with configurable pagesize
- spatial filtering: e.g. bounding box or any other
- attribute filtering, including get-feature-by-id: e.g. get feature by (gml) id
Advantages are:
- no need for GML request construction (like Filters) and response parsing and GML-GeoJSON mapping
- support not just WFS v2 but (m)any OGR source-types like GPKG, Shapefile, FGDB etc
- Proxy support for URL-based backends
- multiple auth methods support for URL-based backends
Things to consider:
- OGR WFS Driver may issue WFS GetCapabilities request on each
Open()
call (need investigation) - the regular WFS v1, v2 performance considerations like complex filters bringing a WFS down
Software Design consideration:
- Develop a single
OGRProvider
as a one-in-all Provider, or as an ABC with specializedOGRProvider
-derived classes likeOGRWFSProvider
,OGRShapefileProvider
etc.
I would aim first for OGRProvider
with specialized Driver/OGRSource-type-specific helper-classes.
We do something very similar within a Stetl OGRInput ETL class. Main difference (per Source-type) is really on per-OGR-Driver-specific Source options (next to standard GDAL/OGR options). We can catch this within the PGA config YAML like:
.
.
provider:
name: OGR
data:
source_type: WFS
source: WFS:http://geodata.nationaalgeoregister.nl/rdinfo/wfs?
source_options:
VERSION: 2.0.0
OGR_WFS_PAGING_ALLOWED: YES
OGR_WFS_LOAD_MULTIPLE_LAYER_DEFN: NO
gdal_ogr_options:
GDAL_CACHEMAX: 64
GDAL_HTTP_PROXY: (optional proxy)
GDAL_PROXY_AUTH: (optional auth for remote WFS)
CPL_DEBUG: NO
id_field: gml_id
layer: rdinfo:stations
.
.
provider:
name: OGR
data:
source_type: ESRI Shapefile
source: tests/data/rdinfo.shp
source_options:
.
gdal_ogr_options:
GDAL_CACHEMAX: 64
id_field: fid
layer: stations
I can take stab at this, would be nice to have something running to e.g. wrap all PDOK WFS-es.
refs:
- WGS WFS Driver: https://gdal.org/drv_wfs.html
- Python GDAL-OGR Cookbook: https://pcjericks.github.io/py-gdalogr-cookbook/vector_layers.h
from pygeoapi.
Thanks @justb4. This approach works. I'm glad the plugin framework is flexible enough to encompass an OGR driver. In one way, OGR can replace every other provider that implements the same thing. It will be good to have them all, and let them mature/evolve over time.
from pygeoapi.
update: OGR Provider can even do (paged) fetch from ESRI FeatureServers!
from pygeoapi.
Think we're done with OGRProvider
for WFS2 backends. Tested with GeoServer, MapServer and deegree WFSs (see tests), including bbox
queries. Other OGR backends (Drivers) work as well like GPKG, ESRI Shapefile, ESRI FeatureServer, except for paging (will need to use OGR SQL for most). Will open separate issue(s) for those.
So closing here, re-open if still issues with WFS2 OGR backends.
from pygeoapi.
Related Issues (20)
- implement RFC4 HOT 1
- Issue serving the same collection using both, a vector tile and a raster tile providers HOT 3
- TypeError: Object of type date is not JSON serializable : Admin API patch method HOT 5
- gzip compression not applied in starlette (invalid header handling) HOT 2
- Can't delete jobs in django mode
- Pagination issue in v0.16 HOT 2
- `get_collection_tile_items` not available in django mode, only `get_collection_tile_metadata` is being called HOT 1
- TypeError: DummyManager.execute_process() got an unexpected keyword argument 'subscriber' HOT 3
- Allow providing a default value in config when env variable does not exist HOT 1
- add a urlattr filter to html output HOT 2
- Time querying in `xarray_edr.py` HOT 2
- Adding `cftime` dependency for EDR HOT 1
- Issue with Hateoas Provider
- how would one expose 2 (or more) linked datasets in pygeoapi HOT 3
- How can one replace existing WMS/WFS service with pygeoapi.
- Error Generating OpenAPI document with EDR Provider HOT 1
- Server setting to allow for bigger payloads for Processess HOT 4
- provide more error description for processes
- PostgreSQL provider does not support datetime HOT 7
- `title` text of links need to be translatable for multilingual support
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pygeoapi.