nlsfi / hakunapi Goto Github PK
View Code? Open in Web Editor NEWOGC API Features Java server
Home Page: https://github.com/nlsfi/hakunapi
License: MIT License
OGC API Features Java server
Home Page: https://github.com/nlsfi/hakunapi
License: MIT License
For example a query like with bbox spanning the antimeridian:
https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/collections/addresses/items?bbox=160.6,-55.95,-170,-25.89
This is empty by dataset, should test and validate with some global dataset.
Documentation about different use cases (under "docs" folder on the repository):
This also relates to #10 .
The support for OGC API Features : Part 3 (Filtering) and CQL 2 standards in Hakunapi has been developed in late 2021 and early 2022 based on the draft standards versions. Still (2023-05-29) waiting for standards to be approved. At least after approval also support in Hakunapi should be enhanced.
First need to analyze and document
Common Query Language (CQL2)
With a collection like
{
"id" : "jobs",
"title" : "jobs",
"description" : "jobs",
"links" : [ ... ],
"itemType" : "feature",
"crs" : [ "http://www.opengis.net/def/crs/OGC/1.3/CRS84", "http://www.opengis.net/def/crs/EPSG/0/4326", "http://www.opengis.net/def/crs/EPSG/0/3903" ],
"storageCrs" : "http://www.opengis.net/def/crs/EPSG/0/3903"
}
and requesting items with ?crs=http://www.opengis.net/def/crs/EPSG/0/3903
the axis order of the response is not reversed (has E,N), while the CRS has N,E.
There seems to be some confusion on the axis order that should be followed on the response (crs-based or always geojson-easting-northing), but the consensus seems to be on prior arragement: client must understand what it requests, and the response header Content-Crs
defined in part 2 now "explicitly and unambiguously" defines the response coordinate contents => axis order follows crs.
geopython/pygeoapi#1174 (comment)
qgis/QGIS#52366 (comment)
This breaks hakunapi-based OAPIF layer use for example in QGIS (3.28.7+, 3.30.2+, 3.32.0+) where the implementation was changed to flip the order according to the crs.
Using datetime (or a custom attribute parameter with for example DatetimeRangeMapper
) does not accept other than UTC timestamp values as input.
?datetime=2020-01-01T00:00:00Z
works?datetime=2020-01-01T02:00:00+02:00
fails with 'datetime' value could not be parsed
OAPIF spec says RFC 3339, 5.6, which allows offsets.
After implementing #7.
Current HTML application templates with external resources does not work when web server has strict security settings.
Todo
Update ftl template resource references to latest versions.
Add configuration option to load custom ftl templates from an external directory.
(This includes possibility to use a npm built application)
# optional external template directory for ftl templates
formats.html.templates.dir=/var/www/html/features/templates
See #18 for background discussions.
The following request to a collection
EPSG:3067
/collections/{collectionId}/items?crs=http://www.opengis.net/def/crs/EPSG/0/3047&bbox=6854058.383,333886.496,6854061.684,333893.007&bbox-crs=http://www.opengis.net/def/crs/EPSG/0/3047
fails with error:
ERROR: ST_Intersects: Operation on mixed SRID geometries (Polygon, 3067) != (Polygon, 3047)
What?
This was piloted late 2023 by pull request #74.
A "minimal" sample feature server with a small spatial dataset (with a valid open data license to allow bundling).
This sample should demonstrate:
Support and documentation for:
Currently the finnish_addresses example is available online (https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/) without access restriction, but not in a managed cloud service.
Previous versions of Hakunapi supported also Java 8 - this can be removed.
Requirements for Java version support (minimum):
Also related to #16 other java servlet related requirements:
Feature request
Implement generic, extendable and configurable telemetry/usage monitor for operations GetCollectionItems, GetCollectionItemById.
Add interfaces to hakunapi-core
Add basic logging telemetry implementation as hakunapi-telemetry module.
Add webapp with telemetry as hakunapi-telemetry-webapp-javax module
Add a basic implementation based on SLF4J/Log4j
Add a configurable OpenTelemetry appender for SLF4J.
Basic operations
Add functionality around GetCollectionItems and GetCollectionItemById to support storing request-response information to a Span.
Store Span as JSON record for each request with collection response counts and a configurable set of headers from HTTP request
2024-01-18 12:44:52.828|INFO |fi.nls.hakunapi.telemetry.items||{"username":"<username>", "example-collection":15 "}
Logging configuration
Service configuration
telemetry.mode=json
telemetry.logger=fi.nls.hakunapi.telemetry
telemetry.collections=example-collection,...
telemetry.fields=username,...
telemetry.fields.username.header=remote-user
Optional opentelemetry implementation
Add functionality and configuration to connect SLF4J to opentelemetry appender
After implementing #6.
OGC API Features services by nls.fi using Hakunapi:
Public Hakunapi package version are tested on development environment and issues noticed are reported here.
Alternatives:
Maven Central requirements and guides:
About signing process and GPG keys (when publishing on Maven Central)
About group id on Maven Central
The alternative 1 was selected, steps:
According to OGC API Features discussion Support for OpenAPI 3.1 proper support for application schemas has waited OpenAPI version 3.1.
OpenAPI 3.1 has been released in 2021-02-16.
Related to OGC API Features there is proposal for an extension called OGC API - Features - Part n: Schemas. And there is a project page in GitHub for it. However no information about schedule for first draft?
Maybe there's two approaches to publish schemas in distant future:
/api
resource (based on separate conformance class to Open API 3.0 document, that is indicated by the conformance class http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30
)?returnables
resources?
See opengeospatial/ogcapi-features#338
The name is set here and probably there is nothing else to fix but I am not sure.
For example a CLI tool (or other user interface) that reads database schema and creates Hakunapi configuration templates (mapping database tables to feature collections).
Closing this issue. Originally opened because Teams Engine compliance tests on demo service based on Hakunapi complained about some requirements ("Precondition: requirement class http://www.opengis.net/spec/ogcapi-features-2/1.0/req/crs must be implemented"). However this referred to a standard requirement class (req
), not to a conformance class (conf
).
Hakunapi currently provides conformance classes in conf
form when requested by /conformance
resource as specified by code ConformanceClass.java. This should be correct and so this issue is closed. If there are some services based on Hakunapi that uses req
form, those services must be fixed.
The demo service should be fine:
https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/conformance
Not clear is this an issue with the OGC API Features standard definitions (that has conflicting definitions) or an issue with Hakunapi implementation.
OGC API - Features - Part 2 (https://docs.opengeospatial.org/is/18-058r1/18-058r1.html) has some conflicting definitions:
http://www.opengis.net/spec/ogcapi-features-2/1.0/conf/crs.
http://www.opengis.net/spec/ogcapi-features-2/1.0/req/crs
http://www.opengis.net/spec/ogcapi-features-2/1.0/conf/crs
Latest approved standard: https://docs.ogc.org/is/17-069r4/17-069r4.html
Chapter 2 (conformance) defines:
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/html
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/gmlsf0
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/gmlsf2
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30
But then chapters 7 - 9 (specifications of these classes) define something other:
http://www.opengis.net/spec/ogcapi-features-1/1.0/req/core
http://www.opengis.net/spec/ogcapi-features-1/1.0/req/html
http://www.opengis.net/spec/ogcapi-features-1/1.0/req/geojson
http://www.opengis.net/spec/ogcapi-features-1/1.0/req/gmlsf0
http://www.opengis.net/spec/ogcapi-features-1/1.0/req/gmlsf2
http://www.opengis.net/spec/ogcapi-features-1/1.0/req/oas30
Chapter 7.4. Declaration of conformance classes has a sample for conformance/
resource:
{
"conformsTo": [
"http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core",
"http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30",
"http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/html",
"http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson"
]
}
The Annex A: Abstract Test Suite (Normative) :
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/gmlsf0
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/gmlsf2
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/html
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30
Hakunapi usage can be demonstrated by demo app (published in #18) : https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/conformance
Hakunapi uses following classes, which is correct according to chapter 2 and (normative) annex A but not correct according to chapters 7-9:
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/html
Also Hakunapi uses following class that's not defined by standard:
http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/gpkg
Latest draft: http://docs.ogc.org/DRAFTS/19-079r1.html
Chapter 2 (conformance):
http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/filter
http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/features-filter
Chapters 6 - 9
http://www.opengis.net/spec/ogcapi-features-3/1.0/req/queryables
http://www.opengis.net/spec/ogcapi-features-3/1.0/req/queryables-query-parameters
http://www.opengis.net/spec/ogcapi-features-3/1.0/req/filter
http://www.opengis.net/spec/ogcapi-features-3/1.0/req/features-filter
Annex A: Abstract Test Suite (Normative)
http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/queryables
http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/queryables-query-parameters
http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/filter
http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/features-filter
Again conflicting definitions on the standard draft.
Hakunapi uses currently
http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/filter
http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/features-filter
Latest draft: https://docs.ogc.org/DRAFTS/21-065.html
Chapter 2 (conformance)
Chapter 6 - 8 defines a format like http://www.opengis.net/spec/cql2/1.0/req/basic-cql2
for all conformance/requirement classes mentioned above. And appendix A using conf
The standard OGC API - Common - Part 1: Core
http://www.opengis.net/spec/ogcapi-common-1/1.0/conf/core
http://www.opengis.net/spec/ogcapi-common-1/1.0/req/html
http://www.opengis.net/spec/ogcapi-common-1/1.0/req/json
http://www.opengis.net/spec/ogcapi-common-1/1.0/conf/json
This would clearly suggest that we should use the identifier, here for JSON the identifier is http://www.opengis.net/spec/ogcapi-common-1/1.0/conf/json
to announce (for example in /conformance resource) a conformance class that is defined by a requirement class.
But then this standard strikes again.
B.2. Conformance Examples
This example response in JSON is for an implementation of the OGC API-Common Standard that supports OpenAPI 3.0 for the API definition, as well as HTML and JSON as encodings for resources.
{
"conformsTo": [
"http://www.opengis.net/spec/ogcapi-common-1/1.0/req/core",
"http://www.opengis.net/spec/ogcapi-common-1/1.0/req/landing-page",
"http://www.opengis.net/spec/ogcapi-common-1/1.0/req/oas30",
"http://www.opengis.net/spec/ogcapi-common-1/1.0/req/html",
"http://www.opengis.net/spec/ogcapi-common-1/1.0/req/json"
]
}
This would suggest using http://www.opengis.net/spec/ogcapi-common-1/1.0/req/json
as an identifier!
After #10 is finalized on the repository and tested locally.
To publish a server for OGC API demos by NLS of Finland is used.
For example, a service based on nls.fi version of Hakunapi may output links like:
{
"id" : "syvyyskayransyvyysarvo",
"title" : "Syvyyskäyrän syvyysarvo",
"description" : "Syvyyskäyrään tallennettava käyrän korkeusarvoa osoittava luku (tekstikohde).",
"links" : [ {
"href" : "https://avoin-paikkatieto.maanmittauslaitos.fi/maastotiedot/features/v1/collections/syvyyskayransyvyysarvo/items",
"rel" : "items",
"type" : "application/geo+json"
}, {
"href" : "https://avoin-paikkatieto.maanmittauslaitos.fi/maastotiedot/features/v1/collections/syvyyskayransyvyysarvo/items?api-key=YOUR-API-KEY&f=json",
"rel" : "items",
"type" : "application/geo+json"
}, {
"href" : "https://avoin-paikkatieto.maanmittauslaitos.fi/maastotiedot/features/v1/collections/syvyyskayransyvyysarvo/schema?api-key=YOUR-API-KEY",
"rel" : "describedBy",
"type" : "application/schema+json"
}, {
"href" : "https://avoin-paikkatieto.maanmittauslaitos.fi/maastotiedot/features/v1/collections/syvyyskayransyvyysarvo/queryables?api-key=YOUR-API-KEY",
"rel" : "http://www.opengis.net/def/rel/ogc/1.0/queryables",
"type" : "application/schema+json"
} ],
"itemType" : "feature",
"crs" : [ "http://www.opengis.net/def/crs/OGC/1.3/CRS84", "http://www.opengis.net/def/crs/EPSG/0/4326", "http://www.opengis.net/def/crs/EPSG/0/3067", "http://www.opengis.net/def/crs/EPSG/0/4258", "http://www.opengis.net/def/crs/EPSG/0/3046", "http://www.opengis.net/def/crs/EPSG/0/3047", "http://www.opengis.net/def/crs/EPSG/0/3048", "http://www.opengis.net/def/crs/EPSG/0/3873", "http://www.opengis.net/def/crs/EPSG/0/3874", "http://www.opengis.net/def/crs/EPSG/0/3875", "http://www.opengis.net/def/crs/EPSG/0/3876", "http://www.opengis.net/def/crs/EPSG/0/3877", "http://www.opengis.net/def/crs/EPSG/0/3878", "http://www.opengis.net/def/crs/EPSG/0/3879", "http://www.opengis.net/def/crs/EPSG/0/3880", "http://www.opengis.net/def/crs/EPSG/0/3881", "http://www.opengis.net/def/crs/EPSG/0/3882", "http://www.opengis.net/def/crs/EPSG/0/3883", "http://www.opengis.net/def/crs/EPSG/0/3884", "http://www.opengis.net/def/crs/EPSG/0/3885" ],
"storageCrs" : "http://www.opengis.net/def/crs/EPSG/0/3067"
}
Here a link to items
is without API-key, but items?f=json
resource has it.
How this should be implemented consistently?
As a reference Hakunapi demo service:
https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/collections?f=json
{
"links" : [ {
"href" : "https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/collections",
"rel" : "self",
"type" : "application/json",
"title" : "This document"
} ],
"collections" : [ {
"id" : "addresses",
"title" : "Simple Addresses",
"description" : "Addresses of Finlands Buildings",
"links" : [ {
"href" : "https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/collections/addresses/items",
"rel" : "items",
"type" : "application/geo+json"
}, {
"href" : "https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/collections/addresses/items",
"rel" : "items",
"type" : "text/html"
}, {
"href" : "https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/collections/addresses/items?f=json",
"rel" : "items",
"type" : "application/geo+json"
}, {
"href" : "https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/collections/addresses/items?f=html",
"rel" : "items",
"type" : "text/html"
}, {
"href" : "https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/collections/addresses/schema",
"rel" : "describedBy",
"type" : "application/schema+json"
}, {
"href" : "https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/collections/addresses/queryables",
"rel" : "http://www.opengis.net/def/rel/ogc/1.0/queryables",
"type" : "application/schema+json"
} ],
"itemType" : "feature",
"crs" : [ "http://www.opengis.net/def/crs/OGC/1.3/CRS84", "http://www.opengis.net/def/crs/EPSG/0/4326", "http://www.opengis.net/def/crs/EPSG/0/3067", "http://www.opengis.net/def/crs/EPSG/0/4258" ],
"storageCrs" : "http://www.opengis.net/def/crs/EPSG/0/3067"
} ]
}
There is four items-resource links. Two links to items
resource (both GeoJSON and HTML) and separately items?f=json
and items?f=html
links.
Currently hakunapi doesn't support Link headers. This is problematic with GeoPackage (and less used GML) as there's no way to access for example next link (which is crucial for pagination).
WFS3 näyttää vielä kummittelevan joissakin paikoissa, mahtaako olla tarkoitus?
WFS3Service.java
ConformanceOperation.java
ConformanceClass.java
Tuo direct-api on määritelty noin insinöröintiraportissa https://docs.ogc.org/per/18-078.html#direct_access_path, mutta sen kohdetta ei oikeasti ole olemassa.
Onkohan tämä GPKG-conformance myös jostain kokeilusta peräisin:
GPKG("http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/gpkg"
Pitäisikö noihin kohteisiin (DIRECT_TILE ja GPKG), joita ei oikeasti löydy, lisätä kommentit?
There should be introduction documentation (features, technical requirements, dependencies, ..) about each modules the Hakunapi repository provides.
For example a separate readme.md for each module folder.
In some services Hakunapi feature configuration files are relatively large.
In this enhancements Hakunapi read a folder containing multipe configuration files and merges these when starting a server.
Currently hakunapi doesn't require spatial extent to be configured for collections with geometry properties. This can lead to issues that are difficult to debug as some clients (ETS validator for example) use spatial extent of a collection to determine if the collection is a spatial feature collection.
This could be useful for users running postgis on the same machine as hakunapi.
After implementing #5.
This might not be that big of a change?
Use NOPProjectionTransformer
and HakunaPropertyGeometry will provide the raw (already transformed) value via its' mapper function.
See the official github page about: OGC Features and Geometries JSON that is OGC's candidate standard.
According to documentation OGC Features and Geometries JSON will
This might be one of candidate enhancements for Hakunapi after 1.0 version.
Before deciding some analysis is needed:
Support at least these features first:
Non goals (for first iteration):
Standard conformance classes to be supported (from the draft standard):
Currently at least Open API document represented as JSON provided by https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/api.json fails when analyzing it in Swagger editor (or other tools).
Gives multiple errors like:
Structural error at paths./.get.responses.200.content.text/html
should NOT have additional properties
additionalProperty: exampleSetFlag
Jump to line 33
When accessing Open API document as HTML content in browser (https://beta-paikkatieto.maanmittauslaitos.fi/inspire-addresses/features/v1/api) HTML document is shown without errors.
The issue is related to the exampleSetFlag
property (that informs "whether an example is set to null intentionally") in OpenAPI class of Hakunapi - this property should not be serialized to Open API document output.
Currently modules (https://github.com/nlsfi/hakunapi/tree/main/src) contains:
These should be renamed with "javax" extension etc. to support Hakunapi servers with JDK11+ / tomcat 9 / jersey 2 / javax / webapp
This would allow in future similar modules with "jakarta" dependencies (targeting JDK11+ / tomcat 10 / jersey 3 /jakarta / webapp).
Still currently the support for "javax" is the main focus - may change in future.
Initial implementation in pull request #85
After implementing #25 it will be possible to control the content-type of each resource in the api with a query parameter. It would be nice if the default HTML templates used by hakunapi would include a link to the JSON representation of the current resource. This is pretty common functionality in an OGC API Features implementation.
Support for Oracle data store to provide OGC API Features service.
Some thoughts was discussed on pull request #86
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.