Use geographical coordinates for match scoring

[wetneb]

Use geographical coordinates for match scoring

[planemad]

Thanks for the explanation and the useful documentation @wetneb. Did not know we can set the columns for matching with P625@lat, P625@lng

Currently a score of 0 is reached when the points are 1km away from each other

As a first step it would be great if this could be tuned by the user since it would vary on the type of dataset. For example, for matching city locations, the centroid could vary from Wikidata by over 5 kms and it still should be ok. For a mountain range this could be 100kms etc.

[wetneb]

https://github.com/wetneb/openrefine-wikidata/blob/43375fc5ba03bc032c096fdf3283b66ebc8768fd/wdreconcile/wikidatavalue.py#L254-L273

[wetneb]

@planemad ideally i'd prefer not to hard-code this in the reconciliation interface. AFAICT the ideal situation would be that people use the dedicated "precision" field in Wikibase in a meaningful way with regard to the object they are representing. But unfortunately this precision field is not really used yet (it is poorly exposed in the UI and in the SPARQL query service).

[wetneb]

Hi @planemad, thanks!
This is already possible to some extent: https://github.com/OpenRefine/OpenRefine/wiki/Reconciliation#comparing-values

The issue is that we are limited by the reconciliation API: it does not let the server return the matching scores for individual properties, only one global matching score per query. However, once the matching is done, it is straightforward to retrieve the coordinates of the matched items (especially with the data extension dialog), and compute the distance in OpenRefine.

I agree the user story you have would be ideal, but it requires some changes on OpenRefine's side (which we should totally consider! it will just take more time to agree on an improved protocol, and so on).

[wetneb]

@planemad yes I agree this should be user-defined… but how?
Essentially, the problem we have here is that all the matching heuristics are server-side, while we would need to have some finer control on them client-side, and the reconciliation API does not give any control over that…

[planemad]

@wetneb do you know where the code is that scores matches only upto 1km?

[planemad]

The only idea I can think of is to store a mapping of common instance types and acceptable distance threshold for a match. eg:

• building: 50m
• road: 100m
• airport: 5km
• city: 20km
• river:100km
• country: 200km
• landform: 500km

Tbh, for most general cases 1km would work fine. This is an issue only for features that are bigger than 1km and maybe the use case is too limited.

[planemad]

Closing this, the orignal issue of matching using coordinates is already documented.

Agree that either 'precision' or some other property on Wikidata should indicate to external services how to set a suitable proximity threshold. This is out of scope for this API.