stevesong/kml2ofds

KML2OFDS is a python script for converting KML maps of fibre optic network infrastructure to the Open Fibre Data Standard.

KML2OFDS

KML2OFDS is a python script for converting KML maps of fibre optic network infrastructure to the Open Fibre Data Standard. Consult the documentation for more info.

Assumptions

Core to OFDS is the principle that any span of fibre must be terminated at either end by a Point of Presence of some kind. Here we are using Point of Presence in a very loose sense. This might be a simple access point such as a manhole or it might be a full point of presence access point. Consult the standard for more details.

As such, KML2OFDS expects a KML file that contains both fibre optic routes as well as points of presence.

In broad strokes the script:

• parses a KML document for features and separates them into a collection of nodes (any point feature in the KML) and spans (any LineString or collection of LineStrings) in the KML;
• checks for duplicate nodes based on a combination of node 'name' and node latitude, longitude based on an adjustable level of location precision
• the script snaps nodes to the closest point on the closest span, if they are not already somewhere on a span;
• it then breaks each span at every point where a node intersects a span, resulting in a larger number of shorter spans;
• a node is then associated each with the "start" and "end" of each of the spans;
• where a span doesn't have a node at the start or end, a node is auto generated in order to comply with the OFDS standard;
• adds meta data to the spans and nodes. at the moment on the most basic meta data is added during the export process
• where two auto-generated nodes are found in close proximity, it is assumed that the spans they are connected to should be joined, the two nodes are merged and the end points of the spans connect to the merged node;

Install kml2ofds

Using pipx

The recommended way to install kml2ofds as a user is with pipx.

First ensure pipx is installed, then to install the latest kml2ofds:

pipx install git+https://github.com/stevesong/kml2ofds.git@main

and then the command should be available to use (you may need to restart your shell):

kml2ofds [--options]

Using pip

To install kml2ofds inside an existing python virtual environment or conda environment:

# First activate your environment, then:
pip install git+https://github.com/stevesong/kml2ofds.git@main

Developing kml2ofds

To test running kml2ofds while developing, it's necessary to install the package as editable using pip -e, e.g.:

cd path/to/kml2ofds/
source .venv/bin/activate # If using a python virtual environment

# Install as editable
pip install -e .

To add, remove or update requirements, edit the dependencies section in pyproject.toml, then run pip install -e . again to update the dependencies in your virtual environment.

kml2ofds's Issues

Consider cases where a node terminates close to another span but not near either end.

In OFDS, a network fork should be a node at which three spans terminate. When network mappers draw a network fork in KML, they may choose to draw the fork by drawing a new line that begins close to an existing polyline. The current code does not have the capability to detect and resolve forks that are created this way.

This could be an Auto generated missing node. (note that "Auto generated missing nodes" are created to terminate any polyline that doesn't end in a node, in order to be compliant with OFDS which requires all spans to be terminated by a node at either end)

or a named node

There is a need for functionality that would detect nodes that are within a certain proximity to a polyline but not either end of the polyline. In these instances, we should break the polyline at the nearest vertex of the polyline and use the node as the terminating point.

Add option to join two strings that terminate close to each other but which don't have a node

Currently the code assigns a node to any span endpoint that does not already have one. In some cases, where two spans terminate close to each other, it may be that the the map is intended to be contiguous at this point. We could look for spans that terminate very close to each other and join those spans into a single span.

Alternatively, further on in the code, we could look for two "Auto generated nodes" that are close together and reduce it to a single node connecting the two spans.

MultiGeometry with two LineStrings that have a common start Lat/Lon but which go in opposite directions

The current algorithm assumes that LineStrings are more or less sequential and heading in the same direction. I am not sure how to parse cases where two LineStrings, that are part of the same MultiGeometry object, both start line middle and work their way outward. For the time being will have to mention it as a caveat until a workaround is found.

Move to python click module to accept all parameters from the command line.

Consider cases where an Auto generated node terminates close to an existing named point

Look at buffered_area = MultiPolygon(buffered_points)

add the capacity to specific Placemarks to ignore e.g. manholes

For some detailed KML maps, users may wish to exclude certain features from the export. Add the option to specify features to ignore in the .ini file.

pykml parser is not recognising Placemark or LineString elements with embedded ids

Namibia's Grid fibre network https://thegridonline.network/?page_id=770 contains Placemark and LineString elements with IDs e.g.

Will probably have to use findall rather than iter. e.g.

# Parse your KML file
with open('your_file.kml', 'rb') as kml_file:
    root = parser.parse(kml_file).getroot()

# Define the namespace
namespace = "{http://www.opengis.net/kml/2.2}"

# Find all Placemark elements regardless of their ID
placemarks = root.findall(".//" + namespace + "Placemark")

# Iterate over found Placemarks
for placemark in placemarks:
    # Do whatever you need with each Placemark
    # For example, print its ID
    placemark_id = placemark.get("id")

Test for crossed polylines and generate error

The current code doesn't cope well with polylines that intersect each other. Short term strategy is to test for this and generate an error, suggesting that the KML should be adjusted to remove intersections.

add functionality to open KMZ as well as KML files