Giter Site home page Giter Site logo

Comments (8)

mattijn avatar mattijn commented on May 28, 2024 1

TLDR: I think, if you set the shared_coords=False it will work for the above mentioned cases (see https://mattijn.github.io/topojson/example/settings-tuning.html#shared_coords for info).

No solutions for the issue in the strategy shared_coords=True, but a better understanding where it goes wrong. Single touching coordinates are in the shared_coords are used as cutting-points and afterwards the non-shared segments are tried to be merged again into a contagious segment, where possible.

This process doesn't work properly as is proven in this issue and I can reproduce it with two linestrings.

from shapely.wkt import loads
import topojson
from topojson.core.extract import Extract
from topojson.core.cut import Cut
from topojson.core.dedup import Dedup

data = [
    {"type": "LineString", "coordinates": [(1,0),(2,0),(3,0),(4,0),(5,0)]},
    {"type": "LineString", "coordinates": [(5,0),(4,-1),(4,0),(4,1),(3,1),(3,0),(2,1),(2,0),(1,0),(1,1)]},
]

# topojson.Topology(data, shared_coords=False)  # this works (in this case)
topojson.Topology(data, shared_coords=True)
TypeError: 'NoneType' object is not iterable  # v1.0rc10
TypeError: cannot unpack non-iterable NoneType object  # master
# Extract works
e = Extract(data, options={'prequantize':False})
e.to_svg(separate=True)

image

# Cut works
c = Cut(data, options={'prequantize':False})
c.to_svg(include_junctions=True, separate=False)

image
Dedup class fails in https://github.com/mattijn/topojson/blob/master/topojson/core/dedup.py#L115-L146. The function _find_merged_linestring returns nothing.

# 3 out of 4 arcs are detected in te linemerge, but current code only covers 2 of n or n of n
Dedup(data, options={'prequantize':False})
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-22-22bbfc364764> in <module>
----> 1 Dedup(data, options={'prequantize':False})

~\Documents\notebooks\topojson\topojson\core\dedup.py in __init__(self, data, options)
     25 
     26         # execute main function of Dedup
---> 27         self.output = self._deduper(self.output)
     28 
     29     def __repr__(self):

~\Documents\notebooks\topojson\topojson\core\dedup.py in _deduper(self, data)
     97             # apply linemerge on geoms containing contigious arcs and maintain
     98             # bookkeeping
---> 99             self._merge_contigious_arcs(data, sliced_array_bk_ndp)
    100 
    101             # pop the merged contigious arcs and maintain bookkeeping.

~\Documents\notebooks\topojson\topojson\core\dedup.py in _merge_contigious_arcs(self, data, sliced_array_bk_ndp)
    236                 # get the idx of the linestring which was merged
    237                 idx_merg_arc, consec_behavior = self._find_merged_linestring(
--> 238                     data, no_ndp_arcs, ndp_arcs, ndp_arcs_bk
    239                 )
    240 

TypeError: cannot unpack non-iterable NoneType object

In _find_merged_linestring the following logic should be covered

from IPython.display import SVG, display
from shapely.ops import linemerge

def svg_split_view(geom):
    svg_custom = geom._repr_svg_()
    for c in ['green','blue', 'orange', 'red']:    
        svg_custom = svg_custom.replace('stroke="#66cc99"', f'stroke="{c}"', 1)
    display(SVG(svg_custom))
    
in_geoms = loads('MULTILINESTRING ((5 0, 4 -1, 4 0), (4 0, 4 1, 3 1, 3 0), (3 0, 2 1, 2 0), (1 0, 1 1))')
svg_split_view(in_geoms) # 4 non shared arcs

image

out_geoms = linemerge(in_geoms)
svg_split_view(out_geoms) # 3 out of 4 can be merged

image

The issue is in the process when the remaining non-shared arcs are tried to be merged again.
In this process (_find_merged_linestring) are currently two options covered
(1) the first and last non-shared arc can be merged and
(2) all arcs can be merged.

Until know this were the two situations. It seems the assumption that this logic works for all cases has been proven wrong.
Somehow need to find a better method to detect which segment-indices where merged of the non-shared arcs.

from topojson.

mattijn avatar mattijn commented on May 28, 2024 1

Thanks again for raising the issue and the great discussion. This was very helpful and the wrong behavior is fixed by PR #105.

from topojson.

mattijn avatar mattijn commented on May 28, 2024

Thanks for raising the issue! I can reproduce this on master, not sure what happens yet.

from topojson.

IvKor avatar IvKor commented on May 28, 2024

@mattijn, @robbibt

I hit the same issue and tried to narrow down the problem by limiting the input data to three polygons.

Code:

import pandas
import geopandas
import topojson as tp
from shapely.wkt import loads

df = pandas.DataFrame({
    "name": ["P1", "P2", "P3"],
    "geometry": [
        "POLYGON ((60.05 88.85, 60.3 86.9, 61.9 73.4, 51.85 72.1, 50.8 80.5, 57.95 81.4, 57.5 85.05, 59.05 85.25, 58.85 87.15, 60.05 88.85))",
        "POLYGON ((66.35 90.55, 65.75 86.8, 64.5 81.1, 63.7 77, 63 73.55, 61.9 73.4, 60.3 86.9, 64.15 87.45, 64.85 90.8, 66.35 90.55))",
        "POLYGON ((65.75 86.8, 70.5 87.45, 71.45 79, 69.85 78.85, 70.3 74.5, 64.15 73.75, 63.7 77, 64.5 81.1, 66.45 81.35, 65.75 86.8))"
    ]
})

df['geometry'] = df['geometry'].apply(loads)
gdf = geopandas.GeoDataFrame(df, geometry='geometry', crs='EPSG:27700')
topo = tp.Topology(gdf, prequantize=False)

Error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
c:\Users\IK\Desktop\SimStock_SA\t.py in 
     16 gdf = geopandas.GeoDataFrame(df, geometry='geometry', crs='EPSG:27700')
     17 
---> 18 topo = tp.Topology(gdf, prequantize=False)
     19 
     20 # gdf.boundary.plot()

~\Miniconda3\envs\sa\lib\site-packages\topojson\core\topology.py in __init__(self, data, topology, prequantize, topoquantize, presimplify, toposimplify, shared_coords, prevent_oversimplify, simplify_with, simplify_algorithm, winding_order)
     99         options = TopoOptions(locals())
    100         # execute previous steps
--> 101         super().__init__(data, options)
    102 
    103         # execute main function of Topology

~\Miniconda3\envs\sa\lib\site-packages\topojson\core\hashmap.py in __init__(self, data, options)
     16     def __init__(self, data, options={}):
     17         # execute previous step
---> 18         super().__init__(data, options)
     19 
     20         # execute main function of Hashmap

~\Miniconda3\envs\sa\lib\site-packages\topojson\core\dedup.py in __init__(self, data, options)
     25 
     26         # execute main function of Dedup
---> 27         self.output = self._deduper(self.output)
     28 
     29     def __repr__(self):

~\Miniconda3\envs\sa\lib\site-packages\topojson\core\dedup.py in _deduper(self, data)
     99             # apply linemerge on geoms containing contigious arcs and maintain
    100             # bookkeeping
--> 101             self._merge_contigious_arcs(data, sliced_array_bk_ndp)
    102 
    103             # pop the merged contigious arcs and maintain bookkeeping.

~\Miniconda3\envs\sa\lib\site-packages\topojson\core\dedup.py in _merge_contigious_arcs(self, data, sliced_array_bk_ndp)
    235             if no_ndp_arcs != no_ndp_arcs_bk:
    236                 # get the idx of the linestring which was merged
--> 237                 idx_merg_arc, consec_behavior = self._find_merged_linestring(
    238                     data, no_ndp_arcs, ndp_arcs, ndp_arcs_bk
    239                 )

TypeError: cannot unpack non-iterable NoneType object

It seems the issue is triggered by two polygons (P2 and P3) touching in a single point (65.75 86.8)
1

As soon as I split polygons by moving the point in the P2 to the left (65.75 becomes 65.7) the error disappears.

df = pandas.DataFrame({
    "name": ["P1", "P2", "P3"],
    "geometry": [
        "POLYGON ((60.05 88.85, 60.3 86.9, 61.9 73.4, 51.85 72.1, 50.8 80.5, 57.95 81.4, 57.5 85.05, 59.05 85.25, 58.85 87.15, 60.05 88.85))",
        "POLYGON ((66.35 90.55, 65.7 86.8, 64.5 81.1, 63.7 77, 63 73.55, 61.9 73.4, 60.3 86.9, 64.15 87.45, 64.85 90.8, 66.35 90.55))",
        "POLYGON ((65.75 86.8, 70.5 87.45, 71.45 79, 69.85 78.85, 70.3 74.5, 64.15 73.75, 63.7 77, 64.5 81.1, 66.45 81.35, 65.75 86.8))"
    ]
})

df['geometry'] = df['geometry'].apply(loads)
gdf = geopandas.GeoDataFrame(df, geometry='geometry', crs='EPSG:27700')
topo = tp.Topology(gdf, prequantize=False)

However, what puzzles me the most is that I cannot reproduce the error with only P1 and P2 or P2 and P3 as input data.

df = pandas.DataFrame({
    "name": ["P1", "P2"],
    "geometry": [
        "POLYGON ((60.05 88.85, 60.3 86.9, 61.9 73.4, 51.85 72.1, 50.8 80.5, 57.95 81.4, 57.5 85.05, 59.05 85.25, 58.85 87.15, 60.05 88.85))",
        "POLYGON ((66.35 90.55, 65.75 86.8, 64.5 81.1, 63.7 77, 63 73.55, 61.9 73.4, 60.3 86.9, 64.15 87.45, 64.85 90.8, 66.35 90.55))"
    ]
})

df['geometry'] = df['geometry'].apply(loads)
gdf = geopandas.GeoDataFrame(df, geometry='geometry', crs='EPSG:27700')
topo = tp.Topology(gdf, prequantize=False)
gdf.boundary.plot()

3

df = pandas.DataFrame({
    "name": ["P2", "P3"],
    "geometry": [
        "POLYGON ((66.35 90.55, 65.75 86.8, 64.5 81.1, 63.7 77, 63 73.55, 61.9 73.4, 60.3 86.9, 64.15 87.45, 64.85 90.8, 66.35 90.55))",
        "POLYGON ((65.75 86.8, 70.5 87.45, 71.45 79, 69.85 78.85, 70.3 74.5, 64.15 73.75, 63.7 77, 64.5 81.1, 66.45 81.35, 65.75 86.8))"
    ]
})

df['geometry'] = df['geometry'].apply(loads)
gdf = geopandas.GeoDataFrame(df, geometry='geometry', crs='EPSG:27700')
topo = tp.Topology(gdf, prequantize=False)
gdf.boundary.plot()

2

from topojson.

robbibt avatar robbibt commented on May 28, 2024

Thanks @IvKor! It looks like your TypeError: cannot unpack non-iterable NoneType object error is slightly different from my TypeError: 'NoneType' object is not iterable, but it's quite possible they're slightly different versions of the same thing.

I've also tried to simplify my example down to a smaller set of three polygons - unfortunately one of them is still quite large, but at least it's a bit easier to manage. Like in @IvKor's example above, the error only occurs with three polygons, not two.

polygons_subset.zip

import topojson as tp
import geopandas as gpd

# Read file
gdf = gpd.read_file('polygons_subset.geojson')

# Construct topology and simplify
topo = tp.Topology(gdf, prequantize=False)
simplified_gdf = topo.toposimplify(30).to_gdf()

Error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-4c7209c7f772> in <module>
      6 
      7 # Construct topology and simplify
----> 8 topo = tp.Topology(gdf, prequantize=False)
      9 simplified_gdf = topo.toposimplify(30).to_gdf()

~/.digitalearthau/dea-env/20200612/local/lib/python3.6/site-packages/topojson/core/topology.py in __init__(self, data, topology, prequantize, topoquantize, presimplify, toposimplify, shared_coords, prevent_oversimplify, simplify_with, simplify_algorithm, winding_order)
     99         options = TopoOptions(locals())
    100         # execute previous steps
--> 101         super().__init__(data, options)
    102 
    103         # execute main function of Topology

~/.digitalearthau/dea-env/20200612/local/lib/python3.6/site-packages/topojson/core/hashmap.py in __init__(self, data, options)
     16     def __init__(self, data, options={}):
     17         # execute previous step
---> 18         super().__init__(data, options)
     19 
     20         # execute main function of Hashmap

~/.digitalearthau/dea-env/20200612/local/lib/python3.6/site-packages/topojson/core/dedup.py in __init__(self, data, options)
     25 
     26         # execute main function of Dedup
---> 27         self.output = self._deduper(self.output)
     28 
     29     def __repr__(self):

~/.digitalearthau/dea-env/20200612/local/lib/python3.6/site-packages/topojson/core/dedup.py in _deduper(self, data)
     99             # apply linemerge on geoms containing contigious arcs and maintain
    100             # bookkeeping
--> 101             self._merge_contigious_arcs(data, sliced_array_bk_ndp)
    102 
    103             # pop the merged contigious arcs and maintain bookkeeping.

~/.digitalearthau/dea-env/20200612/local/lib/python3.6/site-packages/topojson/core/dedup.py in _merge_contigious_arcs(self, data, sliced_array_bk_ndp)
    236                 # get the idx of the linestring which was merged
    237                 idx_merg_arc, consec_behavior = self._find_merged_linestring(
--> 238                     data, no_ndp_arcs, ndp_arcs, ndp_arcs_bk
    239                 )
    240 

TypeError: 'NoneType' object is not iterable

image
image

from topojson.

robbibt avatar robbibt commented on May 28, 2024

Hey @mattijn , thanks for the reply and the great explanation! I've just tested out shared_coords=False, and had some mixed results. Both of my zipped geojson above work correctly with shared_coords=False, however @IvKor's simple example still fails.

I've also run another similar analysis with shared_coords=False, and unfortunately hit the same issue. Here's another reproducible example for a small area containing three polygons:

image

import topojson as tp
import geopandas as gpd
import json

# Load JSON geometry
json_string = '{"type": "FeatureCollection", "features": [{"id": "0", "type": "Feature", "properties": {"certainty": 4}, "geometry": {"type": "Polygon", "coordinates": [[[556395.0, -2289375.0], [556485.0, -2289375.0], [556485.0, -2289735.0], [556455.0, -2289735.0], [556455.0, -2289705.0], [556395.0, -2289705.0], [556395.0, -2289735.0], [556335.0, -2289735.0], [556335.0, -2289705.0], [556365.0, -2289705.0], [556365.0, -2289675.0], [556395.0, -2289675.0], [556395.0, -2289615.0], [556365.0, -2289615.0], [556365.0, -2289555.0], [556335.0, -2289555.0], [556335.0, -2289465.0], [556365.0, -2289465.0], [556365.0, -2289435.0], [556395.0, -2289435.0], [556395.0, -2289375.0]]]}}, {"id": "1", "type": "Feature", "properties": {"certainty": 4}, "geometry": {"type": "Polygon", "coordinates": [[[556065.0, -2289075.0], [556155.0, -2289075.0], [556155.0, -2289135.0], [556125.0, -2289135.0], [556125.0, -2289195.0], [556095.0, -2289195.0], [556095.0, -2289225.0], [556065.0, -2289225.0], [556065.0, -2289375.0], [556095.0, -2289375.0], [556095.0, -2289465.0], [556125.0, -2289465.0], [556125.0, -2289525.0], [556155.0, -2289525.0], [556155.0, -2289615.0], [556125.0, -2289615.0], [556125.0, -2289645.0], [556155.0, -2289645.0], [556155.0, -2289675.0], [556125.0, -2289675.0], [556125.0, -2289735.0], [556155.0, -2289735.0], [556155.0, -2289765.0], [556185.0, -2289765.0], [556185.0, -2289795.0], [556215.0, -2289795.0], [556215.0, -2289825.0], [556245.0, -2289825.0], [556245.0, -2289855.0], [556305.0, -2289855.0], [556305.0, -2289825.0], [556335.0, -2289825.0], [556335.0, -2289795.0], [556365.0, -2289795.0], [556365.0, -2289825.0], [556395.0, -2289825.0], [556395.0, -2289855.0], [556455.0, -2289855.0], [556455.0, -2289825.0], [556485.0, -2289825.0], [556485.0, -2289885.0], [556455.0, -2289885.0], [556455.0, -2289915.0], [556425.0, -2289915.0], [556395.0, -2289915.0], [556395.0, -2289945.0], [556365.0, -2289945.0], [556305.0, -2289945.0], [556305.0, -2289975.0], [556275.0, -2289975.0], [556245.0, -2289975.0], [556245.0, -2290005.0], [556215.0, -2290005.0], [556155.0, -2290005.0], [556155.0, -2290035.0], [556125.0, -2290035.0], [556095.0, -2290035.0], [556095.0, -2290065.0], [556065.0, -2290065.0], [556035.0, -2290065.0], [556035.0, -2290095.0], [556005.0, -2290095.0], [556000.0, -2290095.0], [556000.0, -2289135.0], [556005.0, -2289135.0], [556035.0, -2289135.0], [556035.0, -2289105.0], [556065.0, -2289105.0], [556065.0, -2289075.0]]]}}, {"id": "2", "type": "Feature", "properties": {"certainty": 0}, "geometry": {"type": "MultiPolygon", "coordinates": [[[[556000.0, -2290095.0], [556005.0, -2290095.0], [556035.0, -2290095.0], [556035.0, -2290065.0], [556065.0, -2290065.0], [556095.0, -2290065.0], [556095.0, -2290035.0], [556125.0, -2290035.0], [556155.0, -2290035.0], [556155.0, -2290005.0], [556215.0, -2290005.0], [556245.0, -2290005.0], [556245.0, -2289975.0], [556275.0, -2289975.0], [556305.0, -2289975.0], [556305.0, -2289945.0], [556365.0, -2289945.0], [556395.0, -2289945.0], [556395.0, -2289915.0], [556425.0, -2289915.0], [556455.0, -2289915.0], [556455.0, -2289885.0], [556485.0, -2289885.0], [556485.0, -2290250.0], [556000.0, -2290250.0], [556000.0, -2290095.0]]], [[[556485.0, -2289000.0], [556485.0, -2289375.0], [556455.0, -2289375.0], [556395.0, -2289375.0], [556395.0, -2289435.0], [556365.0, -2289435.0], [556365.0, -2289465.0], [556335.0, -2289465.0], [556335.0, -2289555.0], [556365.0, -2289555.0], [556365.0, -2289615.0], [556395.0, -2289615.0], [556395.0, -2289675.0], [556365.0, -2289675.0], [556365.0, -2289705.0], [556335.0, -2289705.0], [556335.0, -2289735.0], [556395.0, -2289735.0], [556395.0, -2289705.0], [556455.0, -2289705.0], [556455.0, -2289735.0], [556485.0, -2289735.0], [556485.0, -2289825.0], [556455.0, -2289825.0], [556455.0, -2289855.0], [556395.0, -2289855.0], [556395.0, -2289825.0], [556365.0, -2289825.0], [556365.0, -2289795.0], [556335.0, -2289795.0], [556335.0, -2289825.0], [556305.0, -2289825.0], [556305.0, -2289855.0], [556245.0, -2289855.0], [556245.0, -2289825.0], [556215.0, -2289825.0], [556215.0, -2289795.0], [556185.0, -2289795.0], [556185.0, -2289765.0], [556155.0, -2289765.0], [556155.0, -2289735.0], [556125.0, -2289735.0], [556125.0, -2289675.0], [556155.0, -2289675.0], [556155.0, -2289645.0], [556125.0, -2289645.0], [556125.0, -2289615.0], [556155.0, -2289615.0], [556155.0, -2289525.0], [556125.0, -2289525.0], [556125.0, -2289465.0], [556095.0, -2289465.0], [556095.0, -2289375.0], [556065.0, -2289375.0], [556065.0, -2289225.0], [556095.0, -2289225.0], [556095.0, -2289195.0], [556125.0, -2289195.0], [556125.0, -2289135.0], [556155.0, -2289135.0], [556155.0, -2289075.0], [556065.0, -2289075.0], [556065.0, -2289105.0], [556035.0, -2289105.0], [556035.0, -2289135.0], [556005.0, -2289135.0], [556000.0, -2289135.0], [556000.0, -2289000.0], [556485.0, -2289000.0]]]]}}]}'
json_data = json.loads(json_string)

# Convert to GeoDataFrame
gdf = gpd.GeoDataFrame.from_features(json_data["features"])

# Construct topology and simplify (with shared_coords=False)
topo = tp.Topology(gdf, shared_coords=False, prequantize=False)
simplified_gdf = topo.toposimplify(30).to_gdf() 

Error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-944ce03f791f> in <module>
     11 
     12 # Construct topology and simplify
---> 13 topo = tp.Topology(gdf, shared_coords=False, prequantize=False)
     14 simplified_gdf = topo.toposimplify(30).to_gdf()

~/.digitalearthau/dea-env/20200612/local/lib/python3.6/site-packages/topojson/core/topology.py in __init__(self, data, topology, prequantize, topoquantize, presimplify, toposimplify, shared_coords, prevent_oversimplify, simplify_with, simplify_algorithm, winding_order)
     99         options = TopoOptions(locals())
    100         # execute previous steps
--> 101         super().__init__(data, options)
    102 
    103         # execute main function of Topology

~/.digitalearthau/dea-env/20200612/local/lib/python3.6/site-packages/topojson/core/hashmap.py in __init__(self, data, options)
     16     def __init__(self, data, options={}):
     17         # execute previous step
---> 18         super().__init__(data, options)
     19 
     20         # execute main function of Hashmap

~/.digitalearthau/dea-env/20200612/local/lib/python3.6/site-packages/topojson/core/dedup.py in __init__(self, data, options)
     25 
     26         # execute main function of Dedup
---> 27         self.output = self._deduper(self.output)
     28 
     29     def __repr__(self):

~/.digitalearthau/dea-env/20200612/local/lib/python3.6/site-packages/topojson/core/dedup.py in _deduper(self, data)
     99             # apply linemerge on geoms containing contigious arcs and maintain
    100             # bookkeeping
--> 101             self._merge_contigious_arcs(data, sliced_array_bk_ndp)
    102 
    103             # pop the merged contigious arcs and maintain bookkeeping.

~/.digitalearthau/dea-env/20200612/local/lib/python3.6/site-packages/topojson/core/dedup.py in _merge_contigious_arcs(self, data, sliced_array_bk_ndp)
    236                 # get the idx of the linestring which was merged
    237                 idx_merg_arc, consec_behavior = self._find_merged_linestring(
--> 238                     data, no_ndp_arcs, ndp_arcs, ndp_arcs_bk
    239                 )
    240 

TypeError: 'NoneType' object is not iterable

from topojson.

mattijn avatar mattijn commented on May 28, 2024

Thanks for trying! It’s now clear that current implementation (both strategies) is inadequate..

from topojson.

robbibt avatar robbibt commented on May 28, 2024

Hi @mattijn, this is great! How can I access the updated version? Will the fix be included if I do another pip install topojson?

Edit: I think this does the trick:

!pip install --user git+https://github.com/mattijn/topojson/

from topojson.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.