songtaohe/laneextraction

laneextraction's Issues

When will you release the inference code and evaluation metrics

Enviroment packages and version

Hi,

I had some problem when I run this code because package's version is incorrect, such as Tensorflow, Scipy.

Could you please provide an environment packages and version list to ensure that I can create a correct environment?

Thank you so much!

Questions about package versions

Hi,

Thanks for providing such great source code and dataset. I would like to train your model but found that it's a little bit hard to create an environment.

I used Conda to create a virtual environment with python==3.5.2 and installed the tensorflow==1.15.0. But I found that it has some import errors:

Traceback (most recent call last):
File "train.py", line 2, in
from model import LaneModel
File "/mnt/c/Users/yonliang/Sources/LaneExtraction/code/laneAndDirectionExtraction/model.py", line 7, in
from cnnmodels.resnet34unet import resnet34unet_v3, unet, unet_dilated
File "/mnt/c/Users/yonliang/Sources/LaneExtraction/code/cnnmodels/resnet34unet.py", line 13, in
from resnet import resblock as residual_block
File "/home/lyq/miniconda3/envs/tf/lib/python3.5/site-packages/resnet/init.py", line 2, in
from .resnet152 import ResNet152
File "/home/lyq/miniconda3/envs/tf/lib/python3.5/site-packages/resnet/resnet152.py", line 27, in
from keras.applications.imagenet_utils import _obtain_input_shape
ImportError: cannot import name '_obtain_input_shape'

Could you provide a requirements.txt file that we can follow?

Thanks,
Yongqing

How can I install the library osm.graph_ops to import the function graphDensifyPixel from the file segtograph/segtograph.py?, I could not find this library anywhere. If the function is not available anymore, what is this function for?

Questions about how to solve the extremely unbalanced data

Hello, Songtao:
When I was reproducing your program laneAndDirectionExtraction in PyTorch, I encountered the extremely imbalanced data that made it difficult for the network to learn and extract roads (roads only account for 0.37% in the training set). Even if I followed the celoss and dice loss in the paper and this program. Could you share how you solve this problem? I would be very grateful if you could reply!

Weights of trained models

Thank you very much for providing the code for your nice publication!

Would it be possible to share the weights of the trained networks?

Cannot reproduce the result

Dear Songtao,

Hi, I recently tried the released code of "Lane-Level Street Map Extraction from Aerial Imagery" on github, nice work! And thank you for releasing the dataset and the code.

However, the inference codes of turningLaneExtraction and turningLaneValidation are missing, so I re-implemented them by myself. I cannot reproduce the results provided in your paper, and here are some problems that I encountered, could you please give me some advice or hints? Thank you so much for your help.

1. For endpoint extraction, I followed instructions in your paper by (1) obtain the skeleton of the segmentation of straight lane centerlines, and (2) extract endpoints of these skeletons. Here is one sample direction map I obtained:      
   

Red points are obtained endpoints. As you can see, there are incorrect disconnections on the straight-lane centerlines. But the general performance is fine. Do my results look good to you?

2. Based on obtained endpoints, I run inference of turnLaneValidation. However, here comes the problem: since you train the LinkModel by ground-truth direction maps, which are quite different from the predicted ones, the model presents very bad performance. It ignores almost all turning lanes. For example:
    
   Cyan points are node A and node B defined in the paper. These two points should be connected by a turning lane. But the model outputs a very strong reject to this connection ( the softmax result is: [1.4065192033285712e-07, 0.9999998807907104], indicating that the model is very sure that these two points should not be connected). From my perspective, the gap between the training data and inference data leads to this problem. How do you solve this?

Thank you so much for your kind help, and looking forward to your reply!

Best regards

osm and satellite plackages are not available

Hi,

could you provide installation guideline on installing osm and satellite packages? It seems there are not available.

@songtaohe

TuriningLaneValiation训练出错 / Error during TuriningLaneValiation Training

您好！

我根据环境需求配置好了环境，并且成功跑通了LaneAndDirectionExtraction和TurningLaneExtration的代码，并且训练得到了模型，但是我在运行TuriningLaneValiation/train的时候发现会随机停住，不能继续训练下去。

[ERROR] links[ 2 ][ 6 ]: (1949, 661)  is not in pos2nid
ind is : _14
pos2nid keys is :  [(1744, 1350), (1842, 674), (1527, 1675), (97, 1550), (1678, 1368), (1398, 1934), (1843, 610), (143, 1501), (416, 1732), (168, 1649), (577, 1757), (786, 842), (458, 1855), (405, 1753), (304, 1573), (186, 1661), (1713, 1447), (114, 1533), (1579, 1600), (495, 742), (1556, 1692), (616, 637), (440, 1694), (548, 632), (1859, 1071), (224, 579), (440, 1845), (559, 1775), (1872, 689), (160, 480), (1767, 1359), (1869, 995), (533, 1815), (524, 1830), (456, 1673), (291, 1596), (245, 1649), (1834, 989), (1692, 1432), (1874, 608), (1832, 1059), (262, 1629), (159, 1477), (1601, 1618), (1917, 1057), (861, 887), (292, 483), (846, 827), (1644, 1706), (620, 744)]
pos2nid is :  dict_keys([(1744, 1350), (1842, 674), (1527, 1675), (97, 1550), (1678, 1368), (1398, 1934), (1843, 610), (143, 1501), (416, 1732), (168, 1649), (577, 1757), (786, 842), (458, 1855), (405, 1753), (304, 1573), (186, 1661), (1713, 1447), (114, 1533), (1579, 1600), (495, 742), (1556, 1692), (616, 637), (440, 1694), (548, 632), (1859, 1071), (224, 579), (440, 1845), (559, 1775), (1872, 689), (160, 480), (1767, 1359), (1869, 995), (533, 1815), (524, 1830), (456, 1673), (291, 1596), (245, 1649), (1834, 989), (1692, 1432), (1874, 608), (1832, 1059), (262, 1629), (159, 1477), (1601, 1618), (1917, 1057), (861, 887), (292, 483), (846, 827), (1644, 1706), (620, 744)])
< 2 ! >

[ERROR] links[ 2 ][ 1 ]: (1998, 1852)  is not in pos2nid
ind is : _11
pos2nid keys is :  [(314, 1471), (888, 1885), (397, 1389), (1891, 1846), (268, 1363), (1934, 1787), (822, 1828), (690, 1597), (1918, 1866), (883, 1824), (675, 1703), (624, 1580), (552, 1679), (1903, 1783)]
pos2nid is :  dict_keys([(314, 1471), (888, 1885), (397, 1389), (1891, 1846), (268, 1363), (1934, 1787), (822, 1828), (690, 1597), (1918, 1866), (883, 1824), (675, 1703), (624, 1580), (552, 1679), (1903, 1783)])
< 2 ! >

 [ERROR] links[ 2 ][ 6 ]: (278, 1680)  is not in pos2nid
ind is : _14
pos2nid keys is :  [(1203, 159), (290, 924), (1538, 1247), (1270, 49), (395, 1692), (1704, 234), (304, 672), (173, 1298), (1290, 52), (207, 595), (366, 1705), (1604, 1229), (1688, 274), (295, 577), (1352, 150), (238, 1339), (1896, 1281), (360, 296), (1258, 1147), (1213, 112), (274, 1333), (231, 861), (234, 963), (1617, 1107), (210, 962), (1342, 194), (1192, 181), (1532, 194), (222, 1265), (1516, 235), (256, 868), (1335, 219), (1680, 301), (374, 1632), (252, 1267), (1223, 1097), (1541, 169), (2021, 1239), (1355, 126), (278, 662), (1512, 261), (1308, 1113), (1498, 1148), (340, 1628), (1714, 211), (1217, 94), (1927, 1168), (1592, 139), (327, 584), (1614, 144)]
pos2nid is :  dict_keys([(1203, 159), (290, 924), (1538, 1247), (1270, 49), (395, 1692), (1704, 234), (304, 672), (173, 1298), (1290, 52), (207, 595), (366, 1705), (1604, 1229), (1688, 274), (295, 577), (1352, 150), (238, 1339), (1896, 1281), (360, 296), (1258, 1147), (1213, 112), (274, 1333), (231, 861), (234, 963), (1617, 1107), (210, 962), (1342, 194), (1192, 181), (1532, 194), (222, 1265), (1516, 235), (256, 868), (1335, 219), (1680, 301), (374, 1632), (252, 1267), (1223, 1097), (1541, 169), (2021, 1239), (1355, 126), (278, 662), (1512, 261), (1308, 1113), (1498, 1148), (340, 1628), (1714, 211), (1217, 94), (1927, 1168), (1592, 139), (327, 584), (1614, 144)])
< 2 ! >

通过检查代码后发现问题出现在turingLaneValidation/dataloader.py，大概168行。这里检查了links的数据是否在pos2nid中，但是部分数据集的数据并不满足这个条件，所以会执行最后的exit()退出当前的dataloader子进程，但是另外的dataloader子进程也因此卡住没有继续训练下去。

if (links[2][j][-1][0], links[2][j][-1][1]) not in pos2nid:
    print('\n', '[ERROR] links[', 2,'][', j,']:', (links[2][j][-1][0], links[2][j][-1][1]),' is not in pos2nid')
    print('ind is :', ind)
    print('pos2nid keys is : ', list(pos2nid.keys()))
    print('pos2nid is : ', pos2nid.keys())
    print('< 2 ! >')
    exit()

我对代码唯一的改动是将links[].keys()修改为了list(links[].keys())，因为我使用的的是python3，python3的dict.keys()方法返回的不是列表，而是视图对象，加上list()方法将其转换为列表。其他的代码除了print输出内容都没有改动。

我的数据集是运行create_training_data.py来生成，并且在LaneAndDirectionExtraction和TurningLaneExtration的训练中没有问题。

我不知道该怎么解决这个问题，我想对数据集进行完整的训练，不知道是数据集的数据有问题还是故意为之，让其直接中止训练。希望可以得到您的答疑解惑。

not found hdmapeditor

Hi

I am trying to read the dataset, but it seems the file hdmapeditor is not available. Where can I find it?

@songtaohe

download.sh is empty

Hi

I have opened download.sh file to download the pretrained models, but it is empty. Where can I find the models?

@songtaohe