fabvio/cascade-ld

Lane detection and classification in an end-to-end Deep Learning fashion

Lane Detection and Classification using Cascaded CNNs

This is the repository of the paper "Lane Detection and Classification using Cascaded CNNs". Only the inference code is public, and it is documented using a Jupyter Notebook. To see the system in action, please open the notebook.

You can find the dataset here and the class labels here.

Citation

If the code, the pretrained model or the idea is useful for you, we would appreciate a citation to our paper:

@misc{pizzati2019lane,
    title={Lane Detection and Classification using Cascaded CNNs},
    author={Fabio Pizzati and Marco Allodi and Alejandro Barrera and Fernando García},
    year={2019},
    eprint={1907.01294},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

cascade-ld's Issues

About the double line

In the TuSimple dataset, all the double lines are labeled like single lines. I mean, for example, a double solid yellow line is labeled with many points, but if you only project the polyline with a width of 5px. There will be no difference between a double line and a single line. How can you classify double lines and single lines? Can you tell me how it performs on double lines classification?

I am confused with some details in the paper and looking forward to your code for training

About Tusimple test dataset

Hello, thank you very much for publishing the lane category of the Tusimple train data, which is of great help to my experiment. Could you please publish the category of the Tusimple test set?

MORE INFO ON LICENSE

Can u provide more info on license "Example providing MIT license ", mentioning openness for reuse even commercially, like usage of this trained model for inference in a Software without restriction, limitation to use, copy, modify, merge, publish, distribute etc etc.. current license by citation is not clear

How to pretrain ERFNet

Hey Fabvio,

Appreciate your great work here! I'm trying to reproduce the results and got some issue with the pretrained ERFNet. Correct me if I'm wrong. So we have to pretrain a pixel-wise segmentation network with 0 for background and 1 for lanes. Then we are going to replace BCE loss with the loss function you provided in this repo. But if we specify num_classes=5, we are going to have a 5-channel output at the end of Decoder. Should I add a 1 by 1 filter to get a one-channel output and then a sigmoid function after that in the first training stage? And throw 1 by filter and sigmoid away when we fine-tune the network?

Can you explain how you get the descriptors?

In your paper, the points in descriptor is 'extracted in the original image and arranged in squared images'. Does this means that you need 128*128 points in original image to be arranged in squared descriptor?

can you provide train code?

can you provide train code? I am leaning the method, thanks.

More information about LCNet

Friend please help me. Please explain to me how the descriptors of different lanes could be classified simultaneously. I really don't understand how this happens. How does the output get 4 channels?

Train Code?

Hello sir,
Could you please provide train code, if not please suggest how to train for our own dataset

Creating a {descriptor, class} object dataset

Hi Fabvio,

Great work. I would love to know when the training code will be released. Meanwhile, I'm trying to train only the classifier CNN and for that, I'm trying to create a {descriptor, class} dataset as mentioned in the paper. In my approach, I'm directly drawing the lanes on an image(binary) from the tusimple dataset lane annotations, resizing the original and binary image to 360*640, converting it to torch tensors, and passing it to extract_descriptor function.

Expected Output -
For a four-lane image, four descriptors should be extracted

Actual output -
No descriptor is extracted

I am wondering if my approach is correct and if it is correct, then why is the extract_descriptor function is not working as expected.

How can the segmentation network discriminate different lanes?

Hi, @fabvio
Thanks for your great work. I've met some problems about training my own network. The main problem is how can the segmentation network discriminate different lanes. I'm trying the position of the lane to tell the difference which only leads the segmentation network learn other things. Did you use some post-processing methods?
Looking forward to your reply.

A issue about the lane line pixel coordinates

Thanks for your sharing!Would you please tell me what to do if I want to get the lane line pixel coordinates after the instance segmentation work?

[extract_descriptors] function failing when number of points belonging to a lane is zero

In the following snippet of main.py

single_lane = label.eq(i).view(-1).nonzero().squeeze()
        
# As they could be not continuous, skip the ones that have no points
if single_lane.numel() == 0:
    continue

# Points to sample to fill a squared desciptor
sample = torch.zeros(DESCRIPTOR_SIZE * DESCRIPTOR_SIZE)
if torch.cuda.is_available():
    sample = sample.cuda()
    
sample = sample.uniform_(0, single_lane.size()[0] - eps).long()

if single_lane has no points, then single_lane.size()[0] throws error. This scenario happened in one image of mine.

Need to add

if single_lane.numel() == 0 or len(single_lane.size()) == 0:
    continue

Can you release the code for training?

Running the main ipython script

Hi, your network seems promising and I'd love to see what it can do; however, I'm unable to run main.ipynb in jupiter. I get an error about matplotlib
AttributeError Traceback (most recent call last)
/usr/local/lib/python3.5/dist-packages/ipykernel/pylab/backend_inline.py in show(close, block)
37 display(
38 figure_manager.canvas.figure,
---> 39 metadata=_fetch_figure_metadata(figure_manager.canvas.figure)
40 )
41 finally:

/usr/local/lib/python3.5/dist-packages/ipykernel/pylab/backend_inline.py in _fetch_figure_metadata(fig)
172 """Get some metadata to help with displaying a figure."""
173 # determine if a background is needed for legibility
--> 174 if _is_transparent(fig.get_facecolor()):
175 # the background is transparent
176 ticksLight = _is_light([label.get_color()

/usr/local/lib/python3.5/dist-packages/ipykernel/pylab/backend_inline.py in _is_transparent(color)
193 def _is_transparent(color):
194 """Determine transparency from alpha."""
--> 195 rgba = colors.to_rgba(color)
196 return rgba[3] < .5

AttributeError: module 'matplotlib.colors' has no attribute 'to_rgba'

What versions of python and necessary libraries do you use to make it work?

Can you explain the loss function used for training ? what are the steps you followed for data pre-processing and what does the shape and the label look like.