tsinghua-mars-lab / hdmapnet Goto Github PK
View Code? Open in Web Editor NEWLicense: GNU General Public License v3.0
License: GNU General Public License v3.0
Hi, thx for your nice work! When I train the model, I find a problem. In train dataset, IOU can achieve about[0.8 0.6 0.8], while its eval IOU is only about [0.04 0.01 0.05]. Here it is:
So Is there any wrong with function eval_iou()? It seems to be an order of magnitude smaller. Thanks for your time.
Hey, guys, according to my test, the whole post-processing process takes 900ms, which will lead to low FPS?
Thank you for your amazing work.
I notice that there is a resize operation in the data fetching phase that shrinks the image size.
However, to keep the projection relation between ground truth 3D lanes and the (resized)image pixels, we also need to adjust the camera intrinsic during the data fetching phase. (For example, if the image is downsampled by two, the first two rows of the intrinsic matrix should also become one half smaller)
In HDMapNet, because we use linear layers to learn the mapping between the front view and the BEV view, there is no need for intrinsic parameters, and the code works fine.
However, in IPMNet and Lift-Splat-Shoot, correct camera intrinsic parameters are essential to correctly project the FV features to BEV. The inconsistent camera intrinsic matrix could probably hurt the performance of IPMNet and LSS in the experiments mentioned in your paper. Did the ablation experiments in your paper also use the code in this repo to reproduce?
In https://pytorch.org/get-started/previous-versions/, there is not torch=1.10.2+cu113
I'm looking forword to the code. thanks.
e.g.,
step1 train.py
step2 test.py
step3 evalute.py
Nice work! I'm sure the whole code, including dataloader, model and evaluation, will provide the community a great guide to this task.
I wanted to ask if I could get the pre-trained weights of HDMapNet models that produced the results in the paper.
I would really appreciate it.
Thank you very much
Hi, @liqi17thu
I am wondering which set is applied in Table.1, validation set or mini-validation set? Thanks.
感谢提供标签生成和评估脚本
1.Little bug in your devkit
data/rasterize.py file in line 22 you have defination:
def mask_for_lines(lines, mask, thickness, idx)
while in line 53 and 55 you switch the order of the last two argument:
mask_for_lines(new_single_line, map_mask, idx, thickness)
this is not correct and it will influence the data flow of metric evaluation enoumously
2.Some data label generating details
the visualization of the label generated based on your orginal script is:
the visualization of the label generated after my modification is:
which is actually more reasonable
3.Question about the papers:
(1)
I actually read your paper thoroughly in great detail and reimplement your decoder part into my architecture with some changes. But I keep your direction predict part. One confusion I have is that: In your paper you said that you use softmax as the activation function for classification and your gt label are all zero except the two opposite direction, which is 1. I am wondering what kind of loss function you use for direction prediction. Do you use the ordinary cross entropy even though you have two 1's? Or do you use binary cross entropy like what multilabel prediction would usually do? But if later, Then why not use sigmoid as the activation functions.
(2)
It can be seen from your script that your bev map is [-30, 30, 0.15] [-15,15,0.15]. Is this also the default settings in Table 2 in your paper? This is not clear in your paper. Also I am wondering the influence of x-y-range and resolution (in your script it is 0.15) on the metric
i'm trying to use my own dataset to train hdmapnet, and my cameras are not horizontal(differ from nusenes),so i want to know that is the plane generated from z_pitch_yaw better ? @simonyilunw @hangzhaomit @Mrmoore98 @liqi0126
Hi, @liqi17thu @hangzhaomit ,
When will the code be released? The link (https://github.com/Tsinghua-MARS-Lab/HDMapNet-dev) attached in your website is broken now.
Thanks~
hi,
Is the evaluation result in Table 1 of the paper on the nuscene val set or the mini-val set?
thanks!!!
按我看代码的理解,在运行vis_pred.py时相当于是使用训练得到的model.pt对数据集进行推理以及可视化了吧?但是这里在读取数据集时使用的仍是data/dataset.py里的semantic_dataset函数,在这个python脚本里我注意到有读取nuscenes数据集的map和Lidar的数据。这个是用来做什么的呢?在推理阶段不可以直接输入六张图片得到地图分割的结果吗?
希望能解答一下
When will you release complete code/model?
How to figure this out, thx!
TRAIN[ 0]: [ 449/7031] Time: 0.4546 Loss: 1.3768 IOU: [0.000 0.000 0.000]
this is log of my training, IOU is always 0.0
is that right?
Hi, thanks for your great work! I have a question regarding your architecture of the BEV decoder, according to the paper:
The BEV decoder is a fully convolutional network with 3 branches, namely semantic segmentation branch, instance embedding branch, and direction prediction branch…ResNet with three blocks is used as the BEV decoder.
If I understand it right, the input BEV feature to the decoder has shape , the segmentation branch has output shape (K' different for each branch). It's not very clear to me how a ResNet can be applied here.
My guess is you use a U-Net structure with a shared encoder(which uses the last 3 encoders of ResNet with dilated convs to avoid downsampling) and a separate decoder for each branch, but this seems unnecessary given that the input is already a feature vector. Or do you mean each decoder is simply made up of 3 residual blocks? Thanks!
Also as a minor request, could you upload icon/car.png
so that we can run vis_label.py
with the ego car icon included in the g.t. map label as well?
In vectorize process, what dose the sort_points_by_dist function do?
HI, as mentioned in the title,
I'm not able to install the specified requirement.txt
thus not able to run this codebase.
Could you provide a docker image for us to run your code?
I appreciate your effort a lot.
'MultiLineString' object is not iterable
Hi, first thanks for your great job!
I wonder whether you will release your trained weights? thanks.
Thanks for your great research. Will you public complete project?
I saw a temporal fusion in the paper,this part of the code is not open source?
Hi, nice work!
What kind of GPU do you used that produced the results in the paper?
Thanks
Hi, thx for your nice work! When I read the code and try ro run python vis_label.py
to produce label, I meet this error:XIO: fatal IO error 0 (Success) on X server "localhost:10.0 after 18050 requests (18050 known processed) with 862 events remaining.
How can I solve this? I wonder if you could give some advice. Thanks for your time!
Hi, thank you for your complete and great code.
When i tried to use the command 'python evaluate.py' as readme.md said, I got some error message:
AttributeError: 'NoneType' object has no attribute 'seek'. You can only torch.load from a file that is seekable. Please pre-load the data into a buffer like io.BytesIO and try to load from it instead.
I have trained the model, but i can't solve the problem above.
Besides, there is another question: i could't find a argument named 'result_path' as your readme.md said. Should i add the argument by myself or the version of code has some different?
Thank you for your time.
in which folder predicted result save?
Hello I have a Data set, how to predict my data and detect its lane and how to visualize it?
if we want to get visualize all of the sample in nuScens what's should I do?
Hi,
it is March now. Any plan to release the complete project. I am looking forward to it.
Hello,
I have some questions when reviewing the code base. I know the function of IPM Net in the model. I think the grid plane (BEV map) should transfer to the pixel level coordinate system based on the code logics. That's to say, the Ln 192 and 193 of homography.py is to multiply the external trins, intrins, and post_RTs to transform the grid plane.
However, the Ks (intrins) seems to be an identity matrix because the Ln 71 of the hdmapnet.py is empty. The Ks is not initialized by the intrins of the camera. It seems a strange error here.
By the way, why the post_RTs is None at the Ln 78 of the hdmapnet.py? It should be none or it is an error?
Great work!
Just curious if the world coordinates are available...
I just found that --direction-pred
only works when --instance-seg
is activated
because direction_pred
and instance_seg
share the self.up1_embedded
layer
Line 131 in 7911227
i guessed it is kinda typo. because there is another layer designed for direction_pred
, 'self.up1_direction'. but never used
Line 102 in 7911227
To create submission file try to run -
python export_gt_to_json.py
After running above file you will get submission.json
file.
To evaluate file
python evaluate_json.py --result_path [submssion_file_path]
Hope this will save time for those who getting stuck with Readme
instructions
Hello, I am really amazed by your work, HDMapNet shows great performance.
But apart from HDMapNet, the other networks in your code including VPN, Lift splat and IPMNet all have bugs in it and are unable to run, which really harm the credibility of this work and is really regretful.
I do believe that maybe you have others version to successfully run VPN, Lift splat and IPMNet but has not yet update them to your HDMapNet pipeline, could you please provide the correct code of them? Thank you very much!
Hello! This is a great work! I am a new commer in this field, and I am confused with the threshold setting of CD in evaluate_json.py, where you set THRESHOLDS = [2, 4, 6], which is different from the [0.2, 0.5, 1] that is claimed in your paper. Will this cause different evaluation result of AP? I would appreciate it a lot if someone can explain it to me.
Hi, first thanks for your great job!
I wonder whether you will release your training code? thanks.
hi,
What is the resolution of the images sent to the PV image encoder?
thanks!
I am trying to use vis_label.py with nuScenes v1.0 mini dataset but it is not working.
This is the error
Traceback (most recent call last): File "/content/HDMapNet/vis_label.py", line 95, in <module> vis_label(args.dataroot, args.version, args.xbound, args.ybound) File "/content/HDMapNet/vis_label.py", line 24, in vis_label dataset = HDMapNetDataset(version=version, dataroot=dataroot, data_conf=data_conf, is_train=False) File "/content/HDMapNet/data/dataset.py", line 31, in __init__ self.nusc = NuScenes(version=version, dataroot=dataroot, verbose=False) File "/usr/local/lib/python3.10/dist-packages/nuscenes/nuscenes.py", line 62, in __init__ assert osp.exists(self.table_root), 'Database version not found: {}'.format(self.table_root) AssertionError: Database version not found: dataroot=/content/HDMapNet/dataset/nuScenes/v1.0-mini
Great work for bird-eye view perception with multi-view images. I am wondering how long can it take before the source code is released
Thanks for the great work! I run the vis_labe.py following the readme. And I found that the gt lines projected on surrounding views had some misalignment with the real line elements. The results may be attributed to supposing the Z=0 for all map elements in https://github.com/Tsinghua-MARS-Lab/HDMapNet/blob/main/vis_label.py#L75-L76
Will the gt misalignment between top-down map and perspective images hinder the performance of the surrounding branch of HDMapNet? And how to align the gt?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.