beniko95j/mlf-vo

Multi-Layer Fusion Visual Odometry

mlf-vo's Issues

Training Custom dataset

@Beniko95J
Hello,
I would like to appreciate your efforts. I have successfully implemented pretrained model with the kitti dataset but when applied with my own dataset, there is huge difference in results. So i would like to know regarding the custom dataset training? do you have any plan to opensource the training file? my results are given below.

looking forward to your feedback....
regards
arkin

Multiple training sessions with different results

If we train multiple models with exactly the same configuration, they should perform the same, but there are some differences in their performance. Have you encountered this problem, please? Thanks!

about the KITTI odometry result files

Are the KITTI odometry result files of the MKF-VO in "MLF-VO/kitti_odom_eval/comparison/others/ablation_models/." or "MLF-VO/kitti_odom_eval/comparison/others/tsn_split/ours/" ?
I have evaluated the pose files in those two directories and can not match the results reported in the paper.
Could you give me some advises?
Thanks!

About the result of the training

Dear officer:
It seems that the final train result can t reach the result you mentioned in the paper and also the master branch .I train the model in the same manner using NVIDIA RTX 3090 and other option are the same. My question is can you provide more detail about how to training?

About the "RumtimeError"

Hi!
MLF-VO is a remarkable work. But I met a question when I train the model following the readme.md:

"RuntimeError: DataLoader worker (pid 42438) exited unexpectedly with exit code 1. Details are lost due to multiprocessing. Rerunning with num_workers=0 may give better error trace."

However, I cannot solve this issue when set "num_workers=0".
Would you mind giving me some suggestions?

Leveraging velodyne laser data

Dear @Beniko95J ,

Thanks for your work!

Since one of the main contributions of MLF-VO is the multi-layer fusion of rgb and depth information, and as KITTI dataset also provides Lidar point cloud, I wonder if I could create input depth images by performing depth completion [https://www.cvlibs.net/datasets/kitti/eval_depth.php?benchmark=depth_completion] on current image and Lidar scan.

Theoretically, MLF-VO is able to obtain good results with these ground-truth depths. But I don't quite understand what exactly is this cur_disp (

MLF-VO/run_odometry.py

Line 117 in 659bc82

disp_inputs = torch.cat([prev_disp.repeat(1, 3, 1, 1), cur_disp.repeat(1, 3, 1, 1)], dim=1)

) , is it the disparity, the inverse depth or the depth. Could you please specify the mean and also the unity of it? Thanks in advance!

Yours,
X. Shen

A general question

你好，想问一个关于Channel exchange 的问题，A和B 做 CE后， 假设A变得越来越important，B越来越不important，为什么最后还是fusionA和B呢， 而不是只用A？？

train file and loss function

Thank you for your work. Do you have any plans for open source loss functions and training files? Thanks!

ModuleNotFoundError: No module named 'layers'

Hi,
Got this error, installed layers using pip but in vain. complete error is given below;

File "/home/ubuntu/Music/MLF-VO/run_odometry.py", line 15, in <module> from third_party.monodepth2.networks import ResnetEncoder File "/home/ubuntu/Music/MLF-VO/third_party/monodepth2/networks/__init__.py", line 2, in <module> from .depth_decoder import DepthDecoder File "/home/ubuntu/Music/MLF-VO/third_party/monodepth2/networks/depth_decoder.py", line 14, in <module> from layers import * ModuleNotFoundError: No module named 'layers'

Regards
arkin

About the Invert

Hi, I found that the code about infer pose in test file makes me confuse in test file both in master and dev branchs. When using the function "transformation_from_parameters", why it need to set the invert "True"? In my opinion, the NN outputs the relative transformation from frame -1 to frame 0. For kitti results, it need to caculate the pose transformation from start point to frame 0. So why does it need to caculate the transformation 0 -> -1 ?

Wish to get your answer!
Best.

regularization loss

Could you give more details about regularization loss in implementation?