tri-ml/kp3d

Code for "Self-Supervised 3D Keypoint Learning for Ego-motion Estimation"

kp3d's Issues

Train using our own data

Hi,

Thank you for this amazing paper!
I was wondering if you could provide the code to train the models.
I believe that the best way to prove the real performance of your approach is by letting people train it by themselves.

Thank you,

Code coming soon?

How soon?

Need some advice for training keypoint detector with camera pose supervision

Hi! Great work on using ego-motion to train the local feature and the depth jointly. I'm trying to train a network under the supervision of camera poses in MegaDepth. But it seems to be hard to converge. Would you please provide some advice? Maybe on WeChat? My personal email is [email protected].

Thank you!

Incorrect normalization for depth network

Hey, just wanted to let you know that the input to the depth network for the odometry evaluation is incorrectly normalized. The depth encoder already includes normalization, so the to_color_normalized function should not be applied to the inputs for the depth network, e.g. as done here
https://github.com/TRI-ML/KP3D/blob/master/kp3d/evaluation/evaluate_keypoint_odometry.py#L71-L83

also note that this function multiplies by the standard deviation instead of dividing by it. this is probably suboptimal, but I guess since it's trained that way, it shouldn't be fixed (at least not for the pretrained keypoint net).

KP3D/kp3d/utils/image.py

Lines 112 to 115 in 3eb9d25

	assert len(images.shape) == 4
	images -= 0.5
	images *= 0.225
	return images

using the proper normalization for the depth network slightly improves the results

Baseline pretrained model

Hello, I'm looking for the resnet-baseline model which is only trained with coco dataset and got high performance in HPatches homography estimation (not uploaded model which is trained also with kitti.)

Can you upload the model above mentioned you have trained before?

Thank you in advance.

F2F

Hello,
In your paper, you mention F2F in the qualitative comparison graphs.
What paper/model does it refer to?

code release ?

Hi,

when is the code getting published?

thank you!
Dane

When is the code published?

Hello, very interesting project! Is there a planned date for the code to be published?

How to obtain the trajectory image

How to obtain the trajectory image，thanks！

How to visualize the depth map obtained ?

How to backproject 3D points for rendering in pangolin ?

How to apply on DSO

Hello
I'm trying to check the keypoint to the SLAM to evaluate visual odometry such as DSO as your paper tried.

But, there is c++ implemented code on https://github.com/JakobEngel/dso but, i think it is very hard to apply the KP3D
because DSO is implemented on c++, and it is very complicated.

Was there any idea or specific modification on code to apply it more easily when you did experiment?

Thanks in advance :)

Will training code be released?

Hi,
Thanks for sharing the code of this very impressive work? I am wondering whether the training code will be released. We hope to reproduce the training procedure. We would really appreciate your help!

Best,
CH