twjiannuo/edgedepth-release

Github Repo for Paper "The Edge of Depth: Explicit Constraints between Segmentation and Depth"

edgedepth-release's Issues

Where is the location of Finetuning Loss code?

Thanks for your perfect work! But in the paper, you finetune your model with 3 more epochs. But I don't find the fine-tuning progress in your code! Could you please explain to me?

Share of the Segmentation Results

@TWJianNuo Thank you a lot for sharing the code of this brilliant work!

We are following your work to conduct semantic augmented depth estimation. Due to the absent of semantic label of kitti dataset, could you please share the semantic segmentation results used in your work? Thanks! :D

Kernel building error (and fix)

I could successfully compile the torch tutorial on how to make your own cuda kernel available in pytorch. However, I couldn't compile the kernel in this repo with errors like this:

python3.8/site-packages/torch/include/c10/util/Exception.h:625:3: error: expected ‘;’ before ‘do’
EdgeDepth-Release/bnmorph/bnmorph_getcorpts.cpp:50:5: error: expected ‘;’ before ‘std’

I guess the error was fairly self-explanatory, that the compiler thinks a ; is probably missing. Without too much understanding of what's really going on (and why it worked for the authors without the ;), my fix was to add a ; to the end of this line:

#define CHECK_INPUT(x) CHECK_CUDA(x); CHECK_CONTIGUOUS(x); //added the last `;`

Now the kernel compiles for me without any error and I was able to import it in Python.

I was using ninja and my compiler version was:

yuheng@mypc:~/projects/EdgeDepth-Release$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Mar_21_19:15:46_PDT_2021
Cuda compilation tools, release 11.3, V11.3.58
Build cuda_11.3.r11.3/compiler.29745058_0
yuheng@mypc:~/projects/EdgeDepth-Release$ c++ -v
Using built-in specs.
COLLECT_GCC=c++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.4.0-1ubuntu1~20.04.1' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-Av3uEd/gcc-9-9.4.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)

Any plans of extension to monocular images?

Hi,

Nice improvement on top of monodepth2. Any plans of extending the work to monocular images?

这是官方源码吗

Invalid Precomputed Semantics Label Link

kitti_predSemantics.zip cannot be downloaded by google drive.

training code

Thx for the good job, but when are you planning to release the training code? Thanks!

Question about the Depth Morph Step.

Hi,

Thank again for your brilliant work. I am a little confused when I read the Depth Morph part of the paper. In the implementation, what I understand is that you find the pixels $x$ around depth boundaries and then compute their corresponding pixel positions $x^{}$ according to Eq.(7), then the depth of $x$ are replaced by the depth of $x^{}$ to complete the morph.

I wonder whether I was right in understanding the idea of your paper. Thanks!

not all images are labeled in folder "2011_09_28"

Hi, first thanks for your great work. But I notice that in the semantic labels you supplied, only two subfolders are labeled. So during training, you guys are only use this incomplete dataset?

Speed of morphing operation

What is the runtime overhead introduced by morphing? Using cuda makes it run faster, so what's the speed in the actual implementation?

How's the pre-trained semantic segmentation model trained on, Cityscape or KITTI?

Hi,

Thanks for your solid work and the datasets!

As you provided the pre-computed semantic labels for KITTI using method of Zhu etal , I wonder if the semantic segmentation model you use is trianed on KITTI or Cityscape? Or a mixture of KITTI and Cityscape? I notice there is no pretrained models and training code for KITTI dataset on Zhu's github page. Thanks a lot! :D

Where is the location of Finetuning Loss code?

文章里面提到在正常训练20个Epoch后，又用Finetuning Loss训练了3个epoch，但我并没有在代码中看到相关实现，请问这部分代码位置在哪？谢谢！

Choice of F.grid_sample align_corners flag due to different pytorch version

Hi,

Thank you for the helpful code! I have a question regarding F.grid_sample used in your code. When I run it, there is a warning: "UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.". Since the pytorch version is not specified in the README, may I ask what is the environment of your experiment? I am now running it with pytorch 1.7.0, where the default align_corners=False. Is it the same as your case? Thank you in advance!