p-ranav/merged_depth

Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

merged_depth runs (1) AdaBins, (2) DiverseDepth, (3) MiDaS, (4) SGDepth, and (5) Monodepth2, and calculates a weighted-average per-pixel absolute depth estimation.

Quick Start

First, download the pretrained models using the download_models script.

Next, run the infer script - this will run on all images in test/input and save the results to test/output.

python3 -m pip install -r requirements.txt
python3 -m merged_depth.utils.download_models
python3 -m merged_depth.infer

If you're using Anaconda3, the following has been tested to work (in Windows):

conda create --name merged_depth python=3.6
conda activate merged_depth
conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge
python3 -m pip install -r requirements.txt
python3 -m merged_depth.utils.download_models
python3 -m merged_depth.infer

The results include (1) a _depth.npy file that you can load (see load_and_display_depth.py), (2) a _stacked.png file that shows the original and colorized depth images.

To run the predictor on a single input, use infer_single.py

python3 -m merged_depth.infer_single ~/foo/bar/test.png

Sample Output

The output depth is absolute depth in meters. The colorizer range is [0, 20].

merged_depth's Issues

Issue with installation - bad zip file

Error received:

Traceback (most recent call last):
  File "/Users/ashok/opt/anaconda3/envs/wsn/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/ashok/opt/anaconda3/envs/wsn/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/ashok/Desktop/wsnr/merged_depth/merged_depth/utils/download_models.py", line 65, in <module>
    main()
  File "/Users/ashok/Desktop/wsnr/merged_depth/merged_depth/utils/download_models.py", line 62, in main
    extract_models(pretrained_models)
  File "/Users/ashok/Desktop/wsnr/merged_depth/merged_depth/utils/download_models.py", line 56, in extract_models
    with zipfile.ZipFile(file, 'r') as zip_ref:
  File "/Users/ashok/opt/anaconda3/envs/wsn/lib/python3.9/zipfile.py", line 1266, in __init__
    self._RealGetContents()
  File "/Users/ashok/opt/anaconda3/envs/wsn/lib/python3.9/zipfile.py", line 1333, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

Relative depth to absolute

In one of your response, related relative to absolute depth conversion, you mentioned : " I'm using some linear parameters to scale these to absolute depth". Could you please point to the particular code block where this transformation implemented.
Thank you.

Output grayscale depth map only

How to output non-stacked grayscale depth maps only?

SGDepth Predictions Inverted

It seems to me that the predicted values from SGDepth are inverted compared to other predictions.

(Also thank you for having this repository public! Really nice for comparing different depth networks!)

RAM Issue

When i run it (file version) with 19 images, RAM explotes (16 GB), how are you testing it?

GPU utilization

Running on GTX 1060 6Gb. GPU utilization ` 0 -1%. Processed by CPU only that terribly slow.
"Standalone" Midas3 and monodepth2 utilize GPU just perfectly.
Solved by downgrading to torch from 1.9 to 1.8. This issue could be closed

clarification about output

Hii @p-ranav
thank so much for sharing your amazing work
i wanted to know what output image represents, is it representing absolute depth or anything proportional to absolute depth?
what are the values in average_depth in infer.py represent?
if i want to get the absolute depth how can i get that

Thank you

Error downloading files.

merged_depth >python utils\download_models.py
Traceback (most recent call last):
  File "utils\download_models.py", line 65, in <module>
    main()
  File "utils\download_models.py", line 62, in main
    extract_models(pretrained_models)
  File "utils\download_models.py", line 56, in extract_models
    with zipfile.ZipFile(file, 'r') as zip_ref:
  File "c:\g\vr\lw\python\lib\zipfile.py", line 1258, in __init__
    self._RealGetContents()
  File "c:\g\vr\lw\python\lib\zipfile.py", line 1325, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

Pretrained weights issue

Hi, i am trying to use your code but it is impossible to download the pretrained weighs via download script. I guess the GDrive url is wrong.

Upgrade to Midas V.3

It looks like you use Midas v.2.1. Any chance to upgrade to Midas v.3.0 with dpt_large-midas-2f21e586.pt model ?
I replaced the "core" Midas to Midas V.3 in the ...\merged_depth\nets and everything works just fine.
However attempt to change the model self.midas_model_path = "./pretrained/MiDaS_f6b98070.pt" to
self.midas_model_path = "./pretrained/dpt_large-midas-2f21e586.pt" throws multiple errors upon execution.
My great appreciation for your suggestion.

Can't download models

When I try python3 -m merged_depth.utils.download_models, following errors occurred

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/me/merged_depth/merged_depth/utils/download_models.py", line 65, in <module>
    main()
  File "/home/me/merged_depth/merged_depth/utils/download_models.py", line 62, in main
    extract_models(pretrained_models)
  File "/home/me/merged_depth/merged_depth/utils/download_models.py", line 56, in extract_models
    with zipfile.ZipFile(file, 'r') as zip_ref:
  File "/usr/lib/python3.8/zipfile.py", line 1269, in __init__
    self._RealGetContents()
  File "/usr/lib/python3.8/zipfile.py", line 1336, in _RealGetContents
    raise BadZipFile("File is not a zip file")
zipfile.BadZipFile: File is not a zip file

The content of pretrained/pretrained.zip is

<HTML>
<HEAD>
<TITLE>Not Found</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>Not Found</H1>
<H2>Error 404</H2>
</BODY>
</HTML>

How can I get the model?
Thanks!

Thank you!

Success in setup and run (on Windows). Thank you for creating this package.