fidler-lab/defgrid-release

Official PyTorch implementation of Deformable Grid (ECCV 2020)

defgrid-release's Issues

CUDA Problem

Hi, When I try to train the model on v100 16 GB GPU, I confront with CUDA problem. To solve this issue, I just change the --resolution as 256 512. I can run the model with this --resolution successfully. On other hand, the loss is

As you can see, values are fluctuating. How can I solve this issue? Which GPUs are you used when you are doing training?

PreTrained Weights

Can you please provide the final weights file for Cityscapes leftImg8bit_trainvaltest dataset.

Trained Model Parameters

Hello,

Thank you for your great work and code.
Could you please share your trained model parameters that you trained on the BSD500 dataset for superpixels?

Thank you

ImportError: No module named 'line_variance_parallel'

Detected CUDA files, patching ldflags
Emitting ninja build file /tmp/torch_extensions/line_variance_parallel/build.ninja...
Building extension module line_variance_parallel...
1.10.0.git.kitware.jobserver-1
Loading extension module line_variance_parallel...
Traceback (most recent call last):
  File "scripts/train/train_def_grid_full.py", line 21, in <module>
    from Models.deformable_grid import DeformableGrid
  File "/apdcephfs/private_nickccnie/wrpaper/Code/defgrid/Models/deformable_grid.py", line 9, in <module>
    from layers.DefGrid.diff_variance import LatticeVariance
  File "/apdcephfs/private_nickccnie/wrpaper/Code/defgrid/layers/DefGrid/diff_variance.py", line 10, in <module>
    from layers.DefGrid.variance_function_atom.line_distance_func_parallel.utils import variance_f_in_one_atom_parallel as line_variance_parallel
  File "/apdcephfs/private_nickccnie/wrpaper/Code/defgrid/layers/DefGrid/variance_function_atom/line_distance_func_parallel/utils.py", line 13, in <module>
    verbose=True)
  File "/usr/local/lib64/python3.6/site-packages/torch/utils/cpp_extension.py", line 680, in load
    is_python_module)
  File "/usr/local/lib64/python3.6/site-packages/torch/utils/cpp_extension.py", line 877, in _jit_compile
    return _import_module_from_library(name, build_directory, is_python_module)
  File "/usr/local/lib64/python3.6/site-packages/torch/utils/cpp_extension.py", line 1084, in _import_module_from_library
    file, path, description = imp.find_module(module_name, [path])
  File "/usr/lib64/python3.6/imp.py", line 297, in find_module
    raise ImportError(_ERR_MSG.format(name), name=name)
ImportError: No module named 'line_variance_parallel'

Learnable Downsampling : Downsampling to 1/4 size

Which feature map is grid-pooled when being downsampled to 1/4 of the original image size? As the shallow encoder's output features are the ones given to grid-pooling function; and those features are already of 1/4 size of the original image. Then grid-pooling them to 1/4 size wouldn't make sense, unlike when using them for downsampling to 1/8 and 1/16 size. Is a different feature map other than shallow encoder's output is used in the case of 1/4?

Help about interpolated sum function.

Hello,

I am trying to figure out how did you pass extracted features to grid points. I think interpolated_sum function is doing this but I have really hard time to understand your function. Could you provide some explanation about it or maybe a reference link to guide me.

Thank you

pytorch cpp extensions

Hello, I got an error from running the training script that says:

Using /tmp/torch_extensions as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file /tmp/torch_extensions/mean_feature/build.ninja...
Building extension module mean_feature...
Traceback (most recent call last):
File "scripts/train/train_def_grid_full.py", line 21, in
from Models.deformable_grid import DeformableGrid
File "/home/selenaling/Desktop/defgrid-release/Models/deformable_grid.py", line 9, in
from layers.DefGrid.diff_variance import LatticeVariance
File "/home/selenaling/Desktop/defgrid-release/layers/DefGrid/diff_variance.py", line 8, in
from layers.DefGrid.mean_feature.mean_feature import get_grid_mean_feature
File "/home/selenaling/Desktop/defgrid-release/layers/DefGrid/mean_feature/mean_feature.py", line 11, in
verbose=True)
File "/home/selenaling/anaconda3/envs/defgrid27/lib/python2.7/site-packages/torch/utils/cpp_extension.py", line 658, in load
is_python_module)
File "/home/selenaling/anaconda3/envs/defgrid27/lib/python2.7/site-packages/torch/utils/cpp_extension.py", line 827, in _jit_compile
with_cuda=with_cuda)
File "/home/selenaling/anaconda3/envs/defgrid27/lib/python2.7/site-packages/torch/utils/cpp_extension.py", line 880, in _write_ninja_file_and_build
_build_extension_module(name, build_directory, verbose)
File "/home/selenaling/anaconda3/envs/defgrid27/lib/python2.7/site-packages/torch/utils/cpp_extension.py", line 972, in _build_extension_module
message += ": {}".format(error.output.decode())
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3915: ordinal not in range(128)

Am wondering if this is encountered before and how can I solve it? I am running with python2.7. Thank you in advance!

training iteration doesn't actually update the grid base point positions

Hello, I set up the gridpool.py to train. While looking into the codes, it seems every forward iteration of DGNet doesn't actually update the initial grid base point positions, i.e. the predicted offset is always with respect to the initial base point positions, which is a little confusing to me. I'm wondering if this is a bug or if you could point me to the part that updates the grid positions every iteration? Thank you in advance!

cannot find gtFine images

Hello, there is another issue that the training script says cannot find '/media/selenaling/Elements/cityscapes/leftImg8bit_trainvaltest/gtFine/train/strasbourg/strasbourg_000001_014258_gtFine_labelIds.png', and it isn't straightforward to me how to generate this ground truth given the json files. Am wondering if there are instructions on this part? Thank you in advance.