adamkortylewski/compositionalnets

Official implementation of CVPR2020 paper: "Compositional Convolutional Neural Networks: A Deep Architecture with Innate Robustness to Partial Occlusion" https://arxiv.org/abs/2003.04490

compositionalnets's Issues

question about the mix models

Hello, thanks for your work.
When we download your models,we found that each category has a corresponding model,
such as"mmodel_aeroplane_K4_FEATDIM512_pool4_specific_view.pickle" but we didn't find the method to generate the model in the code
We want to know how to generate such model so that we can add new categories.

Getting error when run python Code/demo.py

I got error message "ModuleNotFoundError: No module named 'Code'" when I tried to run demo.py in folder Code. How can I fix this problem?

About the code in model.py

Thanks for your excellent work.

CompositionalNets/Code/model.py

Line 146 in 9affa4d

mean_vec[v]/clust_cnt[v].cuda(gpu_id)).t()

Why is there no normalization in the calculation of 'general'?

CompositionalNets/Code/model.py

Line 122 in 9affa4d

clutter_feats/mean_activation, dim=0)

But the former 'noise' and 'white' are normalized.

Empty clutter_feats in demo.py

I'm trying to experiment with the code by running python Code/demo.py but get the following error:

Traceback (most recent call last):
  File "Code/demo.py", line 34, in <module>
    net = Net(extractor, weights, vMF_kappa, occ_likely, mix_models, bool_mixture_bg=bool_mixture_model_bg,compnet_type=compnet_type, num_mixtures=num_mixtures, vc_thresholds=cfg.MODEL.VC_THRESHOLD)
  File "/content/CompositionalNets/Code/model.py", line 48, in __init__
    compnet_type, vMF_kappa)
  File "/content/CompositionalNets/Code/model.py", line 129, in get_clutter_model
    vMF_kappa, max_it=150, tol=1e-10)
  File "/content/CompositionalNets/Code/vMFMM.py", line 45, in fit
    centers_i.append(np.random.choice(rdn_index))
  File "mtrand.pyx", line 1125, in mtrand.RandomState.choice
ValueError: 'a' cannot be empty unless no samples are taken

This happens when trying to fit the 'vmf' clutter model at line 129 in get_clutter_model by passing an empty list of clutter_feats. Any chance I could get help with this?

My environment:
OS: Ubuntu 18.04.3 LTS
Release: 18.04
Python: 3.6.9
Cuda: Release 10.1, V10.1.243
PyTorch: 1.2.0

Code for detection?

Hi Adam,

Your latest paper in IJCV uses compositional voting for generalizing to object detection also. I am not able to find the template code in this repository for the detection layer.

Can you share that?

Error when training from scratch

The model runs into following error if training from scratch and not running train.py:

Total imgs for test of occ_level ZERO and occ_type  4341
Testing
  0%|                                                                                         | 0/4341 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/CompositionalNets/Code/test.py", line 138, in <module>
    acc,scores = test(models=nets, test_data=test_imgset, batch_size=1)
  File "/home/CompositionalNets/Code/test.py", line 40, in test
    output, *_ = models[0](input)
  File "/home/toshi/base/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/CompositionalNets/Code/model.py", line 62, in forward
    mix_likeli = self.pointwiseinference(vmf_activations)
  File "/home/toshi/base/lib/python3.9/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/CompositionalNets/Code/model.py", line 223, in forward
    foreground = foreground.reshape((-1, self.num_classes,
RuntimeError: shape '[-1, 10, 4, 1, 7, 12]' is invalid for input of size 3612

Steps to recreate -

1. Add to (top of) vMF_clustering.py, comptSimMat.py, Learn_mix_model_vMF_view.py, Code/test.py files:

import os, sys 
p = os.path.abspath('.') 
sys.path.insert(1, p) 

2. In comptSimMat.py, change (due to TypeError) -

imgset = Imgset(imgs, masks, labels, imgLoader, bool_square_images=False,bool_cutout=False,bool_pytorch=bool_pytorch) 

To -

imgset = Imgset(imgs, masks, labels, imgLoader, bool_square_images=False) 

3. mv models/init_vgg/dictionary_vgg/dictionary_pool5_512.pickle models/init_vgg/dictionary_vgg/dictionary_pool5.pickle

where is the dataset? How can I download the dataset?

data/pascal3d+_occ/ , I want to get the pascal3d+_occ dataset. Can you help me?

pretrained_model

Sorry for bother you again!
After you give the initialization code, we try to train with our own data. When we run the demo.py with our own initialization model, we find this question:
size mismatch for mix_model: copying a param with shape torch.Size([48, 512, 41, 91]) from checkpoint, the shape in current model is torch.Size([46, 512, 23, 31]).
size mismatch for pointwiseinference.mix_model: copying a param with shape torch.Size([48, 512, 41, 91]) from checkpoint, the shape in current model is torch.Size([46, 512, 23, 31]).
terminate called without an active exception
We guess the pretrained_model best.pth we download from your github may not mathcing with our model . Can you provide the code to generate this pretrained_model.
Thank you very much!

No such file or directory: 'models/init_vgg/dictionary_vgg/dictionary_pool5.pickle'

torch version

There is a problem with the old version of torch. When I try to install the requirements.txt I get the following error:

Looking in links: https://download.pytorch.org/whl/torch_stable.html
ERROR: Could not find a version that satisfies the requirement torch===1.2.0 (from versions: none)
ERROR: No matching distribution found for torch===1.2.0

Which version do you recommend using?

About getImg

i don't know if there's a little problem in geImg(helpers.py) function.

if dataset == 'pascal3d+': if occ_level == 'ZERO': filelist = data_path + 'pascal3d+_occ/' + category + '_imagenet_train' + '.txt' img_dir = data_path + 'pascal3d+_occ/TRAINING_DATA/' + category + '_imagenet'

it seems even if i set bool_train_with_occluders = True still no occluded images will be considerd in training process because occ_level = 'one' 'five' or 'nine' are not considered.