harsh-99/scl

Implementation of "SCL: Towards Accurate Domain Adaptive Object Detection via Gradient Detach Based Stacked Complementary Losses"

scl's Issues

Run trainval_net_SCL.py with custom dataset raises RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0.

Traceback (most recent call last):
File "trainval_net_SCL.py", line 183, in
data_s = next(data_iter_s)
File "/home/skl/anaconda3/envs/python27/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 286, in next
return self._process_next_batch(batch)
File "/home/skl/anaconda3/envs/python27/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 307, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
RuntimeError: Traceback (most recent call last):
File "/home/skl/anaconda3/envs/python27/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 57, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/home/skl/anaconda3/envs/python27/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 138, in default_collate
return [default_collate(samples) for samples in transposed]
File "/home/skl/anaconda3/envs/python27/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 115, in default_collate
return torch.stack(batch, 0, out=out)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 426 and 384 in dimension 2 at /pytorch/aten/src/TH/generic/THTensorMath.c:3586

However, with the same setting, Strong-Weak DA F-RCNN can be trained.

About the trained model.

Hi, when I use your trained model for clipart. The model can not find these parameters:

'RCNN_base2.0.2.bn2.num_batches_tracked', 'RCNN_base3.0.5.bn2.num_batches_tracked', 'RCNN_base1.4.0.bn3.num_batches_tracked', 'RCNN_base3.0.6.bn1.num_batches_tracked', 'RCNN_base3.0.18.bn2.num_batches_tracked', 'RCNN_base3.0.4.bn1.num_batches_tracked', 'RCNN_base3.0.12.bn3.num_batches_tracked', 'RCNN_base3.0.16.bn3.num_batches_tracked', 'RCNN_base3.0.9.bn2.num_batches_tracked', 'RCNN_base1.4.0.downsample.1.num_batches_tracked', 'RCNN_base2.0.0.bn3.num_batches_tracked', 'RCNN_base1.4.2.bn3.num_batches_tracked', 'RCNN_base3.0.13.bn1.num_batches_tracked', 'RCNN_base3.0.20.bn2.num_batches_tracked', 'RCNN_base3.0.4.bn2.num_batches_tracked', 'RCNN_base1.4.0.bn1.num_batches_tracked', 'RCNN_base3.0.13.bn2.num_batches_tracked', 'RCNN_top.0.0.downsample.1.num_batches_tracked', 'RCNN_base3.0.12.bn1.num_batches_tracked', 'RCNN_base3.0.20.bn3.num_batches_tracked', 'RCNN_base3.0.3.bn3.num_batches_tracked', 'RCNN_base3.0.1.bn2.num_batches_tracked', 'RCNN_base3.0.5.bn1.num_batches_tracked', 'RCNN_base1.1.num_batches_tracked', 'netD3.bn1.num_batches_tracked', 'RCNN_base3.0.16.bn2.num_batches_tracked', 'RCNN_base3.0.15.bn2.num_batches_tracked', 'RCNN_base3.0.21.bn1.num_batches_tracked', 'RCNN_base3.0.18.bn3.num_batches_tracked', 'RCNN_base3.0.8.bn1.num_batches_tracked', 'RCNN_base2.0.0.downsample.1.num_batches_tracked', 'netD_inst.bn2.num_batches_tracked', 'RCNN_base3.0.21.bn2.num_batches_tracked', 'RCNN_top.0.1.bn1.num_batches_tracked', 'RCNN_base3.0.21.bn3.num_batches_tracked', 'RCNN_base3.0.1.bn1.num_batches_tracked', 'RCNN_base3.0.0.bn2.num_batches_tracked', 'RCNN_base2.0.3.bn2.num_batches_tracked', 'RCNN_base3.0.1.bn3.num_batches_tracked', 'RCNN_base2.0.0.bn1.num_batches_tracked', 'RCNN_base3.0.14.bn2.num_batches_tracked', 'RCNN_base3.0.17.bn1.num_batches_tracked', 'RCNN_base3.0.20.bn1.num_batches_tracked', 'RCNN_base3.0.2.bn3.num_batches_tracked', 'RCNN_base2.0.2.bn1.num_batches_tracked', 'RCNN_base3.0.7.bn1.num_batches_tracked', 'RCNN_base3.0.22.bn1.num_batches_tracked', 'RCNN_base2.0.2.bn3.num_batches_tracked', 'RCNN_base3.0.11.bn3.num_batches_tracked', 'RCNN_top.0.2.bn1.num_batches_tracked', 'RCNN_base1.4.2.bn1.num_batches_tracked', 'RCNN_top.0.1.bn3.num_batches_tracked', 'RCNN_base3.0.19.bn2.num_batches_tracked', 'RCNN_base2.0.1.bn1.num_batches_tracked', 'RCNN_base3.0.7.bn2.num_batches_tracked', 'RCNN_top.0.2.bn3.num_batches_tracked', 'RCNN_top.0.1.bn2.num_batches_tracked', 'RCNN_base1.4.0.bn2.num_batches_tracked', 'RCNN_base1.4.1.bn3.num_batches_tracked', 'RCNN_base3.0.10.bn1.num_batches_tracked', 'RCNN_base3.0.19.bn3.num_batches_tracked', 'RCNN_base3.0.17.bn3.num_batches_tracked', 'RCNN_base3.0.11.bn2.num_batches_tracked', 'RCNN_base3.0.19.bn1.num_batches_tracked', 'RCNN_base3.0.8.bn3.num_batches_tracked', 'RCNN_base3.0.14.bn1.num_batches_tracked', 'RCNN_base3.0.12.bn2.num_batches_tracked', 'RCNN_base1.4.2.bn2.num_batches_tracked', 'RCNN_base3.0.11.bn1.num_batches_tracked', 'RCNN_base1.4.1.bn2.num_batches_tracked', 'RCNN_base2.0.0.bn2.num_batches_tracked', 'RCNN_base3.0.6.bn2.num_batches_tracked', 'RCNN_base3.0.10.bn2.num_batches_tracked', 'RCNN_base3.0.6.bn3.num_batches_tracked', 'RCNN_base2.0.3.bn1.num_batches_tracked', 'RCNN_base3.0.9.bn3.num_batches_tracked', 'netD2.bn1.num_batches_tracked', 'RCNN_base3.0.2.bn1.num_batches_tracked', 'RCNN_base2.0.3.bn3.num_batches_tracked', 'RCNN_base3.0.0.downsample.1.num_batches_tracked', 'RCNN_base3.0.22.bn3.num_batches_tracked', 'RCNN_top.0.0.bn1.num_batches_tracked', 'RCNN_base3.0.4.bn3.num_batches_tracked', 'RCNN_top.0.0.bn2.num_batches_tracked', 'RCNN_base1.4.1.bn1.num_batches_tracked', 'RCNN_base3.0.10.bn3.num_batches_tracked', 'RCNN_base3.0.3.bn2.num_batches_tracked', 'netD2.bn2.num_batches_tracked', 'RCNN_base3.0.5.bn3.num_batches_tracked', 'RCNN_base3.0.3.bn1.num_batches_tracked', 'RCNN_base3.0.7.bn3.num_batches_tracked', 'RCNN_base3.0.22.bn2.num_batches_tracked', 'RCNN_base3.0.18.bn1.num_batches_tracked', 'netD3.bn3.num_batches_tracked', 'RCNN_base3.0.17.bn2.num_batches_tracked', 'RCNN_base3.0.2.bn2.num_batches_tracked', 'RCNN_base3.0.15.bn3.num_batches_tracked', 'RCNN_base3.0.0.bn1.num_batches_tracked', 'RCNN_base3.0.8.bn2.num_batches_tracked', 'RCNN_base3.0.13.bn3.num_batches_tracked', 'RCNN_base2.0.1.bn2.num_batches_tracked', 'netD2.bn3.num_batches_tracked', 'RCNN_top.0.0.bn3.num_batches_tracked', 'RCNN_base3.0.0.bn3.num_batches_tracked', 'RCNN_top.0.2.bn2.num_batches_tracked', 'RCNN_base3.0.14.bn3.num_batches_tracked', 'netD3.bn2.num_batches_tracked', 'RCNN_base2.0.1.bn3.num_batches_tracked', 'RCNN_base3.0.16.bn1.num_batches_tracked', 'RCNN_base3.0.15.bn1.num_batches_tracked', 'RCNN_base3.0.9.bn1.num_batches_tracked'

When I ignore this error and test, the mAP is only 40.4. How to get the correct mAP?

AP for aeroplane = 0.3302
AP for bicycle = 0.4909
AP for bird = 0.3590
AP for boat = 0.2591
AP for bottle = 0.3835
AP for bus = 0.5573
AP for car = 0.3868
AP for cat = 0.1590
AP for chair = 0.3883
AP for cow = 0.5839
AP for diningtable = 0.1884
AP for dog = 0.2369
AP for horse = 0.3690
AP for motorbike = 0.6995
AP for person = 0.6064
AP for pottedplant = 0.4975
AP for sheep = 0.2572
AP for sofa = 0.3483
AP for train = 0.4715
AP for tvmonitor = 0.5138
Mean AP = 0.4043

How many epochs should I train?The default is 20. Is that too many?

Can this method be used as unsupervised object detection, if yes then how to do object detection on unlabelled target domain data

Hi,

How can we use this method as unsupervised domain adaptation with object detection and run object detection on unlabelled target domain data like clipart

Applying SCL on SSD

Hi,
Has anyone try applying this method or Strong-Weak on SSD or any other Single Shot Detectors?

Can I run this Solution in real time or without the need to train the target data

Hi,

I want to train this solution in real-time by capturing images and run algorithm on top of it, but also i don't want to train target data, can this be done?

Thanks,
Nilesh

visualization of features from PASCAL to Clipart

I have question about visualizing data after domain adaptation training. how you visualized data using tsne after training? could you provide idea how can I visualize source and target data after training?

Experimental details

When performing the task K2C, only the car class is used when training the KITTI dataset? Or do all classes train and only show the AP of the car class in the final result?

Using multilple GPUs to accomplish distributed training

Hello! Thanks for sharing the excellent work !!!

When I run the codes using multiple GPUs with nn.parallel.DistributedDataParallel, it'll always raise an error as follows :
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [2]] is at version 3; expected version 2 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

However, when I run using a single GPU, no errors raise, I am confused...

Heat Map

How to generate an attention map，thanks

Import Error

`ImportError: 'No module named 'lib.model.utils.cython_bbox''

I only find this file: cython_bbox.cpython-36m-x86_64-linux-gnu.so
How can I solve it?

after filtering, there are 0 images

Hi,
I am trying to train my own dataset. I have already prepared my dataset as you described. Annotations, ImageSets etc. are also prepared according to VOC2007. But filter_roidb filters all images.

File "/home/lty/anaconda3/envs/SWDA/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 301, in forward self.padding, self.dilation, self.groups) RuntimeError: cublas runtime error : the GPU program failed to execute at /opt/conda/conda-bld/pytorch_1525909934016/work/aten/src/THC/THCBlas.cu:249

Your implementation of domain adaptive faster rcnn performs better than paper values, what might be the reason?

Hi Zhiqiang,

I tried to run your re-implementation of domain adaptive faster rcnn in adaptation from pascal voc to clipart, and I found it reached much better mAP (30 mAP) than the value reported in paper "stong weak distribution alignment" paper. (19.8 mAP)

May I ask what's your opinion on this? Why is your re-implementation much better? Or did I probably ignore some details in training script and misuse them? (I used your trainval_net_dfrcnn.py, and didn't use any rendered datasets).

Thanks a lot for any possible help in advance!

Best,
Anton

How do I use cityscape and foggy cityscapes datasets?

Hello, author. How do I use cityscape and foggy cityscapes datasets? Da fasterrcnn generates the JSON format file of coco type. Could you talk about how to generate the dataset format used by the code? Thank you very much.

preparing clipart dataset problem and multigpu training

After training started, I faced those errors but I have no idea why I got those errors. could you explain?
I tried to train network with multi-GPU by using CUDA_VISIBLE_DEVICES=0,1 but it is just using GPU 0, why?

about INIT dataset

Hi,

Could you please explain how did you utilize the INIT dataset?
It seems it has 2 batches of images. How did you split the train and test set?

Thanks.

Your implementation of da-faster-rcnn seems very different from the original paper, right?

About cityscape-->foggy cityscape test set

About cityscape and foggy dataset

Hi,

I run the dataset preparation code in DA-Faster RCNN, but it can only generate trainval.txt. I just want to know how to create the train.txt?

Besides, the dataset preparation code only generate a folder contains both cityscape and foggy images. However, it seems that I need two different folder path from the lib/datasets/config_dataset.py. Could you explain how to generate this two folders?

dataset of clipart

Hello, when I try to run the experiment of VOC to Clipart, the result can not match as the paper reported by strong-weakly alignment, so I want to know the result is based on all the test data? And the hyperparameter(such as learning rate) of this experiment is set as?

ImportError: No module named cython_bbox

Hello, when I train with trainval_net_global_local.py, I meet the problem of Import error, What should I do?

from roi_data_layer.roidb import combined_roidb
...
from model.utils.cython_bbox import bbox_overlaps
ImportError: No module named cython_bbox

Looking forward to your reply, thank you very much!

how to train this model

I have a question when I try to train a model based on my own dataset which is in voc format. But when I begin to train, there was a strange problem. After the pretrained model is loaded， the code stays static and stops at 'Loading pretrained weights from ./data/pretrained_model/vgg16_caffe.pth
'. Without any error,I wonder whether it cannot be trained rightly. I am eager to get your help.

Thanks

did you tried with EfficientNet?

I want to try with Efficientnet. could you explain how can I use Efficientnet?

Can i generate or train model using this method and run it on different dataset/images that is similar to target domain data but this new dataset don't exist in target domain data when it was trained

Hi,

I hope you understood my question, can we use the model that was generated from training source domain and target domain by your method and test the model by running on completely new dataset/images that is kind of similar to target domain data.
If yes, can you please help me with logic and explain how can we do that?

Thanks in advance.

About sim10k dataset

Hi, when I download the sim10k from the website, I can only get the trainval.txt. How do you split it into train.txt and val.txt?

How can I add custom new class labels, lets say-x classes to a SCL trained model( which is already trained on y classes). So I do not replace those y classes and in total I have x+y class labels.

I need to train the old objects +new objects again?or is there a way to train for only new labels.

How to do Object detection on target domain(ex: Clipart) that don't have annotation.xml label

hey harsh-99,
I want to do object detection on clipart(target domain) dataset and i am training my model using pascal dataset as (source domain) and clipart dataset as (target domain) both have annotation.xml, but what if clipart dataset(target domain) don't have annotation.xml, how to do object detection on clipart sample that don't have annotation.xml, i tried to remove/comment annotation.xml code but i am getting error, how to solve or approach to this issue,Thanks in advance.

question of t-sne

Dear authors:
could you provide the code of t-sne between cityscapes and foggy cityscapes. Thanks a lot!

what is the difference between strong-weakly detach method with your proposed Gradients Detach Updating?

Hello~I have read the source code for "SCL: Towards Accurate Domain Adaptive Object Detection via
Gradient Detach Based Stacked Complementary Losses" and "Strong-Weak Distribution Alignment for Adaptive Object Detection" and found that the implementation of the latter already used the detach method and there doesn't seem to be much of a difference in the way it is implemented in your SCL. Can you explain the specific differences?