chengchunhsu/everypixelmatters

Implementation of ECCV 2020 paper "Every Pixel Matters: Center-aware Feature Alignment for Domain Adaptive Object Detector"

everypixelmatters's Issues

Unable to download weight

I cannot download the weight. Could you please provide the weight? Thank you!

reproducibility and domain classifier loss

Hello,
Thanks for providing the code. I was able to run the code for Sim10k --> Cityscapes and got scores lower than reported. mAP 24.8, mAP @0.5 42.6, mAP @0.75 24.4 , mAP @s 5.3 , mAP @m 27.3 , mAP@L 51.1 . One thing that I noticed is that the domain classifier loss goes down to zero for all the pyramid scales after few iterations. Could you please let me know what kind of trend you see for the domain classifier losses, which could possibly be the reason for me seeing the lower scores?

Thanks

AttributeError: 'list' object has no attribute 'resize'

Traceback (most recent call last):
File "C:/Users/84957/Desktop/qin/every/EveryPixelMatters-master/tools/train_net_da.py", line 480, in
main()
File "C:/Users/84957/Desktop/qin/every/EveryPixelMatters-master/tools/train_net_da.py", line 469, in main
model = train(cfg, args.local_rank, args.distributed)
File "C:/Users/84957/Desktop/qin/every/EveryPixelMatters-master/tools/train_net_da.py", line 361, in train
arguments,
File "C:\Users\84957\Desktop\qin\every\EveryPixelMatters-master\fcos_core\engine\trainer.py", line 128, in do_train
in enumerate(zip(data_loader_source, data_loader_target), start_iter):
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data\dataloader.py", line 363, in next
data = self._next_data()
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data\dataloader.py", line 989, in _next_data
return self._process_data(data)
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data\dataloader.py", line 1014, in _process_data
data.reraise()
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch_utils.py", line 395, in reraise
raise self.exc_type(msg)
AttributeError: Caught AttributeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data_utils\worker.py", line 185, in _worker_loop
data = fetcher.fetch(index)
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\84957\Desktop\qin\every\EveryPixelMatters-master\fcos_core\data\datasets\coco.py", line 67, in getitem
img, anno = super(COCODataset, self).getitem(idx)
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torchvision\datasets\coco.py", line 118, in getitem
img, target = self.transforms(img, target)
File "C:\Users\84957\Desktop\qin\every\EveryPixelMatters-master\fcos_core\data\transforms\transforms.py", line 15, in call
image, target = t(image, target)
File "C:\Users\84957\Desktop\qin\every\EveryPixelMatters-master\fcos_core\data\transforms\transforms.py", line 60, in call
target = target.resize(image.size)
AttributeError: 'list' object has no attribute 'resize'

I trained according to the data set format you provided, but this error occurred. Does the annotation file need to be converted? Looking forward to your reply

Model link is not work

Hi,can you add link again?

dataset for foggy cityscapes

Hi, i have received the cityscapes dataset. However, when i click the link for foggy cityscapes, i don't know which one is i really need. In the website, i see some foggy cityscapes datasets for semantic segmentation, not for object detection. Can you give me the link for foggy dataset specifically. Thanks!

can you provide the training log of cityscapes to foggy_cityscapes?

hi,
Thanks for your open code! my results have some difference from this paper. and I think that my configs may be wrong, so I want to look at your training log of C to F. Thank you again!

Does your code write detections on images? Or read the detection from pth files and plot them?

why does the 'atten_map' need another sigmoid ?

Hi, your work is great.

I wonder why does the atten_map need another sigmoid at this line?

EveryPixelMatters/fcos_core/modeling/discriminator/fcos_head_discriminator_CA.py

Line 70 in fcb5703

atten_map = (self.center_aware_weight * box_cls_map * centerness_map).sigmoid()

In my view, the box_cls_map and centerness_map are the normalized score after sigmoid already, so
0<box_cls_map<1
0<centerness_map <1
, then,
0 < box_cls_map * centerness_map < 1,
0 <= center_aware_weight * box_cls_map * centerness_map < center_aware_weight
, so,
atten_map >= 0.5 forever.

If we set center_aware_weight to [0, 1], the
center_aware_weight * box_cls_map * centerness_map is also a normalized score,

then why does the atten_map need the sigmoid again?

checkpoint link expire

Could you update the link for the checkpoints (i.e. pretrained models)?

Data augmentation?

I have a general question/feature request about data augmentation. I've been manually adding in additional data augmentation (through the Albumentations package). Applying random photometric distortions (e.g. Gaussian blur, color jitter, etc.) significantly helps with the domain generalization. However, after I started adding random geometric distortions (e.g. random affine distortions), performance on the validation target domain decreased by about 5%, rather than improving, as I've observed in supervised learning. I'm fairly certain this isn't a bug, as I've checked and tested the various pipelines many times. I was wondering, do these types of augmentations mess up the discrimator fooling process? Or, is there anywhere I can find code that adds data augmentation to the fcos training routine?

Thank you!

Confirmation of the step parameter in the config files

I notice the step parameters are set to be (120000, 160000) in most config files while the max_iter is 20,000. I am wondering should it be (12000, 16000) for the step parameters? Or the learning rate for the last iteration is better to equal to base_lr?
Thx!

Error: size mismatch when using the pretained model

I have downloaded both the sim10k and cityscapes datasets and tries to repeat the training process.

But when I use the following commond

python3 -m torch.distributed.launch --nproc_per_node=4 --master_port=$((RANDOM + 10000)) tools/train_net_da.py --config-file ./configs/da_ga_sim10k_VGG_16_FPN_4x.yaml

I have this error:

RuntimeError: Error(s) in loading state_dict for DistributedDataParallel:
size mismatch for module.fpn.fpn_inner3.weight: copying a param with shape torch.Size([256, 1024, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for module.fpn.fpn_inner4.weight: copying a param with shape torch.Size([256, 2048, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 512, 1, 1]).
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for DistributedDataParallel:
size mismatch for module.fpn.fpn_inner3.weight: copying a param with shape torch.Size([256, 1024, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for module.fpn.fpn_inner4.weight: copying a param with shape torch.Size([256, 2048, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 512, 1, 1]).

Can anyone give me any clue why this happens?

ValueError: bbox should have 2 dimensions, got 1

After loading model log contains information below. Can you please see what's the query?
It was running on actual dataset, but on custom data, it is producing this error. Thanks

loading annotations into memory...
Done (t=0.20s)
creating index...
index created!
2020-09-26 17:28:51,585 fcos_core.trainer INFO: Start training
Traceback (most recent call last):
File "tools/train_net_da.py", line 480, in
main()
File "tools/train_net_da.py", line 469, in main
model = train(cfg, args.local_rank, args.distributed)
File "tools/train_net_da.py", line 361, in train
arguments,
File "/home/akhtar/EveryPixelMatters/fcos_core/engine/trainer.py", line 128, in do_train
in enumerate(zip(data_loader_source, data_loader_target), start_iter):
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 819, in next
return self._process_data(data)
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
data.reraise()
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/_utils.py", line 385, in reraise
raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 2.
Original Traceback (most recent call last):
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/akhtar/EveryPixelMatters/fcos_core/data/datasets/sim10k.py", line 46, in getitem
target = self.get_groundtruth(index)
File "/home/akhtar/EveryPixelMatters/fcos_core/data/datasets/sim10k.py", line 65, in get_groundtruth
target = BoxList(anno["boxes"], (width, height), mode="xyxy")
File "/home/akhtar/EveryPixelMatters/fcos_core/structures/bounding_box.py", line 27, in init
"bbox should have 2 dimensions, got {}".format(bbox.ndimension())
ValueError: bbox should have 2 dimensions, got 1

I HAVE CHECKED SEVERAL TIMES THAT THERE IS NO EMPTY ANNOTATIONS

Some questions of training schedule.

Thank you for your amazing work!!!

I have reproduced almost all your results about GA and CA in your papers. Actually, it is really hard to select the best models. I have to save a checkpoint every 250 iters following your configs and test them one by one with some tricks. On ResNet101, I can get 45 AP50 on Kitti, 40 AP50 on CS, and 51.5 AP50 on Sim10k. I am not sure if you select the model by testing a lot of candidates.

However, if I use the same configurations and same model-selection strategies to train a source-only model on pure FCOS, I can easily get 44 AP50 on Kitti, 27 AP50 on CS, and about 47 on Sim10k!! The results on CS seem normal. We have tested both on your codes and pure FCOS environments, achieving similar results. This really confused me a lot.

Therefore, could you please tell me how you train your source-only models? Are the configurations the same as adding DA? For example, the lR is set 0.005, the training iterations for Kitti are less, and report the best results selected from every 500/ 250 iterations with some handcraft tricks?

For example, if we choose the results from 1k iter to 5k iter for Kitti on FCOS, which is a similar iteration choice for DA, the AP50 can achieve 44. Do you train all these source-only models using COCO dataset schedules and report the results of the final models? Thank you so much!

pytorch version 1.3.0 is not available on the official website and unable to install it. what is the alternate solution?

Fail to get the same results

Hi ! Thanks for your wonderful work ~But i fail to get as well as results in your paper. Can you post the ga results with VGG16 backbone in cityscape->foggy cityscape ?
GA:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.174
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.331
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.157
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.019
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.156
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.359
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.146
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.263
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.286
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.052
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.262
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.560

GA_CA:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.183
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.340
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.164
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.023
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.163
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.370
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.149
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.280
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.302
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.050
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.279
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.582

Couldn t find a setup script in /tmp/easy_install-2xsxsqeq/scikit_image-0.20.0.tar.gz

Thank you for your contribution. In the last step of installing：python setup.py build develop
, I encountered problem：Couldn t find a setup script in /tmp/easy_install-2xsxsqeq/scikit_image-0.20.0.tar.gz.
Do you know how to solve it? thank you @chengchunhsu

some question about baseline

Did you use FPN to get the baseline results in the article?（without adapt baseline for example the line that AP50=25.6)

google drive link doesn't work

i want to download model and dataset but google drive's link doesn't exist. can someone help me to download the converted annotation of Sim10k (VOC format) and Cityscapes (COCO format).

Can i use ResNet-101 for training in single GPU?

Data annotation link is not available

Hi ! Thanks for your wonderful work
II tried to download the data to reproduce the results but it failed because the files do not exist or the links are no longer available. Could you update the link to the data annotation?
Thanks!