chengchunhsu / everypixelmatters Goto Github PK
View Code? Open in Web Editor NEWImplementation of ECCV 2020 paper "Every Pixel Matters: Center-aware Feature Alignment for Domain Adaptive Object Detector"
License: Other
Implementation of ECCV 2020 paper "Every Pixel Matters: Center-aware Feature Alignment for Domain Adaptive Object Detector"
License: Other
I have downloaded both the sim10k and cityscapes datasets and tries to repeat the training process.
But when I use the following commond
python3 -m torch.distributed.launch --nproc_per_node=4 --master_port=$((RANDOM + 10000)) tools/train_net_da.py --config-file ./configs/da_ga_sim10k_VGG_16_FPN_4x.yaml
I have this error:
RuntimeError: Error(s) in loading state_dict for DistributedDataParallel:
size mismatch for module.fpn.fpn_inner3.weight: copying a param with shape torch.Size([256, 1024, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for module.fpn.fpn_inner4.weight: copying a param with shape torch.Size([256, 2048, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 512, 1, 1]).
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for DistributedDataParallel:
size mismatch for module.fpn.fpn_inner3.weight: copying a param with shape torch.Size([256, 1024, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 256, 1, 1]).
size mismatch for module.fpn.fpn_inner4.weight: copying a param with shape torch.Size([256, 2048, 1, 1]) from checkpoint, the shape in current model is torch.Size([256, 512, 1, 1]).
Can anyone give me any clue why this happens?
Hi, your work is great.
I wonder why does the atten_map need another sigmoid at this line?
In my view, the box_cls_map and centerness_map are the normalized score after sigmoid already, so
0<box_cls_map<1
0<centerness_map <1
, then,
0 < box_cls_map * centerness_map < 1,
0 <= center_aware_weight * box_cls_map * centerness_map < center_aware_weight
, so,
atten_map >= 0.5 forever.
If we set center_aware_weight to [0, 1], the
center_aware_weight * box_cls_map * centerness_map is also a normalized score,
then why does the atten_map need the sigmoid again?
Could you update the link for the checkpoints (i.e. pretrained models)?
Thank you for your amazing work!!!
I have reproduced almost all your results about GA and CA in your papers. Actually, it is really hard to select the best models. I have to save a checkpoint every 250 iters following your configs and test them one by one with some tricks. On ResNet101, I can get 45 AP50 on Kitti, 40 AP50 on CS, and 51.5 AP50 on Sim10k. I am not sure if you select the model by testing a lot of candidates.
However, if I use the same configurations and same model-selection strategies to train a source-only model on pure FCOS, I can easily get 44 AP50 on Kitti, 27 AP50 on CS, and about 47 on Sim10k!! The results on CS seem normal. We have tested both on your codes and pure FCOS environments, achieving similar results. This really confused me a lot.
Therefore, could you please tell me how you train your source-only models? Are the configurations the same as adding DA? For example, the lR is set 0.005, the training iterations for Kitti are less, and report the best results selected from every 500/ 250 iterations with some handcraft tricks?
For example, if we choose the results from 1k iter to 5k iter for Kitti on FCOS, which is a similar iteration choice for DA, the AP50 can achieve 44. Do you train all these source-only models using COCO dataset schedules and report the results of the final models? Thank you so much!
I notice the step parameters are set to be (120000, 160000) in most config files while the max_iter is 20,000. I am wondering should it be (12000, 16000) for the step parameters? Or the learning rate for the last iteration is better to equal to base_lr?
Thx!
Hi,can you add link again?
Hi, i have received the cityscapes dataset. However, when i click the link for foggy cityscapes, i don't know which one is i really need. In the website, i see some foggy cityscapes datasets for semantic segmentation, not for object detection. Can you give me the link for foggy dataset specifically. Thanks!
Traceback (most recent call last):
File "C:/Users/84957/Desktop/qin/every/EveryPixelMatters-master/tools/train_net_da.py", line 480, in
main()
File "C:/Users/84957/Desktop/qin/every/EveryPixelMatters-master/tools/train_net_da.py", line 469, in main
model = train(cfg, args.local_rank, args.distributed)
File "C:/Users/84957/Desktop/qin/every/EveryPixelMatters-master/tools/train_net_da.py", line 361, in train
arguments,
File "C:\Users\84957\Desktop\qin\every\EveryPixelMatters-master\fcos_core\engine\trainer.py", line 128, in do_train
in enumerate(zip(data_loader_source, data_loader_target), start_iter):
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data\dataloader.py", line 363, in next
data = self._next_data()
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data\dataloader.py", line 989, in _next_data
return self._process_data(data)
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data\dataloader.py", line 1014, in _process_data
data.reraise()
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch_utils.py", line 395, in reraise
raise self.exc_type(msg)
AttributeError: Caught AttributeError in DataLoader worker process 0.
Original Traceback (most recent call last):
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data_utils\worker.py", line 185, in _worker_loop
data = fetcher.fetch(index)
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torch\utils\data_utils\fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "C:\Users\84957\Desktop\qin\every\EveryPixelMatters-master\fcos_core\data\datasets\coco.py", line 67, in getitem
img, anno = super(COCODataset, self).getitem(idx)
File "D:\Anaconda\envs\TF2.1\lib\site-packages\torchvision\datasets\coco.py", line 118, in getitem
img, target = self.transforms(img, target)
File "C:\Users\84957\Desktop\qin\every\EveryPixelMatters-master\fcos_core\data\transforms\transforms.py", line 15, in call
image, target = t(image, target)
File "C:\Users\84957\Desktop\qin\every\EveryPixelMatters-master\fcos_core\data\transforms\transforms.py", line 60, in call
target = target.resize(image.size)
AttributeError: 'list' object has no attribute 'resize'
I trained according to the data set format you provided, but this error occurred. Does the annotation file need to be converted? Looking forward to your reply
I have a general question/feature request about data augmentation. I've been manually adding in additional data augmentation (through the Albumentations package). Applying random photometric distortions (e.g. Gaussian blur, color jitter, etc.) significantly helps with the domain generalization. However, after I started adding random geometric distortions (e.g. random affine distortions), performance on the validation target domain decreased by about 5%, rather than improving, as I've observed in supervised learning. I'm fairly certain this isn't a bug, as I've checked and tested the various pipelines many times. I was wondering, do these types of augmentations mess up the discrimator fooling process? Or, is there anywhere I can find code that adds data augmentation to the fcos training routine?
Thank you!
Hi ! Thanks for your wonderful work ~But i fail to get as well as results in your paper. Can you post the ga results with VGG16 backbone in cityscape->foggy cityscape ?
GA:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.174
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.331
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.157
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.019
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.156
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.359
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.146
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.263
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.286
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.052
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.262
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.560
GA_CA:
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.183
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.340
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.164
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.023
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.163
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.370
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.149
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.280
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.302
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.050
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.279
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.582
i want to download model and dataset but google drive's link doesn't exist. can someone help me to download the converted annotation of Sim10k (VOC format) and Cityscapes (COCO format).
Hello,
Thanks for providing the code. I was able to run the code for Sim10k --> Cityscapes and got scores lower than reported. mAP 24.8, mAP @0.5 42.6, mAP @0.75 24.4 , mAP @s 5.3 , mAP @m 27.3 , mAP@L 51.1 . One thing that I noticed is that the domain classifier loss goes down to zero for all the pyramid scales after few iterations. Could you please let me know what kind of trend you see for the domain classifier losses, which could possibly be the reason for me seeing the lower scores?
Thanks
Thank you for your contribution. In the last step of installing:python setup.py build develop
, I encountered problem:Couldn t find a setup script in /tmp/easy_install-2xsxsqeq/scikit_image-0.20.0.tar.gz.
Do you know how to solve it? thank you @chengchunhsu
Hi ! Thanks for your wonderful work
II tried to download the data to reproduce the results but it failed because the files do not exist or the links are no longer available. Could you update the link to the data annotation?
Thanks!
After loading model log contains information below. Can you please see what's the query?
It was running on actual dataset, but on custom data, it is producing this error. Thanks
loading annotations into memory...
Done (t=0.20s)
creating index...
index created!
2020-09-26 17:28:51,585 fcos_core.trainer INFO: Start training
Traceback (most recent call last):
File "tools/train_net_da.py", line 480, in
main()
File "tools/train_net_da.py", line 469, in main
model = train(cfg, args.local_rank, args.distributed)
File "tools/train_net_da.py", line 361, in train
arguments,
File "/home/akhtar/EveryPixelMatters/fcos_core/engine/trainer.py", line 128, in do_train
in enumerate(zip(data_loader_source, data_loader_target), start_iter):
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 819, in next
return self._process_data(data)
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 846, in _process_data
data.reraise()
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/_utils.py", line 385, in reraise
raise self.exc_type(msg)
ValueError: Caught ValueError in DataLoader worker process 2.
Original Traceback (most recent call last):
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop
data = fetcher.fetch(index)
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/akhtar/epmatters/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 44, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/home/akhtar/EveryPixelMatters/fcos_core/data/datasets/sim10k.py", line 46, in getitem
target = self.get_groundtruth(index)
File "/home/akhtar/EveryPixelMatters/fcos_core/data/datasets/sim10k.py", line 65, in get_groundtruth
target = BoxList(anno["boxes"], (width, height), mode="xyxy")
File "/home/akhtar/EveryPixelMatters/fcos_core/structures/bounding_box.py", line 27, in init
"bbox should have 2 dimensions, got {}".format(bbox.ndimension())
ValueError: bbox should have 2 dimensions, got 1
I HAVE CHECKED SEVERAL TIMES THAT THERE IS NO EMPTY ANNOTATIONS
hi,
Thanks for your open code! my results have some difference from this paper. and I think that my configs may be wrong, so I want to look at your training log of C to F. Thank you again!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.