jshilong / ddq Goto Github PK
View Code? Open in Web Editor NEWDense Distinct Query for End-to-End Object Detection (CVPR2023)
License: Apache License 2.0
Dense Distinct Query for End-to-End Object Detection (CVPR2023)
License: Apache License 2.0
Hi,
Thanks for the great work.
I would like to ask if you tried to train a DETR DDQ model on a single GPU, since I get CUDA out of memory error when using batch_size = 2 , which seems a little bit weird, since in your implementation you used 8 GPU's X 2 samples per GPU, and my setting is the same for single GPU. am I doing something wrong? my GPU is Tesla V100 with 16GB
Thanks.
"When I use train_cfg, the HungarianAssigner throws an error: TypeError: HungarianAssigner.init() got an unexpected keyword argument 'reg_cost'. However, in loss_module.py, what is used is TopkHungarianAssigner. How can I resolve this issue?"
Hi, thanks for sharing this nice work.
I am very interested in your research and my study interest is pedestrian detection. The proposed DDQ framework works well on dense detection and also achieves a SOTA performance on the CrowdHuman dataset. Is it available for sharing the related code of experiments on the CrowdHuman dataset? Thanks.
Hi, thanks for the inspiring work! Could you provide an estimate for when you plan to release the code and models?
Thanks
Hi, thanks for this work, I have read the paper and code, and still have some questions:
Your research achieved SOTA on the target coco dataset, which is amazing. I am also a practitioner in the cv industry. I am very interested in your research, and I have tried to reproduce the code myself, but I have failed due to my limited ability. I really want to learn the network structure you wrote. Can you give me A code for reference, this repository is empty, or you can upload the code to the github repository, thank you very much
Hi, thanks for opensource such a wonderful work! However, I get confused when I read the DDQ-DETR code in file ddq_detr.py
.
1, as far as I can understand, the 2d matrix distinct_query_mask
whose both rows and columns corrosponding to the distinct queries will be set False. In line 46
and line 584
, you take the first row to get indices of distinct queries, but I don't think this operation works: if the first query is selected for distinct queies, the first line in distinct_query_mask
will be all False.
2, I see both the paper and code show that DQS is performed before each decoder layer. However, if 2 distinct queries unfortunately produce similar predictions, the classification loss will still be disturbed because you use predictions other than queries at the certain layer to match gt. I wonder have you tried to do DQS after each decoder layer?
Very good work, if I want to add DDQ to an anchor anchor-based single-stage detection network, how do I need to change it, is it to add the detection header, if it is okay to add an angle regression, can you give some suggestions? Thank you.
Hello,
will this be integrated in the official mmdet library?
Thanks
Hi, congratulations on the acceptance! And thanks for sharing this nice work.
I'm training on a custom dataset, which only has two classes.
But the loss is always 0 and the AP is always -1.
Could you give some suggestions?
05/15 18:42:11 - mmengine - INFO - Epoch(train) [1][ 50/100] lr: 4.9220e-06 eta: 0:05:28 time: 0.2854 data_time: 0.0033 memory: 4989 grad_norm: 1.1815 loss: 0.0139 loss_cls: 0.0011 loss_bbox: 0.0000 loss_iou: 0.0000 d0.loss_cls: 0.0002 d0.loss_bbox: 0.0000 d0.loss_iou: 0.0000 d1.loss_cls: 0.0005 d1.loss_bbox: 0.0000 d1.loss_iou: 0.0000 d2.loss_cls: 0.0001 d2.loss_bbox: 0.0000 d2.loss_iou: 0.0000 d3.loss_cls: 0.0004 d3.loss_bbox: 0.0000 d3.loss_iou: 0.0000 d4.loss_cls: 0.0030 d4.loss_bbox: 0.0000 d4.loss_iou: 0.0000 enc_loss_cls: 0.0086 enc_loss_bbox: 0.0000 enc_loss_iou: 0.0000 dn_loss_cls: 0.0000 dn_loss_bbox: 0.0000 dn_loss_iou: 0.0000 d0.dn_loss_cls: 0.0000 d0.dn_loss_bbox: 0.0000 d0.dn_loss_iou: 0.0000 d1.dn_loss_cls: 0.0000 d1.dn_loss_bbox: 0.0000 d1.dn_loss_iou: 0.0000 d2.dn_loss_cls: 0.0000 d2.dn_loss_bbox: 0.0000 d2.dn_loss_iou: 0.0000 d3.dn_loss_cls: 0.0000 d3.dn_loss_bbox: 0.0000 d3.dn_loss_iou: 0.0000 d4.dn_loss_cls: 0.0000 d4.dn_loss_bbox: 0.0000 d4.dn_loss_iou: 0.0000 0_aux_aux_loss_cls: 0.0000 0_aux_aux_loss_bbox: 0.0000 1_aux_aux_loss_cls: 0.0000 1_aux_aux_loss_bbox: 0.0000 2_aux_aux_loss_cls: 0.0000 2_aux_aux_loss_bbox: 0.0000 3_aux_aux_loss_cls: 0.0000 3_aux_aux_loss_bbox: 0.0000 4_aux_aux_loss_cls: 0.0000 4_aux_aux_loss_bbox: 0.0000 5_aux_aux_loss_cls: 0.0000 5_aux_aux_loss_bbox: 0.0000 aux_enc_aux_loss_cls: 0.0000 aux_enc_aux_loss_bbox: 0.0000
05/15 18:42:25 - mmengine - INFO - Exp name: ddq-detr-4scale_r50_8xb2-12e_coco_20230515_184148
05/15 18:42:25 - mmengine - INFO - Epoch(train) [1][100/100] lr: 9.9240e-06 eta: 0:05:04 time: 0.2680 data_time: 0.0026 memory: 4669 grad_norm: 0.0031 loss: 0.0001 loss_cls: 0.0000 loss_bbox: 0.0000 loss_iou: 0.0000 d0.loss_cls: 0.0000 d0.loss_bbox: 0.0000 d0.loss_iou: 0.0000 d1.loss_cls: 0.0000 d1.loss_bbox: 0.0000 d1.loss_iou: 0.0000 d2.loss_cls: 0.0000 d2.loss_bbox: 0.0000 d2.loss_iou: 0.0000 d3.loss_cls: 0.0000 d3.loss_bbox: 0.0000 d3.loss_iou: 0.0000 d4.loss_cls: 0.0000 d4.loss_bbox: 0.0000 d4.loss_iou: 0.0000 enc_loss_cls: 0.0001 enc_loss_bbox: 0.0000 enc_loss_iou: 0.0000 dn_loss_cls: 0.0000 dn_loss_bbox: 0.0000 dn_loss_iou: 0.0000 d0.dn_loss_cls: 0.0000 d0.dn_loss_bbox: 0.0000 d0.dn_loss_iou: 0.0000 d1.dn_loss_cls: 0.0000 d1.dn_loss_bbox: 0.0000 d1.dn_loss_iou: 0.0000 d2.dn_loss_cls: 0.0000 d2.dn_loss_bbox: 0.0000 d2.dn_loss_iou: 0.0000 d3.dn_loss_cls: 0.0000 d3.dn_loss_bbox: 0.0000 d3.dn_loss_iou: 0.0000 d4.dn_loss_cls: 0.0000 d4.dn_loss_bbox: 0.0000 d4.dn_loss_iou: 0.0000 0_aux_aux_loss_cls: 0.0000 0_aux_aux_loss_bbox: 0.0000 1_aux_aux_loss_cls: 0.0000 1_aux_aux_loss_bbox: 0.0000 2_aux_aux_loss_cls: 0.0000 2_aux_aux_loss_bbox: 0.0000 3_aux_aux_loss_cls: 0.0000 3_aux_aux_loss_bbox: 0.0000 4_aux_aux_loss_cls: 0.0000 4_aux_aux_loss_bbox: 0.0000 5_aux_aux_loss_cls: 0.0000 5_aux_aux_loss_bbox: 0.0000 aux_enc_aux_loss_cls: 0.0000 aux_enc_aux_loss_bbox: 0.0000
05/15 18:42:25 - mmengine - INFO - Saving checkpoint at 1 epochs
/media/New Volume/Project/DETR/DDQ-ddq_detr/mmdet/models/layers/positional_encoding.py:84: UserWarning: floordiv is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
dim_t = self.temperature**(2 * (dim_t // 2) / self.num_feats)
/home/anaconda3/envs/detr/lib/python3.7/site-packages/torch/functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/conda/conda-bld/pytorch_1646755953518/work/aten/src/ATen/native/TensorShape.cpp:2228.)
return _VF.meshgrid(tensors, kwargs) # type: ignore[attr-defined]
/media/New Volume/Project/DETR/DDQ-ddq_detr/mmdet/models/layers/transformer/utils.py:71: UserWarning: floordiv is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
dim_t = temperature(2 * (dim_t // 2) / num_feats)
05/15 18:42:31 - mmengine - INFO - Epoch(val) [1][50/50] eta: 0:00:00 time: 0.0446 data_time: 0.0018 memory: 723
05/15 18:42:31 - mmengine - INFO - Evaluating bbox...
Loading and preparing results...
DONE (t=0.01s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type bbox
DONE (t=0.04s).
Accumulating evaluation results...
DONE (t=0.01s).
Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = -1.000
Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=1000 ] = -1.000
Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=1000 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = -1.000
Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=300 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=1000 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = -1.000
Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = -1.000
Is it possible to release the code and training configuration for training the crowdhuman dataset?
Sorry to disturb but where are the supplementary materials?
I have solved the issue!
作者你好,我按照readme的方法查找ddq_detr的相关文件,并没有相应的结果,请问问题出在哪里呢
您好,我想尝试DDQ算法,只是硬件受限(RTX 2080Ti,Nvidia Driver430.50,CUDA10.1, CUDNN 7.6.5),如果想在mmdet 2.x版本使用该方法进行部署,请问有没有修改建议,谢谢。
Hi, thanks for providing such a wonderful work.
There seems no ablation when using a better loss design for dense queries in the paper. Compared to using focal loss, how much does it improve the performance?
And different from DDQ-FCN, why did you use TOOD loss only for auxiliary in DDQ-DETR?
作者您好,我在搭建DDQ_detr环境时这一步
ln -s mmcv-2.0.0rc4/mmcv ./
会报错ln: failed to create symbolic link ‘./mmcv’: File exists。
好像是因为和文件夹“mmcv-2.0.0rc4/mmcv”重名了,请问这个符号链接的绝对位置应该放在哪里啊。
另外,mmdetection和mmengine还需要构建吗?
Hello, there is a slight difference between the crowdhuman dataset metrics I trained locally and the metrics you displayed. If possible, could you please provide me with your trained model?Best wishes.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.