Detections much worse than YOLOv3?

[google]

Detections much worse than YOLOv3?

[mingxingtan]

@AakashKumarNain This image is challenging: it has a lot of objects and many of them are very small. Therefore, you need to change the visualization settings in inferece.py line 208: (1) change min_score_thresh=0.1 and (2) add max_boxes_to_draw=500.

After that you would get reasonable results. However, since the original image is 1920x1080, much bigger than the default efficientdet-d0 size 512x512, you can simply change the image size of d0 by add "self.params['image_size'] = 1920" in inference.py line 234.

Here are the two updated results:

EfficientDet-D0 (min_score_thresh=0.1 and max_boxes_to_draw=500, image_size=512)

EfficientDet-D0 (min_score_thresh=0.1 and max_boxes_to_draw=500, image_size=1920)

[deepakcrk]

Please post original image also

[AakashKumarNain]

Please post original image also

Done

[fsx950223]

Are these model trained on same dataset？
Yolov3 mark bus as truck.
Efficientdet mark bus as bus.

[AakashKumarNain]

Both are trained on COCO if I am right else how would you compare the performance as reported in the paper?

[fsx950223]

Both are trained on COCO if I am right else how would you compare the performance as reported in the paper?

I try to train it on VOC2007 dataset and compare it with my yolov3 model.
I will use the same input size ,score and iou. Then compare them with speed and map.

[WonTaeYeon]

Maybe..... In anchors.py' it is a good idea to increase MAX_DETECTIONS_PER_IMAGE (line 44) and decrease match_threshold (line 317; Default == 0.5).

[AakashKumarNain]

@mingxingtan Thanks for clearing that up. The threshold for yolov3 is much smaller, that makes sense. Thank you.

Looking forward to the lite model. I basically want to replace all of my detector models to these, especially for edge devices

[mingxingtan]

@AakashKumarNain What's the copyright of this image? I wonder may I use it in this repo? (like tutorial or README)

[AakashKumarNain]

@mingxingtan I took it from google images randomly.

[mingxingtan]

Gotcha. Do you still have the original link?

[AakashKumarNain]

Yes. It's here.

[mingxingtan]

Great, thanks!

[AakashKumarNain]

You are welcome. 😄

[glenn-jocher]

YOLOv3 at 1920. Detection command is here. Model is downloaded automatically.

git clone https://github.com/ultralytics/yolov3
cd yolov3
python3 detect.py --img 1920 --source img.jpg

image 1/1: 1088x1920 76 persons, 32 cars, 64 motorcycles, 2 buss, 1 trucks, 2 stop signs, Done. (0.083s)

[HyunjiEllenPak]

@AakashKumarNain This image is challenging: it has a lot of objects and many of them are very small. Therefore, you need to change the visualization settings in inferece.py line 208: (1) change min_score_thresh=0.1 and (2) add max_boxes_to_draw=500.

After that you would get reasonable results. However, since the original image is 1920x1080, much bigger than the default efficientdet-d0 size 512x512, you can simply change the image size of d0 by add "self.params['image_size'] = 1920" in inference.py line 234.

Here are the two updated results:

EfficientDet-D0 (min_score_thresh=0.1 and max_boxes_to_draw=500, image_size=512)

EfficientDet-D0 (min_score_thresh=0.1 and max_boxes_to_draw=500, image_size=1920)

@mingxingtan
Hi, I have a question about your above answer.
You said that the above image is hard image to detect objects, so we should decrease min_score_thresh=0.1 and increase add_max_boxes_to_draw=500. But I think that lowering the thresh to get the good result for hard images is not robust way because it makes getting over-detection results on easy images.

In the section 1.3 of your jupyter notebook tutorial, your uploaded model efficientDet-d0 got good results even though setting min_score to 0.4. Why our own trained model has poor result than your trained model? I guess most users has their own one GPU, So it has limitation to increase batch_size. But in your paper, you used lot of GPUs and batch size=128, that is generally impossible value to us.
Are the EfficientDet result highly dependent on the batch size? How about your opinion?

[fsx950223]

@AakashKumarNain This image is challenging: it has a lot of objects and many of them are very small. Therefore, you need to change the visualization settings in inferece.py line 208: (1) change min_score_thresh=0.1 and (2) add max_boxes_to_draw=500.
After that you would get reasonable results. However, since the original image is 1920x1080, much bigger than the default efficientdet-d0 size 512x512, you can simply change the image size of d0 by add "self.params['image_size'] = 1920" in inference.py line 234.
Here are the two updated results:

EfficientDet-D0 (min_score_thresh=0.1 and max_boxes_to_draw=500, image_size=512)

EfficientDet-D0 (min_score_thresh=0.1 and max_boxes_to_draw=500, image_size=1920)

@mingxingtan
Hi, I have a question about your above answer.
You said that the above image is hard image to detect objects, so we should decrease min_score_thresh=0.1 and increase add_max_boxes_to_draw=500. But I think that lowering the thresh to get the good result for hard images is not robust way because it makes getting over-detection results on easy images.

In the section 1.3 of your jupyter notebook tutorial, your uploaded model efficientDet-d0 got good results even though setting min_score to 0.4. Why our own trained model has poor result than your trained model? I guess most users has their own one GPU, So it has limitation to increase batch_size. But in your paper, you used lot of GPUs and batch size=128, that is generally impossible value to us.
Are the EfficientDet result highly dependent on the batch size? How about your opinion?

I train and test voc2007 with efficientnet ckpt as backbone and I got 73 mAP.
Do you use efficientnet ckpt?
And I use mine mAP to test performance.
It seems something wrong with coco_metric.