Did anyone reach the result as the paper shows? or Could you show the result test on mser-td500 or ctw1500 by running the code?

[mhliao]

Did anyone reach the result as the paper shows? or Could you show the result test on mser-td500 or ctw1500 by running the code?

[MhLiao]

@shaohailin @kasyoukin Please check the input size of the images. The current default image size is 512 and the resnet50 model in the paper uses 736. Moreover, since td500 is labeled at the text line level instead of the word level, it may require lower value of box_thresh. The current results indicate that you need a lower box_thresh since the recall is much lower than the precision.

[shaohailin]

@MhLiao

[MhLiao]

@shaohailin Please show your running command and the results.

[kasyoukin]

xxx@xxx-ubuntu:~/gitcode/DB$ python3 eval.py experiments/seg_detector/td500_resnet50_deform_thre.yaml --resume models/td500_resnet50 --polygon --box_thresh 0.6
./datasets/TD_TR/TD500/
[INFO] [2019-12-06 11:06:45,065] Resuming from models/td500_resnet50
[INFO] [2019-12-06 11:06:45,713] Resumed from models/td500_resnet50
[INFO] [2019-12-06 11:06:53,777] fmeasure : 0.714142 (1)
[INFO] [2019-12-06 11:06:53,777] recall : 0.611684 (200)
[INFO] [2019-12-06 11:06:53,777] precision : 0.857831 (200)

I think use the param of totaltext to test TD500 is wrong ...

[kasyoukin]

@MhLiao thank you! it is work for me

xxx@xxx-ubuntu:~/gitcode/DB$ python3 eval.py >experiments/seg_detector/td500_resnet50_deform_thre.yaml --resume models/td500_resnet50 --box_thresh 0.5
./datasets/TD_TR/TD500/
[INFO] [2019-12-06 11:38:57,799] Resuming from models/td500_resnet50
[INFO] [2019-12-06 11:38:57,873] Resumed from models/td500_resnet50
[INFO] [2019-12-06 11:39:11,401] recall : 0.807560 (200)
[INFO] [2019-12-06 11:39:11,401] precision : 0.910853 (200)
[INFO] [2019-12-06 11:39:11,401] fmeasure : 0.856102 (1)

[luqiang6q]

@kasyoukin I have the same problem.Can you show me the box_thresh value and image size that you setting in the end.Please....

[luqiang6q]

@MhLiao 您好，我有以下几个问题。
0. 您给的几个模型我测试了，其中td500的两个模型，测试结果与该问题提问者差不多，而且我后续也修改了尺寸和box_thresh，结果均相差不大，甚至我把box_thresh设置为0.1，recall只提升了一个点，我在想是否有可能提供的模型不是最优模型。如果您有空的话，您可以测试下看看。

1. 我从头开始训练了icdar2015数据集resnet50和resnet18的模型，最终结果与论文均差5个点。想问下您在训练时是否用了预训练模型，如果是，用了哪一种呀？
2. 论文图4的SB和DB的注释是否标颠倒了，我个人理解，橙色线应该是SB曲线，蓝色线应该是DB曲线吧
3. 发现一个bug，average_meter.py中的self.avg = self.sum / self.count在测试速度时，可能会报错ZeroDivisionError: division by zero，我遇到了2次
4. 我训练时，想修改学习率。在.yaml文件中修改了类OptimizerScheduler中的lr的值，但是并不起作用，而是在类DecayLearningRate中添加了参数lr并赋值我想要的初始学习率，这样做才能起作用。我想问下您类OptimizerScheduler中的lr是起啥作用的呀？
   谢谢了哈～

[MhLiao]

@luqiang6q
0. td500的两个模型，box_thresh应该设置为0.4~0.6即可取得不错的结果，具体参考楼上kasyoukin的回复，他用0.5的box_thresh。你可以对比一下自己的结果和论文中的结果的precision和recall，如果相比论文，你的结果里的precision和recall一个偏高一个偏低，可以调整box_thresh。

1. 使用了SynthText预训练模型，但是即使不使用预训练模型，也不会差5个点这么多。值得注意的是IC15需要用更大的输入尺度，请参考论文。
2. 论文中的图在新的arxiv版本中已更正。
3. 还不太清楚原因
4. 我看了一下可能确实有这个问题，目前的代码里确实有很多冗余的参数，我之后会重新组织和精简。

[kasyoukin]

@kasyoukin I have the same problem.Can you show me the box_thresh value and image size that you setting in the end.Please....
@shaohailin 736 , the cmd is
python3 eval.py experiments/seg_detector/td500_resnet50_deform_thre.yaml --resume models/td500_resnet50 --box_thresh 0.5

[TyrionChou]

请问ZeroDivisionError: division by zero问题解决了嘛？

[TyrionChou]

ZeroDivisionError: division by zero 是由于eval.py 代码175行的continue导致后面的内容没有计算，把continue注释就好了！

[shaohailin]

@kasyoukin could you sent me the dataset of td500, i met something wrong about it

[shaohailin]

Please check the input size of the images. The current default image size is 512 and the resnet50 model in the paper uses 736.
作者你好！我看代码训练时是将原图resize到640 640，测试时是输入大小是736 736 ，这与paper里的设置是一致的吗？
或者，请作者具体说一下736在哪里设置，是训练还是测试？ @MhLiao @kasyoukin @TyrionChou

[ZhengMengbin]

@shaohailin 同样的疑惑，是不是训练也要resize到736*736，才能达到作者paper里的结果，我按默认的配置文件的结果是这样的，和paper里差几个点：
[INFO] [2020-06-16 10:49:38,603] precision : 0.853933 (200)
[INFO] [2020-06-16 10:49:38,603] recall : 0.783505 (200)
[INFO] [2020-06-16 10:49:38,604] fmeasure : 0.817204 (1)

[shaohailin]

@ZhengMengbin 你的结果是哪个数据集上的？我在td500上默认设置可以达到论文提到的效果 Ctw1500数据集我还没弄好？能否问一下你在icdar2015上的结果？