ericzgw/pyramidbox

A Context-assisted Single Shot Face Detector in TensorFlow

PyramidBox

This is an unofficial Tensorflow re-implementation of PyramidBox: A Context-assisted Single Shot Face Detector, which achieves superior performance among the state-of-the-art on the two common face detection benchmarks, FDDB and WIDER FACE.

Note

There is still a gap in performance from the paper. May be caused by several reasons:

• Without implementing data-anchor-sampling.
• Differences of data augmentation from original.
• The batch size in the paper is 16, but I used 1 because of the limitation of memory.
• Hyperparameters not mentioned in the paper.
• Differences of deep learning framework.

Results

Face Detection

Results on WIDER FACE validation set:

This is just a very casual training result. I believe you can achieve better results after trying some other hyperparameters. For example: batch size, learning rate and some parameters related to the loss function,etc.

Method	AP Easy	AP Medium	AP Hard
original	96.1	95.0	88.9
this repo	90.6	88.8	73.4

Usage

Prerequisites

(Only tested on) Ubuntu 16.04 with:

• Python3
• Tensorflow-gpu 1.4
• OpenCV3

Clone the repo

git clone https://github.com/EricZgw/PyramidBox.git
python makedir.py

Download PyramidBox models form BaiduYun or GoogleDrive .

Demo

Run the following script for visualization:

python demo.py

Train on WIDER FACE Datasets

1. Download pre-trained VGG16 models from here and put it to /checkpoints.
   
2. Download WIDER FACE Datasets and convert to VOC format. Path looks like below:

datasets/
       |->widerface/
       |    |->WIDER_train/
       |    |->WIDER_val/
       |    |->WIDER_test/
       |    |->Annotations/
       |    |->JPEGImages/
       |    |...

3. Run the following script to generate TFRecords:

python datasets/pascalvoc_to_tfrecords.py
You can run `check_data_io.py` to check data. This step is not necessary.

4. The training strategy is two-stages: First run train_model.py with below setting to train additional PyramidBox layers:

self.fine_tune_vgg16 = False

5. Then set self.fine_tune_vgg16 =Ture to run train_model.py to train total network.

Validation

Run the following script for evaluation and get mAP:

python widerface_eval.py
cd eval/eval_tools
octave wider_eval.m

TODO

• Add data-anchor-sampling
• Try more logical and rigorous data augmentation
• Transfer to other backbone networks

Reference

SSD-Tensorflow
SSD_tensorflow_VOC

Contact

If you find any problems, welcome to open a new issue or contact [email protected] .

pyramidbox's Issues

how to change batch_size?

I turn batch_size into 16,an error has occurred.

有人将这ckpt转化为pb

is the loss normal after global step 15000+?

after global step 15000+, the loss is still very large, is it normal?

INFO:tensorflow:global step 15489: loss = 24.8054 (0.23 sec/step)
INFO:tensorflow:global step 15499: loss = 21.6636 (0.24 sec/step)
INFO:tensorflow:global step 15509: loss = 26.9590 (0.24 sec/step)
INFO:tensorflow:global step 15519: loss = 26.0390 (0.24 sec/step)
INFO:tensorflow:global step 15529: loss = 23.1402 (0.26 sec/step)

How to prepare train dataset

Hi, thanks for sharing the codes.

I am trying to train the model myself, but I am encountering the same issue as
#7

Could you please elaborate on how to prepare training data? What are the contents of different directories you mentioned in the readme?

I would really appreciate it if you could give me some help!

widerface_eval.py

为什么widerface_eval.py运行完只生成了57个子集结果？这个问题能解决吗？麻烦告知。

关于demo.py我没测试成功

哈喽，各位大佬，
git clone后我下载ckpt放到./model/
然后运行，结果死了，图片没反应，也不知道哪里错了。
图片读得正确，模型放的也对，
['1.jpg', '2.jpg']

# Restore SSD model.
ckpt_filename = './model/pyramidbox.ckpt'

isess.run(tf.global_variables_initializer())
saver = tf.train.Saver()
saver.restore(isess, ckpt_filename)

请教下这是为啥子？多谢
运行过程也没报错

['1.jpg', '2.jpg']
2019-11-27 17:44:01.041820: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2019-11-27 17:44:02.662671: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0

how to use the trained models to test?

Tensor Flow error

After running demo.py I am getting this error.Can you please help:
RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
return f(*args, **kwds)
2018-08-31 11:47:41.373697: W tensorflow/core/framework/allocator.cc:108] Allocation of 774471680 exceeds 10% of system memory.
2018-08-31 11:47:42.524110: W tensorflow/core/framework/allocator.cc:108] Allocation of 774471680 exceeds 10% of system memory.

Any reason for low accuracy results?

First of all, thanks for your contribution of sharing your work.
I'm wondering if there' re any difference(other than hyperparameters) between the original code and this work that results in lower accuracy compared to the ones in paper. Are there any changes or missing parts from original source code?
Thanks

What is the speed of detection ？

for 640*480，What is the speed of detection？ GPU memory and CPU memory? Thx

Train loss issue

fune train ,after 200000+ the loss is very large?

convert to VOC format.

How to convert to VOC format.

how to frozen model? help me, please

训练效果没有给出的模型好

你好，我按照训练步骤训练了一版，但是效果没有给出的模型那么好，很多误检，是我哪里写的不对吗？代码我也没有改动

AttributeError: 'module' object has no attribute 'plasma'

when I run demo.py , this problem came in and I couldn't solve that，appreciate if anyone can help me.
the details as below:
def colors_subselect(colors, num_classes=21):
dt = len(colors) // num_classes
sub_colors = []
for i in range(num_classes):
color = colors[i*dt]
if isinstance(color[0], float):
sub_colors.append([int(c * 255) for c in color])
else:
sub_colors.append([c for c in color])
return sub_colors
colors_plasma = colors_subselect(mpcm.plasma.colors, num_classes=21)
AttributeError: 'module' object has no attribute 'plasma'

Change Anchor Boxes Aspect Ratio

Dear @EricZgw,
Thank you for your nice work.
If one wants to change the aspect ratios of anchor boxes for example, to 1:1.4 (width:height) the network structure must be changed or just the training code must be changed? Would you please explain how one can change the aspect ratios of anchor boxes?

No module named 'tf_extended'

import tf_extended as tfe
ModuleNotFoundError: No module named 'tf_extended'

train_model

train_model.py
4、The training strategy is two-stages: First run train_model.py with below setting to train additional PyramidBox layers:
self.fine_tune_vgg16 = False
5、Then set self.fine_tune_vgg16 =Ture to run train_model.py to train total network.
第4步什么时候可以停止，然后开始第5步操作？

test results are error.

After running train_model.py, I put model.ckpt-25000 on /model ,but the result of running demo.py has many boxes that all are more smaller than face size and no one detect face successfully.

请问body，head，face这些数据集从哪来啊？

另外，他基于s3fd，控制低层负样本，找最大的负样本和正样本训练时怎么实现的啊，就是max-in-out这个？谢谢

pyramidbox: why anchor down-sampled?

I have a question disturbed me. As we know, the anchor will be expanded to detect head and body from face anchors, but it is down-sampled to calculate IOU with region target to confirm the label. After all, the head and body are larger than face. Thank you very much.

Modification of batch_size results in ValueError

From 1 to 16. I got this:

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1592, in _create_c_op
    raise ValueError(str(e))
ValueError: Dimensions must be equal, but are 520400 and 136400 for 'ssd_losses/or1' (op: 'LogicalOr') with input shapes: [520400], [136400].

I am not familiar with Tensorflow...

test AP

Is there the code to test AP?

Error with provided model.

I downloaded the model from BaiduYun link you provided and put them into model direcotry, then I run python3 demo.py , and this error occured:

DataLossError (see above for traceback): Checksum does not match: stored 189470084 vs. calculated on the restored bytes 1055297358
[[{{node save/RestoreV2}} = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]
[[{{node save/RestoreV2/_301}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_306_save/RestoreV2", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

DataLossError (see above for traceback): Checksum does not match: stored 2973489966 vs. calculated on the restored bytes 4004437628

When I run demo.py , it show :

Traceback (most recent call last):
File "demo2.py", line 51, in
saver.restore(isess, ckpt_filename)

Caused by op 'save/RestoreV2_33', defined at:
File "demo2.py", line 50, in
saver = tf.train.Saver()

DataLossError (see above for traceback): Checksum does not match: stored 2973489966 vs. calculated on the restored bytes 4004437628
[[Node: save/RestoreV2_33 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_33/tensor_names, save/RestoreV2_33/shape_and_slices)]]
[[Node: save/RestoreV2_76/_163 = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_884_save/RestoreV2_76", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]]

How to prepare training data?

Could you please elaborate on how to prepare training data? What are the contents of different directories you mentioned in the readme?

What are the actual implications of rpredictions in the code?

What are the actual implications of rpredictions in the code? Are they the probabilities of detecting or not detecting faces?

Please help me！

how to change batch_size?

I turned batch_size into 16 ,an error has occurred.