unsky / fpn Goto Github PK
View Code? Open in Web Editor NEWFeature Pyramid Networks for Object Detection
Feature Pyramid Networks for Object Detection
rpn_cls_score/p2这个layer在trian.prototxt和test.prototxt,output参数不一致
您好,我最近用ResNet50-fpn在cityscape数据集上训练的时候总是出现梯度爆炸,然后loss出现nan,但是用ResNet50单分支就不会。想请教一下您对这个问题怎么看,这个是哪里出了问题呢。目前,把学习率设置为0.0001不会崩掉,但是训练结果不是很好。我看您的这个网络中,不同的层初始化权重还不一样,这个weight的初始化权重对网络是不是有很大影响呢,我尝试着去调了weight的初始化权重,但是没有出现很好的效果,请问有没有什么调节的技巧吗。
where is resnet?
你好,unsky。我在做小目标检测的实验,之前用你的旧代码(VGG16模型)召回率可以达到90%左右,用新的代码(RESNET50 sharedrcnn)召回率只有70%左右。尝试改了scales为[4,8]和FPN/lib/fast_rcnn/config.py里的一些参数,但是效果都不理想,能不能给一些建议,谢谢!
Dear unsky, have you successfully trained your network? can you provide a copy of the output caffemodel?
hi, unsky
Did you have ever trained FPN on coco dataset? when I did this work, there are some errors. Like,
../lib/rpn/proposal_layer.py", line 209, in forward
pad = npr.choice(keep, size=int(post_nms_topN) - len(keep))
File "mtrand.pyx", line 1121, in mtrand.RandomState.choice
ValueError: a must be non-empty
and,
../lib/fast_rcnn/bbox_transform.py:50: RuntimeWarning: overflow encountered in exp
pred_h = np.exp(dh) * heights[:, np.newaxis]
thanks.
According to the paper,
Finally, we append a 3×3 convolution on each merged map togenerate the final feature map, which is to reduce the alias-ing effect of upsampling.
But in the network you defined, there is no such layer.
Why?
您好,想请教一下FPN的训练测试以及准备数据集是不是和faster rcnn是一样的?我准备好准备工作demo.py出现bash 权限不足的问题是怎么回事呢?希望很快收到您的回复,谢谢!
hi unsky,when i run ./experiments/scripts/FP_Net_end2end.sh 1 VGG16 pascal_voc,i have got the error:
++ set -e
++ export PYTHONUNBUFFERED=True
++ PYTHONUNBUFFERED=True
++ GPU_ID=0
++ NET=VGG16
++ NET_lc=vgg16
++ DATASET=pascal_voc
++ array=($@)
++ len=3
++ EXTRA_ARGS=
++ EXTRA_ARGS_SLUG=
++ case $DATASET in
++ TRAIN_IMDB=voc_2007_trainval
++ TEST_IMDB=voc_2007_test
++ PT_DIR=pascal_voc
++ ITERS=150000
+++ date +%Y-%m-%d_%H-%M-%S
++ LOG=experiments/logs/faster_rcnn_end2end_VGG16_.txt.2017-11-04_19-35-27
++ exec
+++ tee -a experiments/logs/faster_rcnn_end2end_VGG16_.txt.2017-11-04_19-35-27
tee: experiments/logs/faster_rcnn_end2end_VGG16_.txt.2017-11-04_19-35-27: No such file or directory
++ echo Logging output to experiments/logs/faster_rcnn_end2end_VGG16_.txt.2017-11-04_19-35-27
Logging output to experiments/logs/faster_rcnn_end2end_VGG16_.txt.2017-11-04_19-35-27
++ ./tools/train_net.py --gpu 0 --solver models/pascal_voc/VGG16/FP_Net_end2end/solver.prototxt
Called with args:
Namespace(cfg_file=None, gpu_id=0, imdb_name='voc_2007_trainval', max_iters=40000, pretrained_model=None, randomize=False, set_cfgs=None, solver='models/pascal_voc/VGG16/FP_Net_end2end/solver.prototxt')
Using config:
{'DATA_DIR': '/home/tanglin/FPN-master/data',
'DEDUP_BOXES': -1,
'EPS': 1e-14,
'EXP_DIR': 'default',
'GPU_ID': 0,
'MATLAB': 'matlab',
'MODELS_DIR': '/home/tanglin/FPN-master/models/pascal_voc',
'PIXEL_MEANS': array([[[ 102.9801, 115.9465, 122.7717]]]),
'RNG_SEED': 3,
'ROOT_DIR': '/home/tanglin/FPN-master',
'TEST': {'BBOX_REG': True,
'HAS_RPN': False,
'MAX_SIZE': 520,
'NMS': 0.3,
'PROPOSAL_METHOD': 'rpn',
'RPN_MIN_SIZE': 16,
'RPN_NMS_THRESH': 0.7,
'RPN_POST_NMS_TOP_N': 2000,
'RPN_PRE_NMS_TOP_N': 6000,
'SCALES': [375],
'SVM': False},
'TRAIN': {'ASPECT_GROUPING': True,
'BATCH_SIZE': 128,
'BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'BBOX_NORMALIZE_MEANS': [0.0, 0.0, 0.0, 0.0],
'BBOX_NORMALIZE_STDS': [0.1, 0.1, 0.2, 0.2],
'BBOX_NORMALIZE_TARGETS': True,
'BBOX_NORMALIZE_TARGETS_PRECOMPUTED': False,
'BBOX_REG': True,
'BBOX_THRESH': 0.7,
'BG_THRESH_HI': 0.5,
'BG_THRESH_LO': 0.0,
'FG_FRACTION': 0.25,
'FG_THRESH': 0.5,
'HAS_RPN': False,
'IMS_PER_BATCH': 2,
'MAX_SIZE': 520,
'PROPOSAL_METHOD': 'rpn',
'RPN_BATCHSIZE': 256,
'RPN_BBOX_INSIDE_WEIGHTS': [1.0, 1.0, 1.0, 1.0],
'RPN_CLOBBER_POSITIVES': False,
'RPN_FG_FRACTION': 0.5,
'RPN_MIN_SIZE': 0,
'RPN_NEGATIVE_OVERLAP': 0.3,
'RPN_NMS_THRESH': 0.7,
'RPN_POSITIVE_OVERLAP': 0.7,
'RPN_POSITIVE_WEIGHT': -1.0,
'RPN_POST_NMS_TOP_N': 2000,
'RPN_PRE_NMS_TOP_N': 12000,
'SCALES': [375],
'SNAPSHOT_INFIX': '',
'SNAPSHOT_ITERS': 5000,
'USE_FLIPPED': True,
'USE_PREFETCH': False},
'USE_GPU_NMS': True}
Loaded dataset voc_2007_trainval
for training
Set proposal method: rpn
Appending horizontally-flipped training examples...
voc_2007_trainval gt roidb loaded from /home/tanglin/FPN-master/data/cache/voc_2007_trainval_gt_roidb.pkl
loading None
Traceback (most recent call last):
File "./tools/train_net.py", line 104, in
imdb, roidb = combined_roidb(args.imdb_name)
File "./tools/train_net.py", line 69, in combined_roidb
roidbs = [get_roidb(s) for s in imdb_names.split('+')]
File "./tools/train_net.py", line 66, in get_roidb
roidb = get_training_roidb(imdb)
File "/home/tanglin/FPN-master/tools/../lib/fast_rcnn/train.py", line 118, in get_training_roidb
imdb.append_flipped_images()
File "/home/tanglin/FPN-master/tools/../lib/datasets/imdb.py", line 106, in append_flipped_images
boxes = self.roidb[i]['boxes'].copy()
File "/home/tanglin/FPN-master/tools/../lib/datasets/imdb.py", line 67, in roidb
self._roidb = self.roidb_handler()
File "/home/tanglin/FPN-master/tools/../lib/datasets/pascal_voc.py", line 141, in rpn_roidb
rpn_roidb = self._load_rpn_roidb(gt_roidb)
File "/home/tanglin/FPN-master/tools/../lib/datasets/pascal_voc.py", line 151, in _load_rpn_roidb
assert os.path.exists(filename),
File "/usr/lib/python2.7/genericpath.py", line 18, in exists
os.stat(path)
TypeError: coercing to Unicode: need string or buffer, NoneType found
but when i run faster_rcnn_alt_opt.sh,it is ok?
how can i solve it?
@unsky Hi, unsky. I really appreciate the code for FPN. But when I used it I got very bad mAP for my single object detection task on my own dataset. I have changed the parameters in train.prototxt and train_mergercnn.prototxt with changing 21 to 2 (object + background) and 84 to 8. The code could run well and loss also looks fine. So could you please tell me are there other places I need to change parameters to for training on my dataset?
unsky,你好!我在查看rpn网络的时候,发现这五个rpn模块用的都是相同的achor大小,而不是原论文中的那种说法。你是否尝试过每个level只对应一个固定且不同的anchor大小?
训练的时候遇到了这个问题:
/home/AISSD/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py:105: RuntimeWarning: invalid value encountered in sqrt
k = np.floor(k0 + np.log2(np.sqrt(areas)*1.0 / 64))
Traceback (most recent call last):
File "./tools/train_net.py", line 112, in
max_iters=args.max_iters)
File "/home/AISSD/FPN-caffe/tools/../lib/fast_rcnn/train.py", line 162, in train_net
model_paths = sw.train_model(max_iters)
File "/home/AISSD/FPN-caffe/tools/../lib/fast_rcnn/train.py", line 101, in train_model
self.solver.step(1)
File "/home/AISSD/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py", line 117, in forward
if int(item)==p:
ValueError: cannot convert float NaN to integer
输出了一下这个anchor_target_layer.py中的
ws = gt_boxes[:, 2] - gt_boxes[:, 0]
hs = gt_boxes[:, 3] - gt_boxes[:, 1]
areas = ws * hs
#print "debug::",ws,hs
k = np.floor(k0 + np.log2(np.sqrt(areas)*1.0 / 64))
发现hs是负的,然后输出了一下gt_boxes的值:
DEBUG:: [[ 5.97701132e-01 3.91703438e+04 5.20000000e+02 2.70160919e+02 1.00000000e+00]]
但是同样的数据在faster-rcnn中运行没有这个错,请问有谁知道这个问题怎么回事吗?
请问 您这里的caffe-fpn与Ross的py-faster-rcnn使用的caffe什么不同之处?
是只多出了deformable conv的支持吗?
`FPN/caffe-fpn/src/caffe/layer_factory.cpp:80:3: error: ‘DeformableConvolutionParameter’ was not declared in this scope
DeformableConvolutionParameter defor_conv_param = param.deformable_convolution_param();
^
FPN/caffe-fpn/src/caffe/layer_factory.cpp:81:3: error: ‘DeformableConvolutionParameter_Engine’ was not declared in this scope
DeformableConvolutionParameter_Engine engine = defor_conv_param.engine();
^
FPN/caffe-fpn/src/caffe/layer_factory.cpp:83:7: error: ‘engine’ was not declared in this scope
if (engine == DeformableConvolutionParameter_Engine_DEFAULT) {
^
FPN/caffe-fpn/src/caffe/layer_factory.cpp:83:17: error: ‘DeformableConvolutionParameter_Engine_DEFAULT’ was not declared in this scope
if (engine == DeformableConvolutionParameter_Engine_DEFAULT) {
^
FPN/caffe-fpn/src/caffe/layer_factory.cpp:84:14: error: ‘DeformableConvolutionParameter_Engine_CAFFE’ was not declared in this scope
engine = DeformableConvolutionParameter_Engine_CAFFE;
^
FPN/caffe-fpn/src/caffe/layer_factory.cpp:86:7: error: ‘engine’ was not declared in this scope
if (engine == DeformableConvolutionParameter_Engine_CAFFE) {
^
FPN/caffe-fpn/src/caffe/layer_factory.cpp:86:17: error: ‘DeformableConvolutionParameter_Engine_CAFFE’ was not declared in this scope
if (engine == DeformableConvolutionParameter_Engine_CAFFE) {`
@unsky
Hello, thank you for providing the source code. I revised your network with reference to the code you wrote, but when training occurs, loss_bbox = 0. What adjustments do you have in mind? No other methods have been tried. If you make an opinion, thank you very much.
I0314 11:00:26.190989 18580 solver.cpp:229] Iteration 0, loss = 2.42246
I0314 11:00:26.191026 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:00:26.191032 18580 solver.cpp:245] Train net output #1: loss_cls = 3.05018 (* 0.5 = 1.52509 loss)
I0314 11:00:26.191038 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.879062 (* 0.5 = 0.439531 loss)
I0314 11:00:26.191045 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.771223 (* 0.5 = 0.385611 loss)
I0314 11:00:26.191049 18580 sgd_solver.cpp:107] Iteration 0, lr = 0.001
I0314 11:00:34.803771 18580 solver.cpp:229] Iteration 20, loss = 1.20221
I0314 11:00:34.803809 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:00:34.803817 18580 solver.cpp:245] Train net output #1: loss_cls = 1.56594 (* 0.5 = 0.782969 loss)
I0314 11:00:34.803822 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.681606 (* 0.5 = 0.340803 loss)
I0314 11:00:34.803829 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.0196237 (* 0.5 = 0.00981186 loss)
I0314 11:00:34.803836 18580 sgd_solver.cpp:107] Iteration 20, lr = 0.001
I0314 11:00:43.548290 18580 solver.cpp:229] Iteration 40, loss = 1.34465
I0314 11:00:43.548328 18580 solver.cpp:245] Train net output #0: loss_bbox = 4.46253e-06 (* 0.5 = 2.23126e-06 loss)
I0314 11:00:43.548336 18580 solver.cpp:245] Train net output #1: loss_cls = 1.51926 (* 0.5 = 0.75963 loss)
I0314 11:00:43.548341 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.678311 (* 0.5 = 0.339155 loss)
I0314 11:00:43.548346 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.0992929 (* 0.5 = 0.0496464 loss)
I0314 11:00:43.548352 18580 sgd_solver.cpp:107] Iteration 40, lr = 0.001
I0314 11:00:52.411149 18580 solver.cpp:229] Iteration 60, loss = 1.19288
I0314 11:00:52.411186 18580 solver.cpp:245] Train net output #0: loss_bbox = 3.54063e-06 (* 0.5 = 1.77032e-06 loss)
I0314 11:00:52.411193 18580 solver.cpp:245] Train net output #1: loss_cls = 1.48073 (* 0.5 = 0.740365 loss)
I0314 11:00:52.411198 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.694064 (* 0.5 = 0.347032 loss)
I0314 11:00:52.411204 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.450236 (* 0.5 = 0.225118 loss)
I0314 11:00:52.411209 18580 sgd_solver.cpp:107] Iteration 60, lr = 0.001
I0314 11:01:01.210149 18580 solver.cpp:229] Iteration 80, loss = 1.16331
I0314 11:01:01.210186 18580 solver.cpp:245] Train net output #0: loss_bbox = 2.98784e-06 (* 0.5 = 1.49392e-06 loss)
I0314 11:01:01.210193 18580 solver.cpp:245] Train net output #1: loss_cls = 1.58441 (* 0.5 = 0.792203 loss)
I0314 11:01:01.210199 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.758525 (* 0.5 = 0.379262 loss)
I0314 11:01:01.210206 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.193615 (* 0.5 = 0.0968074 loss)
I0314 11:01:01.210211 18580 sgd_solver.cpp:107] Iteration 80, lr = 0.001
I0314 11:01:09.967835 18580 solver.cpp:229] Iteration 100, loss = 1.2145
I0314 11:01:09.967872 18580 solver.cpp:245] Train net output #0: loss_bbox = 1.69471e-06 (* 0.5 = 8.47353e-07 loss)
I0314 11:01:09.967878 18580 solver.cpp:245] Train net output #1: loss_cls = 1.47326 (* 0.5 = 0.736631 loss)
I0314 11:01:09.967885 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.752394 (* 0.5 = 0.376197 loss)
I0314 11:01:09.967890 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.281891 (* 0.5 = 0.140946 loss)
I0314 11:01:09.967895 18580 sgd_solver.cpp:107] Iteration 100, lr = 0.001
I0314 11:01:18.736949 18580 solver.cpp:229] Iteration 120, loss = 0.995279
I0314 11:01:18.736986 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:01:18.736992 18580 solver.cpp:245] Train net output #1: loss_cls = 1.38187 (* 0.5 = 0.690937 loss)
I0314 11:01:18.736999 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.556746 (* 0.5 = 0.278373 loss)
I0314 11:01:18.737004 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.0684626 (* 0.5 = 0.0342313 loss)
I0314 11:01:18.737010 18580 sgd_solver.cpp:107] Iteration 120, lr = 0.001
I0314 11:01:27.601275 18580 solver.cpp:229] Iteration 140, loss = 1.00772
I0314 11:01:27.601315 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:01:27.601321 18580 solver.cpp:245] Train net output #1: loss_cls = 1.34108 (* 0.5 = 0.670541 loss)
I0314 11:01:27.601327 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.575482 (* 0.5 = 0.287741 loss)
I0314 11:01:27.601333 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.134016 (* 0.5 = 0.067008 loss)
I0314 11:01:27.601338 18580 sgd_solver.cpp:107] Iteration 140, lr = 0.001
I0314 11:01:36.287109 18580 solver.cpp:229] Iteration 160, loss = 1.02634
I0314 11:01:36.287149 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:01:36.287156 18580 solver.cpp:245] Train net output #1: loss_cls = 1.29888 (* 0.5 = 0.649439 loss)
I0314 11:01:36.287163 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.508832 (* 0.5 = 0.254416 loss)
I0314 11:01:36.287168 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.225303 (* 0.5 = 0.112652 loss)
I0314 11:01:36.287173 18580 sgd_solver.cpp:107] Iteration 160, lr = 0.001
I0314 11:01:45.057557 18580 solver.cpp:229] Iteration 180, loss = 1.31192
I0314 11:01:45.057595 18580 solver.cpp:245] Train net output #0: loss_bbox = 2.033e-05 (* 0.5 = 1.0165e-05 loss)
I0314 11:01:45.057602 18580 solver.cpp:245] Train net output #1: loss_cls = 1.31927 (* 0.5 = 0.659635 loss)
I0314 11:01:45.057608 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.728466 (* 0.5 = 0.364233 loss)
I0314 11:01:45.057615 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.872437 (* 0.5 = 0.436219 loss)
I0314 11:01:45.057620 18580 sgd_solver.cpp:107] Iteration 180, lr = 0.001
speed: 0.439s / iter
I0314 11:01:53.781239 18580 solver.cpp:229] Iteration 200, loss = 0.924464
I0314 11:01:53.781275 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:01:53.781281 18580 solver.cpp:245] Train net output #1: loss_cls = 1.2256 (* 0.5 = 0.612798 loss)
I0314 11:01:53.781287 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.470908 (* 0.5 = 0.235454 loss)
I0314 11:01:53.781293 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.0965142 (* 0.5 = 0.0482571 loss)
I0314 11:01:53.781298 18580 sgd_solver.cpp:107] Iteration 200, lr = 0.001
I0314 11:02:02.579058 18580 solver.cpp:229] Iteration 220, loss = 0.921759
I0314 11:02:02.579099 18580 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 0.5 = 0 loss)
I0314 11:02:02.579108 18580 solver.cpp:245] Train net output #1: loss_cls = 1.21418 (* 0.5 = 0.607091 loss)
I0314 11:02:02.579118 18580 solver.cpp:245] Train net output #2: rpn_cls_loss = 0.503071 (* 0.5 = 0.251536 loss)
I0314 11:02:02.579125 18580 solver.cpp:245] Train net output #3: rpn_loss_bbox = 0.0527053 (* 0.5 = 0.0263527 loss)
And with the increase of the number of training, a mistake was terminated in the back.
/lib/rpn/proposal_layer.py:248: RuntimeWarning: invalid value encountered in greater_equal
keep = np.where((ws >= min_size) & (hs >= min_size))[0]
Traceback (most recent call last):
File "./tools/train_net.py", line 112, in
max_iters=args.max_iters)
File "/home/sins/py-faster-rcnn/tools/../lib/fast_rcnn/train.py", line 161, in train_net
model_paths = sw.train_model(max_iters)
File "/home/sins/py-faster-rcnn/tools/../lib/fast_rcnn/train.py", line 102, in train_model
self.solver.step(1)
File "/home/sins/py-faster-rcnn/tools/../lib/rpn/proposal_layer.py", line 203, in forward
pad = npr.choice(keep, size=post_nms_topN - len(keep))
File "mtrand.pyx", line 1126, in mtrand.RandomState.choice
Thanks very much!!!!!
Hi,
Is there any reason for adding a group parameter for deconvolution?
I1220 22:57:08.836243 26285 solver.cpp:229] Iteration 13940, loss = 6.60338
I1220 22:57:08.836273 26285 solver.cpp:245] Train net output #0: loss_bbox = 0 (* 1 = 0 loss)
I1220 22:57:08.836279 26285 solver.cpp:245] Train net output #1: loss_cls = 0.0678312 (* 1 = 0.0678312 loss)
I1220 22:57:08.836283 26285 solver.cpp:245] Train net output #2: rpn_cls_loss/p2 = 0.366679 (* 1 = 0.366679 loss)
I1220 22:57:08.836287 26285 solver.cpp:245] Train net output #3: rpn_cls_loss/p3 = 0.195475 (* 1 = 0.195475 loss)
I1220 22:57:08.836292 26285 solver.cpp:245] Train net output #4: rpn_cls_loss/p4 = 0.513616 (* 1 = 0.513616 loss)
I1220 22:57:08.836297 26285 solver.cpp:245] Train net output #5: rpn_cls_loss/p5 = 0.559214 (* 1 = 0.559214 loss)
I1220 22:57:08.836309 26285 solver.cpp:245] Train net output #6: rpn_cls_loss/p6 = 0.303023 (* 1 = 0.303023 loss)
I1220 22:57:08.836313 26285 solver.cpp:245] Train net output #7: rpn_loss_bbox/p2 = 3.02821 (* 1 = 3.02821 loss)
I1220 22:57:08.836316 26285 solver.cpp:245] Train net output #8: rpn_loss_bbox/p3 = 1.88581 (* 1 = 1.88581 loss)
I1220 22:57:08.836330 26285 solver.cpp:245] Train net output #9: rpn_loss_bbox/p4 = 0.740322 (* 1 = 0.740322 loss)
I1220 22:57:08.836334 26285 solver.cpp:245] Train net output #10: rpn_loss_bbox/p5 = 0.256244 (* 1 = 0.256244 loss)
I1220 22:57:08.836349 26285 solver.cpp:245] Train net output #11: rpn_loss_bbox/p6 = 0.278417 (* 1 = 0.278417 loss)
Traceback (most recent call last):
File "./tools/train_net.py", line 104, in
imdb, roidb = combined_roidb(args.imdb_name)
File "./tools/train_net.py", line 69, in combined_roidb
roidbs = [get_roidb(s) for s in imdb_names.split('+')]
File "./tools/train_net.py", line 62, in get_roidb
imdb = get_imdb(imdb_name)
File "/home/t91/pengyulong/workspace/FPN/tools/../lib/datasets/factory.py", line 37, in get_imdb
raise KeyError('Unknown dataset: {}'.format(name))
KeyError: 'Unknown dataset: /home/t91/data/lmdb/train_lmdb'
when i training ,i have a problem :
Traceback (most recent call last):
File "/home/hncs/liuwei/FPN1/tools/../lib/rpn/proposal_layer.py", line 14, in
from fast_rcnn.nms_wrapper import nms
File "/home/hncs/liuwei/FPN1/tools/../lib/fast_rcnn/nms_wrapper.py", line 9, in
from nms.gpu_nms import gpu_nms
ImportError: No module named gpu_nms
Traceback (most recent call last):
File "./tools/train_net.py", line 113, in
max_iters=args.max_iters)
File "/home/hncs/liuwei/FPN1/tools/../lib/fast_rcnn/train.py", line 143, in train_net
pretrained_model=pretrained_model)
File "/home/hncs/liuwei/FPN1/tools/../lib/fast_rcnn/train.py", line 46, in init
self.solver = caffe.SGDSolver(solver_prototxt)
SystemError: NULL result without error in PyObject_Call
But i running on the GPU,and the id is right, i don't know what's wrong.
thank you very much!
@unsky
请问在build/lib里make 提示这个错误怎么办?
@unsky Hi, unsky. I tried to use your code to train FPN network for my dataset and I found an error after changing two parameters in fast_rcnn/config.py to change longest and shortest image side that:
__C.TRAIN.SCALES = (1280,) (the origin is 768)
__C.TRAIN.MAX_SIZE = 2000 (the origin is 1280)
the error part happens here:
I0125 09:46:02.151782 18521 net.cpp:106] Creating Layer p4
I0125 09:46:02.151787 18521 net.cpp:454] p4 <- c4
I0125 09:46:02.151793 18521 net.cpp:454] p4 <- upP5
I0125 09:46:02.151796 18521 net.cpp:411] p4 -> p4
F0125 09:46:02.151804 18521 eltwise_layer.cpp:34] Check failed: bottom[i]->shape() == bottom[0]->shape()
*** Check failure stack trace: ***
./experiments/scripts/FP_Net_end2end.sh: 行 57: 18521 已放弃 (核心已转储) ./tools/train_net.py --gpu ${GPU_ID} --solver models/${PT_DIR}/${NET}/FP_Net_end2end/solver.prototxt --weights data/pretrained_model/ResNet50.v2.caffemodel --imdb ${TRAIN_IMDB} --iters ${ITERS} --cfg experiments/cfgs/FP_Net_end2end.yml ${EXTRA_ARGS}
And if I change back the code could run well. could you please help me deal with it?
Thank you very much!
1.According to the paper,each rpn should have 3 different anchors(1 erea * 3 aspect ratios) because each levle has their own erea. But you still set 9 anchors (3 erea * 3 aspect ratios)at each level of rpn
2.Have you tried to share rpn weights and bias? It will save time and memory, and might improve the result
First of all, Thank you for sharing FPN-caffe version
I have some simple questions.
1)
You apply "as_rois layer" in model/test.prototxt. However, It seems that you did not apply "as_rois_layer" in model/train.prototxt
why this?
2)
fc layers are weight sharing for this version. however, this is time consuming because when fc/h2 is done and then fc/h3 starts....(fc/h2 , fc/h3, fc/h4, fc/h5 each is done and merge for socre & bounding box)
I think it is better gathering roi_pooling results (roi_pool/h2, h3, h4, h5) and forward fc layers. (i.e. merge ->fc // your version : fc2,3,4,5 -> merge)
this is better when training & testing...
Thank you!
Hi @unsky
I train the FPN-caffe in voc dataset, and the AP is 59%. I test the model in our video and the result is poor. Do you test the AP in voc dataset? Do you test the model in other video?
I1103 18:47:46.824345 300 layer_factory.hpp:77] Creating layer loss_cls
F1103 18:47:46.824635 300 loss_layer.cpp:19] Check failed: bottom[0]->num() == bottom[1]->num() (4 vs. 1) The data and label should have the same number.
In your train.prototxt, after roi-data layer, you concat(axis=0) 4 roi_pool, then the num is 4, but the label num is 1, how to solve it?
I run it on my caffe, with your whole rpn files.
THANKS!!
Has anyone done experiments.how about the effect of small objects decetion?
Hi unsky:
when I run ./experiments/scripts/FP_Net_end2end.sh, I got a problem as follow:
Traceback (most recent call last):
File "./tools/train_net.py", line 112, in
max_iters=args.max_iters)
File "/home/qi/Desktop/FPN-caffe-master/tools/../lib/fast_rcnn/train.py", line 157, in train_net
pretrained_model=pretrained_model)
File "/home/qi/Desktop/FPN-caffe-master/tools/../lib/fast_rcnn/train.py", line 43, in init
self.solver = caffe.SGDSolver(solver_prototxt)
File "/home/qi/Desktop/FPN-caffe-master/tools/../lib/roi_data_layer/layer.py", line 125, in setup
top[idx].reshape(1, self._num_classes * 4)
IndexError: Index out of range
And the pretrained_model I used is VGG16.v2.caffemodel, hope you can give me some help, Thanks
I want to use VGG to implement FPN, but I have got a bad mAP( around 60 ). Is it a bad choice to use VGG as the backbone?
i set the batch size si 320 ,but the memory of gpu is 4417MiB .
+------------------------------------------------------+
| NVIDIA-SMI 352.39 Driver Version: 352.39 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX TIT... Off | 0000:02:00.0 Off | N/A |
| 22% 38C P8 15W / 250W | 154MiB / 12287MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
| 1 GeForce GTX TIT... Off | 0000:03:00.0 On | N/A |
| 30% 71C P2 97W / 250W | 4481MiB / 12279MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 20213 C python 128MiB |
| 1 8878 G /usr/bin/X 36MiB |
| 1 20213 C python 4417MiB |
I want to kown ,is it normal?
hi ,i'm a green hand, i can't understand i run the
sudo./experiments/scripts/FP_Net_end2end.sh 1 FPN pascal_voc,
but:
bash: sudo./experiments/scripts/FP_Net_end2end.sh: No such file or directory.
do you have a detailed process thank you very much!
Hello, unsky;
would you please share your python scripts of generating your faster_rcnn_resnet50_fpn or give me some advice of your lib/rpn? I want to reimplement faster_rcnn_resnet100_fpn based on your work.
thanks very much.
I have met the error as follow when using FPN to train my own dataset:
Loading pretrained model weights from data/pretrained_model/ResNet50.v2.caffemodel
I0126 16:08:01.051177 10326 net.cpp:816] Ignoring source layer pool5
I0126 16:08:01.051213 10326 net.cpp:816] Ignoring source layer fc1000
I0126 16:08:01.051214 10326 net.cpp:816] Ignoring source layer prob
Solving...
F0126 16:08:02.152665 10326 syncedmem.cpp:56] Check failed: error == cudaSuccess (2 vs. 0) out of memory
*** Check failure stack trace: ***
./experiments/scripts/FP_Net_end2end.sh: 行 57: 10326 已放弃 (核心已转储) ./tools/train_net.py --gpu ${GPU_ID} --solver models/${PT_DIR}/${NET}/FP_Net_end2end/solver.prototxt --weights data/pretrained_model/ResNet50.v2.caffemodel --imdb ${TRAIN_IMDB} --iters ${ITERS} --cfg experiments/cfgs/FP_Net_end2end.yml ${EXTRA_ARGS}
I tried to reduce __C.TRAIN.IMS_PER_BATCH and __C.TRAIN.BATCH_SIZE in lib/fast_rcnn/config.py, but it doesn't work. How to reduce the batch_size to reduce the cost of memory?
I0103 17:59:46.463167 29140 layer_factory.hpp:77] Creating layer fpn_loss_cls
I0103 17:59:46.463174 29140 net.cpp:106] Creating Layer fpn_loss_cls
I0103 17:59:46.463178 29140 net.cpp:454] fpn_loss_cls <- rpn_cls_score_reshape
I0103 17:59:46.463183 29140 net.cpp:454] fpn_loss_cls <- rpn_labels
I0103 17:59:46.463191 29140 net.cpp:411] fpn_loss_cls -> FPNClsLoss
I0103 17:59:46.463199 29140 layer_factory.hpp:77] Creating layer fpn_loss_cls
F0103 17:59:46.465251 29140 softmax_loss_layer.cpp:47] Check failed: outer_num_ * inner_num_ == bottom[1]->count() (245520 vs. 552960) Number of labels must match number of predictions; e.g., if softmax axis == 1 and prediction shape is (N, C, H, W), label count (number of labels) must be NHW, with integer values in {0, 1, ..., C-1}.
*** Check failure stack trace: ***
Thank you
Why you add stride to pad the input image?
https://github.com/unsky/FPN/blob/master/lib/utils/blob.py#L47
Hi, I add the fpn to ZF and i train the pasalvoc datasets. The loss is high, can you give me some advice?
ps: I just add two feature pyramid layer p4,p3 to ZF
Is that OK to sharing your final model? it may cannot achieve this mAP because of the equipment.
I1114 21:43:12.214639 28872 sgd_solver.cpp:106] Iteration 0, lr = 0.0002
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/proposal_target_layer.py:225: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
fg_inds = npr.choice(fg_inds, size=fg_rois_per_this_image, replace=False)
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/proposal_target_layer.py:236: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
bg_inds = npr.choice(bg_inds, size=bg_rois_per_this_image, replace=False)
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/proposal_target_layer.py:243: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future
labels[fg_rois_per_this_image:] = 0
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py:104: RuntimeWarning: invalid value encountered in sqrt
k = np.floor(k0 + np.log2(np.sqrt(areas)*1.0 / 64))
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py:106: RuntimeWarning: invalid value encountered in less
inds = np.where(k < min_k)[0]
/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py:108: RuntimeWarning: invalid value encountered in greater
inds = np.where(k > max_k)[0]
Traceback (most recent call last):
File "./tools/train_net.py", line 112, in
max_iters=args.max_iters)
File "/home/gavin/icpr/FPN-caffe/tools/../lib/fast_rcnn/train.py", line 160, in train_net
model_paths = sw.train_model(max_iters)
File "/home/gavin/icpr/FPN-caffe/tools/../lib/fast_rcnn/train.py", line 101, in train_model
self.solver.step(1)
File "/home/gavin/icpr/FPN-caffe/tools/../lib/rpn/anchor_target_layer.py", line 116, in forward
if int(item)==p:
ValueError: cannot convert float NaN to integer
Command exited with non-zero status 1
Hello, sir. I met an error in the test phase like this:
"
Traceback (most recent call last):
File "./tools/test_net.py", line 90, in
test_net(net, imdb, max_per_image=args.max_per_image, vis=args.vis)
File "/home/gaokun/FPN-caffe/tools/../lib/fast_rcnn/test.py", line 268, in test_net
scores, boxes = im_detect(net, im, box_proposals)
File "/home/gaokun/FPN-caffe/tools/../lib/fast_rcnn/test.py", line 183, in im_detect
pred_boxes = bbox_transform_inv(boxes, box_deltas)
File "/home/gaokun/FPN-caffe/tools/../lib/fast_rcnn/bbox_transform.py", line 47, in bbox_transform_inv
pred_ctr_x = dx * widths[:, np.newaxis] + ctr_x[:, np.newaxis]
ValueError: operands could not be broadcast together with shapes (225,2) (300,1)
"
I don't know how to solve it. can you give me some advice ?
Thank you!
@unsky
hi,when I train FPN on my own dataset,I met error:
I0312 16:25:25.883342 2983 sgd_solver.cpp:106] Iteration 0, lr = 0.0005
/home/zq/FPN/tools/../lib/rpn/proposal_layer.py:175: RuntimeWarning: invalid value encountered in greater_equal
keep = np.where((ws >= min_size) & (hs >= min_size))[0]
Floating point exception (core dumped)
I try to change lr from 0.001 to 0.0001,but it didn't work.I also change RNG_SEED,and it also didn't work.
I don't know how to solve it.please help me,thanks so much!
FPN是一个多层预测框架,每一个分支都要设置对应的anchors.那么在不同层设置anchors,设计原则是什么?
还有您README中的about the anchor size setting一部分,能不能解释一下.
麻烦了!!
@unsky
你好,代码写的很好,不过我在运行时出现了一个问题,可能是由于我的gpu内存(1050ti,4g)不足导致Initializing net parameters时出现错误,下面是出错的内容:
zq@zq-System-Product-Name:~/FPN-master$
./experiments/scripts/FP_Net_end2end_merge_rcnn.sh 0 FPN pascal_voc
......
I0308 20:29:01.180505 2233 net.cpp:49] Initializing net from parameters:
.....
layer {
name: "res4b6"
type: "Eltwise"
bottom: "res4b5"
bottom: "res4b6_branch2c"
top: "res4b6"
}
layer {
name: "res4b6_relu"
type: "ReLU"
bottom: "res4b6"
top: "res4b6"
}
layer {
name: "res4b7_branch2a"
type: "Convolution"
bottom: "res4b6"
top: "res4b7_branch2a"
param {
lr_mult: 1
}
convolution_param {
num_output: 256
bias_term: false
pad: 0
kernel_size: 1
stride: 1
}
}
layer {
name: "scale4b7_branch2a"
type: "Scale"
bottom: "res4b7_branch2a"
top: "res4b7_branch2a"
param {
lr_mult: 0
decay_mult: 0
}
param {
lr_mult: 0
decay_mult: 0
}
scale_param {
bias_term: true
}
}
layer {
name: "res4b7_branch2a_relu"
type: "ReLU"
bottom: "res4b7_branch2a"
top: "res4b7_branch2a"
}
layer {
name: "res4b7_branch2b"
type: "Convolution"
bottom: "res4b7_branch2a"
top: "res4b7_branch2b"
param {
lr_mult: 1
}
convolution_param {
num_output: 256
bias_term: false
pad: 1
kernel_size: 3
stride: 1
}
}
layer {
name: "scale4b7_branch2b"
type: "Scale"
bottom: "res4b7_branch2b"
top: "res4b7_branch2b"
param {
lr_mult: 0
decay_mult: 0
}
param {
lr_mult: 0
decay_mult: 0
}
scale_param {
bias_term: true
}
}
layer {
name: "res4b7_branch2b_relu"
type: "ReLU"
bottom: "res4b7_branch2b"
top: "res4b7_branch2b"
}
layer {
name: "res4b7_branch2c"
type: "Convolution"
bottom: "res4b7_branch2b"
top: "res4b7_branch2c"
param {
lr_mult: 1
}
convolution_param {
num_output: 1024
bias_term: false
pad: 0
kernel_size: 1
stride: 1
}
}
layer {
name: "scale4b7
I0308 20:29:01.184592 2233 layer_factory.hpp:77] Creating layer input-data
F0308 20:29:01.184634 2233 layer_factory.hpp:81] Check failed: registry.count(type) == 1 (0 vs. 1) Unknown layer type: Python (known types: AbsVal, Accuracy, ArgMax, BNLL, BatchNorm, BatchReindex, Bias, Concat, ContrastiveLoss, Convolution, Data, Deconvolution, Dropout, DummyData, ELU, Eltwise, Embed, EuclideanLoss, Exp, Filter, Flatten, HDF5Data, HDF5Output, HingeLoss, Im2col, ImageData, InfogainLoss, InnerProduct, LRN, Log, MVN, MemoryData, MultinomialLogisticLoss, PReLU, Pooling, Power, ROIPooling, ReLU, Reduction, Reshape, SPP, Scale, Sigmoid, SigmoidCrossEntropyLoss, Silence, Slice, SmoothL1Loss, Softmax, SoftmaxWithLoss, Split, TanH, Threshold, Tile, WindowData)
*** Check failure stack trace: ***
Aborted (core dumped)
你之前在readme 里写过:
“In my expriments, the codes require ~10G GPU memory in training and ~6G in testing. your can design the suit image size, mimbatch size and rcnn batch size for your GPUS.”
但是我的gpu只有4G,而且我不知道要修改的这些batch size在哪些地方,你能告诉我应该怎样修改吗?
非常感谢!!!
单张1080ti上跑训练会报错误:
syncedmem.cpp:56] Check failed: error == cudaSuccess (2 vs. 0) out of memory
我在solver文件把iter_size改为1,还是报一样的错。
是显存的不够的原因吗?
I add your rpn operator to my caffe to train fpn, I set batch_size as 2, base_lr to 0.0001 to 0.001, the loss reduce from larger than 3 to smaller than 1 after hundreds of iterations, then the loss became to Nan. Can you give me some advises? And serveral warnings occured during training progress, shown as below.
i'm a green hand, do you have a detailed process?what can can i should do use my dataset? thank you very much.
Could you share the trained model?
Thank you very much
在源码的RPN中proposal_target_layer.py里
if sample_type == 'fpn':
#print 0
w = (rois[:,3]-rois[:,1])
h = (rois[:,4]-rois[:,2])
s = w * h
s[s<=0]=1e-6
layer_index = np.floor(k0+np.log2(np.sqrt(s)/224))
layer_index[layer_index<2]=2
layer_index[layer_index>5]=5
这部分中的layer_index是怎样计算的呀?有些许不懂,希望有人能够解答,非常感谢
when i run ./experiments/scripts/FP_Net_end2end.sh 1 FPN pascal_voc
mean mAP = 0.719 ,not 0.788 i don't know why. Please help me.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.