stick-to/object-detection-tensorflow

Object Detection API Tensorflow

object-detection-tensorflow's Issues

FCOS train loss NAN

hi, I train FCOS and Centernet use same dataset, Centernet sames ok while FCOS loss is NAN at the early beginning. I tried to use gradient clip,but the loss would keep high.

i retrain centernet,and i got the loss is Nan

i retrain centernet on new dataset,when i training to about 10 epochs,i got loss is Nan.and then ,
I training at the ninth epoch and loss returned to normal.but when I train two more epochs,
the loss become Nan again.can you help me?

Invalid argument: Input to reshape is a tensor with 6220800 values, but the requested shape has 0

Hello, thank you for replicating so many networks. It's really a great job. This problem occurred when I ran the program. I don't know what went wrong.

Traceback (most recent call last):
File "testfcos.py", line 67, in
mean_loss = fcos.train_one_epoch(lr)
File "/home/training/zcc/Object-Detection-API-Tensorflow-master/FCOS.py", line 407, in train_one_epoch
_, loss = self.sess.run([self.train_op, self.loss], feed_dict={self.lr: lr})
File "/home/training/anaconda2/envs/zcc/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 950, in run
run_metadata_ptr)
File "/home/training/anaconda2/envs/zcc/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)
File "/home/training/anaconda2/envs/zcc/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
run_metadata)
File "/home/training/anaconda2/envs/zcc/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: Input to reshape is a tensor with 6220800 values, but the requested shape has 0
[[{{node Reshape_2}}]]
[[IteratorGetNext]]
[[IteratorGetNext/_5]]
(1) Invalid argument: Input to reshape is a tensor with 6220800 values, but the requested shape has 0
[[{{node Reshape_2}}]]
[[IteratorGetNext]]
0 successful operations.
0 derived errors ignored.

would you mind sharing your pretrained model like yolov3 FCOS CenterNet?

请问，怎么用其他的预训练模型，如resnet101之类的

关于数据集格式的请教

作者用的应该是VOC2007数据集吧，我找到了那个数据集，打开里面有包括ImagineSets的若干文件夹，请问作者在项目中的voc2007文件夹里面是什么样子的？

tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [9,1] vs. shape[2] = [8,1] testutils genered corrupted tf records

Traceback (most recent call last):
  File "/home/vkhvorostianyi/anaconda3/envs/refinedet_tf/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
    return fn(*args)
  File "/home/vkhvorostianyi/anaconda3/envs/refinedet_tf/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/home/vkhvorostianyi/anaconda3/envs/refinedet_tf/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [9,1] vs. shape[2] = [8,1]
         [[{{node concat_1}}]]
         [[{{node IteratorGetNext}}]]
         [[{{node IteratorGetNext}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "testrefinedet.py", line 68, in <module>
    mean_loss = refinedet.train_one_epoch(lr)
  File "/home/vkhvorostianyi/Object-Detection-API-Tensorflow/RefineDet.py", line 587, in train_one_epoch
    _, loss = self.sess.run([self.train_op, self.loss], feed_dict={self.lr: lr})
  File "/home/vkhvorostianyi/anaconda3/envs/refinedet_tf/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 929, in run
    run_metadata_ptr)
  File "/home/vkhvorostianyi/anaconda3/envs/refinedet_tf/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1152, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/vkhvorostianyi/anaconda3/envs/refinedet_tf/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1328, in _do_run
    run_metadata)
  File "/home/vkhvorostianyi/anaconda3/envs/refinedet_tf/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1348, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [9,1] vs. shape[2] = [8,1]
         [[{{node concat_1}}]]
         [[node IteratorGetNext (defined at /home/vkhvorostianyi/Object-Detection-API-Tensorflow/RefineDet.py:63) ]]
         [[node IteratorGetNext (defined at /home/vkhvorostianyi/Object-Detection-API-Tensorflow/RefineDet.py:63) ]]

The problem is address to code for VOC dataset, dunno about the rest

how to test in batch?(请问如何进行批测试)

感谢大佬整理的代码，只是在测试的时候只能一张一张地来，如果要批测试如何改？对tf1.x不怎么熟，恳请赐教。

typo

there may be a typo in line 187 in utils/image_augmentor.py.

How to train the gray images with one channel

I want to train the gray images. I ran the code and had the error. Can this code train the gray images? Please help! Thanks

------------------------- epoch 0 -------------------------
reduce lr, lr= 0.0001 now

---

InvalidArgumentError Traceback (most recent call last)
~\Anaconda3\envs\tensor-gpu\lib\site-packages\tensorflow\python\client\session.py in _do_call(self, fn, *args)
1333 try:
-> 1334 return fn(*args)
1335 except errors.OpError as e:

~\Anaconda3\envs\tensor-gpu\lib\site-packages\tensorflow\python\client\session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata)
1318 return self._call_tf_sessionrun(
-> 1319 options, feed_dict, fetch_list, target_list, run_metadata)
1320

~\Anaconda3\envs\tensor-gpu\lib\site-packages\tensorflow\python\client\session.py in _call_tf_sessionrun(self, options, feed_dict, fetch_list, target_list, run_metadata)
1406 self._session, options, feed_dict, fetch_list, target_list,
-> 1407 run_metadata)
1408

InvalidArgumentError: input must have 3 channels but instead has 1 channels.
[[{{node cond_6/adjust_hue/AdjustHue}}]]
[[{{node IteratorGetNext}}]]
[[{{node center_detector/cond_9/SparseToDense}}]]

During handling of the above exception, another exception occurred:

InvalidArgumentError Traceback (most recent call last)
D:\jupyter\Object-Detection-API-Tensorflow-master\testcenternet.py in
72 lr = lr/10.
73 print('reduce lr, lr=', lr, 'now')
---> 74 mean_loss = centernet.train_one_epoch(lr)
75 print('>> mean loss', mean_loss)
76 centernet.save_weight('latest', './centernet/test') # 'latest', 'best

D:\jupyter\Object-Detection-API-Tensorflow-master\CenterNet.py in train_one_epoch(self, lr)
293 num_iters = self.num_train // self.batch_size
294 for i in range(num_iters):
--> 295 _, loss = self.sess.run([self.train_op, self.loss], feed_dict={self.lr: lr})
296 sys.stdout.write('\r>> ' + 'iters '+str(i+1)+str('/')+str(num_iters)+' loss '+str(loss))
297 sys.stdout.flush()

~\Anaconda3\envs\tensor-gpu\lib\site-packages\tensorflow\python\client\session.py in run(self, fetches, feed_dict, options, run_metadata)
927 try:
928 result = self._run(None, fetches, feed_dict, options_ptr,
--> 929 run_metadata_ptr)
930 if run_metadata:
931 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)

~\Anaconda3\envs\tensor-gpu\lib\site-packages\tensorflow\python\client\session.py in _run(self, handle, fetches, feed_dict, options, run_metadata)
1150 if final_fetches or final_targets or (handle and feed_dict_tensor):
1151 results = self._do_run(handle, final_targets, final_fetches,
-> 1152 feed_dict_tensor, options, run_metadata)
1153 else:
1154 results = []

~\Anaconda3\envs\tensor-gpu\lib\site-packages\tensorflow\python\client\session.py in _do_run(self, handle, target_list, fetch_list, feed_dict, options, run_metadata)
1326 if handle is None:
1327 return self._do_call(_run_fn, feeds, fetches, targets, options,
-> 1328 run_metadata)
1329 else:
1330 return self._do_call(_prun_fn, handle, feeds, fetches)

~\Anaconda3\envs\tensor-gpu\lib\site-packages\tensorflow\python\client\session.py in _do_call(self, fn, *args)
1346 pass
1347 message = error_interpolation.interpolate(message, self._graph)
-> 1348 raise type(e)(node_def, op, message)
1349
1350 def _extend_graph(self):

InvalidArgumentError: input must have 3 channels but instead has 1 channels.
[[{{node cond_6/adjust_hue/AdjustHue}}]]
[[node IteratorGetNext (defined at D:\jupyter\Object-Detection-API-Tensorflow-master\CenterNet.py:63) ]]
[[node center_detector/cond_9/SparseToDense (defined at D:\jupyter\Object-Detection-API-Tensorflow-master\CenterNet.py:242) ]]

HELP

when i use your centernet for my dataset(6006003), the testcenternet throw out the problem: ValueError: Dimensions must be equal, but are 75 and 76 for 'upsampling/add_1' (op: 'Add') with input shapes: [8,75,75,256], [8,76,76,256]
can you help? thanks!

pretrained weight issue

I try to reproduce FCOS paper result by use your code . After train and test , found the result not very good. And I found the backbone did not use the pretrained weights, like resnet_v1_50.ckpt. Does the bad result related with this ?

Could you help me to freeze the trained models?

I want to deploy the trained models in C+++, how to freeze them, could you give me some hints?
Thanks,
jack

This error occurs when I run refinedet.py,

This error occurs when I run refinedet.py,
File "/home/lijianfeng/Object-Detection-API-Tensorflow/RefineDet.py", line 431, in _compute_one_image_loss
gbbox_hwti = tf.tile(gbbox_hwti, [1, ashape[1], 1])
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 8430, in tile
"Tile", input=input, multiples=multiples, name=name)
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 513, in _apply_op_helper
raise err
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 510, in _apply_op_helper
preferred_dtype=default_dtype)
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1104, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 235, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 214, in constant
value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 432, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 340, in _AssertCompatible
raise TypeError("List of Tensors when single Tensor expected")
TypeError: List of Tensors when single Tensor expected
@1615070057
Author
Thanks

Reshape error on own dataset

Train CenterNet on own dataset. The dataset is converted to tfrecord format with tfrecord_voc_utils.py.

But I have got the following errors. It seems that the error is from reshape().
I need your help. @Stick-To
`Traceback (most recent call last):
File "E:/AI/project/GitHub/CenterNet_ObjectAsPoints/CenterNet-tensorflow-master/train.py", line 76, in
mean_loss = centernet.train_one_epoch(lr)

File "E:\AI\project\GitHub\CenterNet_ObjectAsPoints\CenterNet-tensorflow-master\CenterNet.py", line 295, in train_one_epoch

_, loss = self.sess.run([self.train_op, self.loss], feed_dict={self.lr: lr})

File "E:\AI\project\GitHub\CenterNet_ObjectAsPoints\CenterNet-tensorflow-master\venv\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
run_metadata_ptr)

File "E:\AI\project\GitHub\CenterNet_ObjectAsPoints\CenterNet-tensorflow-master\venv\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)

File "E:\AI\project\GitHub\CenterNet_ObjectAsPoints\CenterNet-tensorflow-master\venv\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run
run_metadata)

File "E:\AI\project\GitHub\CenterNet_ObjectAsPoints\CenterNet-tensorflow-master\venv\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)

tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 786432 values, but the requested shape has 262144

 [[{{node Reshape_2}}]]
 [[node IteratorGetNext (defined at E:\AI\project\GitHub\CenterNet_ObjectAsPoints\CenterNet-tensorflow-master\CenterNet.py:63) ]]`

can you provide pretrain weights for centernet?

I want to retrain centernet on the new dataset(small).
Because if I train the model with coco data, it will take too much time.
so ,can you provide pretrain weights(coco) for centernet? I'll be very grateful.

i don't find test-model.py, please tell where about it, thank you

coordinate

you seem don't normalize the coordinate into 1.

[refineDet] i did the retrain based on the test.py, but the loss is aroud 4.9. and the tp is 0.....

could anybody share your pretrained weight?
thanks alot

checkpoint

您好，如果训练意外中断，如何加载checkpoint

more specific steps

I already have the voc data. How should I
'fill in utils.voc_classname_encoder.py' and 'run utils.test_voc_utils.py'?

loss nan

请问下，我在训练centernet的时候，设置lr为0.0001，batch_size为4，第一个epoch时运行到三分之一不到就显示 loss nan，何故？先谢过！

anchor variance

I see your code, I find that you don't add variance when you encode box

How can I use my val_generator

it is None

Invalid argument: Input to reshape is a tensor with 2002455 values, but the requested shape has 667485

According to the training steps you provided, after generating the data in TFrecord format, the above errors occurred during training. I tried all the models and it was this error. Is this a problem when generating Tfrecord?
Thank you！

voc error？

FailedPreconditionError (see above for traceback): voc2007/ImageSets; Is a directory
[[node IteratorGetNext (defined at /****/centernet/CenterNet.py:63) ]]

Hello, could you share me the YOLO-v3 pretrained model? plz

Hello, thank you for your work first !
I want to train my dataset use YOLOv3, It is very hard to train the model from scratch with no pretrained model...could you share me the YOLO-v3 pretrained model? Thank you very mych!

Retinanet error, probably cause by the loss function.

Caused by op 'inference/while/ArgMax_1', defined at:
File "testretinanet.py", line 76, in
retinanet = net.RetinaNet(config, trainset_provider)
File "F:\tensorflow-network\RetinaNet.py", line 73, in init
self._build_detection_graph()
File "F:\tensorflow-network\RetinaNet.py", line 210, in _build_detection_graph
state = tf.while_loop(cond, body, init_state)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 3556, in while_loop
return_same_structure)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 3087, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\control_flow_ops.py", line 3022, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "F:\tensorflow-network\RetinaNet.py", line 205, in
tf.squeeze(tf.gather(self.ground_truth, tf.cast(i, tf.int32))),
File "F:\tensorflow-network\RetinaNet.py", line 418, in _compute_one_image_loss
rgindex = tf.argmax(other_agiou_rate,axis=1)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\math_ops.py", line 86, in argmax
return argmax_v2(input, axis, output_type, name)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\math_ops.py", line 115, in argmax_v2
return gen_math_ops.arg_max(input, axis, name=name, output_type=output_type)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 916, in arg_max
name=name)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3300, in create_op
op_def=op_def)
File "e:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 1801, in init
self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Reduction axis 1 is empty in shape [32526,0]
[[node inference/while/ArgMax_1 (defined at F:\tensorflow-network\RetinaNet.py:418) ]]
[[node inference/add_21 (defined at F:\tensorflow-network\RetinaNet.py:217) ]]

Shape mismatch problem

Hello, I encountered a problem when training with my own training data set: the training image size was 384x384, the picture was three-way, standard voc format data was used for training, and testcenternet.py was used for training, and the following error was reported:

Tensorflow. Python. Framework. Errors_impl. InvalidArgumentError: Input to reshape is a tensor with 442368 values, but the requested shape from 147456

[[{{node Reshape_2}}]]

[[{{node IteratorGetNext}}]]

[[{{node IteratorGetNext}}]]

I don't know why the shape mismatching is reported. I hope to get your help. Thank you!

Prepare for trainning centernet

I don't have any ideas for this problem. I thought the reason is my tensorflow version. But i try to reinstall with version of 1.14.0 and it becomes the same situation.

TypeError: List of Tensors when single Tensor expected

This error occurs when I run refinedet.py,
File "/home/lijianfeng/Object-Detection-API-Tensorflow/RefineDet.py", line 431, in _compute_one_image_loss
gbbox_hwti = tf.tile(gbbox_hwti, [1, ashape[1], 1])
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 8430, in tile
"Tile", input=input, multiples=multiples, name=name)
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 513, in _apply_op_helper
raise err
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 510, in _apply_op_helper
preferred_dtype=default_dtype)
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1104, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 235, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/constant_op.py", line 214, in constant
value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 432, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/home/lijianfeng/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/tensor_util.py", line 340, in _AssertCompatible
raise TypeError("List of Tensors when single Tensor expected")
TypeError: List of Tensors when single Tensor expected

problems with the generation of tfrecords

Hey, there are some problems with the generation of tfrecords

Traceback (most recent call last):
  File "test_voc_utils.py", line 8, in <module>
    import utils.tfrecord_voc_utils as voc_utils
ModuleNotFoundError: No module named 'utils'

I've just added __init__.py to the utils folder and fixed import in related files by deleting utils before the inner object. So, believe it'll help someone.

Train own dataset，exit code

FCOS
Process finished with exit code -1073741819 (0xC0000005)。
fcos.load_weight('./fcos/test-95778') ，I deleted this line. python 3.6.7 tf 1.13.0
cuda10.0.
Is there a mistake? Total training to fixed number of steps error .

Where do you use the generated tfrecord file?

After running test_voc_utils.py some tfrecord files were generated,however these tfrecord files seems not used in testcenternet.py

class numbers

关于训练的问题

Invalid argument: Feature: ground_truth (data type: string) is required but could not be found.
在训练的时候出现上面报错请问是哪里出了问题

Validation

Hello,

Sorry for dumb question, but is there a way to include validation step after training each epoch? would it just calculate mean_loss on validation data set? how can it be used for adjusting the weights?

Thank you,
Dimitrius

Retraining

How to retrain centernet on an a new dataset?

'num_train': 5011,

'num_train': 5011,这个参数是做什么用的

my error in keypoints_loss

i met some error when i use the testcenternet.py

tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot batch tensors with different shapes in component 1. First element had shape [32,5] and element 7 had shape [30,5]. [[node IteratorGetNext (defined at /home/zyc/work/experiment/tf_detection/CenterNet.py:63) ]] [[node center_detector/cond_13/SparseToDense (defined at /home/zyc/work/experiment/tf_detection/CenterNet.py:244) ]]

and my config is
`lr = 0.001
batch_size = 8
buffer_size = 256
epochs = 300
reduce_lr_epoch = [180, 250]
config = {
'mode': 'train', # 'train', 'test'
'input_size': 256,
'data_format': 'channels_last', # 'channels_last' 'channels_first'
'num_classes': 4,
'weight_decay': 1e-4,
'keep_prob': 0.5, # not used
'batch_size': batch_size,

'score_threshold': 0.1,                                 
'top_k_results_output': 100,                           

}

image_augmentor_config = {
'data_format': 'channels_last',
'output_shape': [256, 256],
'zoom_size': [300, 300],
'crop_method': 'random',
'flip_prob': [0., 0.5],
'fill_mode': 'BILINEAR',
'keep_aspect_ratios': False,
#'constant_values': 0.,
'color_jitter_prob': 0.5,
#'rotate': [0.5, -10., 10.],
'pad_truth_to': 32,
}`

so can you help me ?

Error while training my own data

------------------------- epoch 0 -------------------------
reduce lr, lr= 0.0001 now
Traceback (most recent call last):

File "", line 1, in
runfile('D:/bu/Object-Detection-API-Tensorflow-master/Object-Detection-API-Tensorflow-master/testcenternet.py', wdir='D:/bu/Object-Detection-API-Tensorflow-master/Object-Detection-API-Tensorflow-master')

File "D:\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile
execfile(filename, namespace)

File "D:\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)

File "D:/bu/Object-Detection-API-Tensorflow-master/Object-Detection-API-Tensorflow-master/testcenternet.py", line 76, in
mean_loss = centernet.train_one_epoch(lr)

File "D:\bu\Object-Detection-API-Tensorflow-master\Object-Detection-API-Tensorflow-master\CenterNet.py", line 295, in train_one_epoch
_, loss = self.sess.run([self.train_op, self.loss], feed_dict={self.lr: lr})

File "D:\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 950, in run
run_metadata_ptr)

File "D:\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)

File "D:\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _do_run
run_metadata)

File "D:\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)

DataLossError: 2 root error(s) found.
(0) Data loss: corrupted record at 0
[[node IteratorGetNext (defined at D:\bu\Object-Detection-API-Tensorflow-master\Object-Detection-API-Tensorflow-master\CenterNet.py:63) ]]
[[center_detector/cond_59/SparseToDense/_829]]
(1) Data loss: corrupted record at 0
[[node IteratorGetNext (defined at D:\bu\Object-Detection-API-Tensorflow-master\Object-Detection-API-Tensorflow-master\CenterNet.py:63) ]]
0 successful operations.
0 derived errors ignored.

Errors may have originated from an input operation.
Input Source operations connected to node IteratorGetNext:
IteratorV2 (defined at D:\bu\Object-Detection-API-Tensorflow-master\Object-Detection-API-Tensorflow-master\utils\tfrecord_voc_utils.py:176)

Input Source operations connected to node IteratorGetNext:
IteratorV2 (defined at D:\bu\Object-Detection-API-Tensorflow-master\Object-Detection-API-Tensorflow-master\utils\tfrecord_voc_utils.py:176)

Original stack trace for 'IteratorGetNext':
File "D:\Anaconda3\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "D:\Anaconda3\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "D:\Anaconda3\lib\site-packages\spyder_kernels\console_main.py", line 11, in
start.main()
File "D:\Anaconda3\lib\site-packages\spyder_kernels\console\start.py", line 310, in main
kernel.start()
File "D:\Anaconda3\lib\site-packages\ipykernel\kernelapp.py", line 505, in start
self.io_loop.start()
File "D:\Anaconda3\lib\site-packages\tornado\platform\asyncio.py", line 148, in start
self.asyncio_loop.run_forever()
File "D:\Anaconda3\lib\asyncio\base_events.py", line 539, in run_forever
self._run_once()
File "D:\Anaconda3\lib\asyncio\base_events.py", line 1775, in _run_once
handle._run()
File "D:\Anaconda3\lib\asyncio\events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "D:\Anaconda3\lib\site-packages\tornado\ioloop.py", line 690, in
lambda f: self._run_callback(functools.partial(callback, future))
File "D:\Anaconda3\lib\site-packages\tornado\ioloop.py", line 743, in _run_callback
ret = callback()
File "D:\Anaconda3\lib\site-packages\tornado\gen.py", line 781, in inner
self.run()
File "D:\Anaconda3\lib\site-packages\tornado\gen.py", line 742, in run
yielded = self.gen.send(value)
File "D:\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 357, in process_one
yield gen.maybe_future(dispatch(*args))
File "D:\Anaconda3\lib\site-packages\tornado\gen.py", line 209, in wrapper
yielded = next(result)
File "D:\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 267, in dispatch_shell
yield gen.maybe_future(handler(stream, idents, msg))
File "D:\Anaconda3\lib\site-packages\tornado\gen.py", line 209, in wrapper
yielded = next(result)
File "D:\Anaconda3\lib\site-packages\ipykernel\kernelbase.py", line 534, in execute_request
user_expressions, allow_stdin,
File "D:\Anaconda3\lib\site-packages\tornado\gen.py", line 209, in wrapper
yielded = next(result)
File "D:\Anaconda3\lib\site-packages\ipykernel\ipkernel.py", line 294, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "D:\Anaconda3\lib\site-packages\ipykernel\zmqshell.py", line 536, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "D:\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2848, in run_cell
raw_cell, store_history, silent, shell_futures)
File "D:\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2874, in _run_cell
return runner(coro)
File "D:\Anaconda3\lib\site-packages\IPython\core\async_helpers.py", line 67, in _pseudo_sync_runner
coro.send(None)
File "D:\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3049, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "D:\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3220, in run_ast_nodes
if (yield from self.run_code(code, result)):
File "D:\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 3296, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 1, in
runfile('D:/bu/Object-Detection-API-Tensorflow-master/Object-Detection-API-Tensorflow-master/testcenternet.py', wdir='D:/bu/Object-Detection-API-Tensorflow-master/Object-Detection-API-Tensorflow-master')
File "D:\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 786, in runfile
execfile(filename, namespace)
File "D:\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "D:/bu/Object-Detection-API-Tensorflow-master/Object-Detection-API-Tensorflow-master/testcenternet.py", line 64, in
centernet = net.CenterNet(config, trainset_provider)
File "D:\bu\Object-Detection-API-Tensorflow-master\Object-Detection-API-Tensorflow-master\CenterNet.py", line 44, in init
self._define_inputs()
File "D:\bu\Object-Detection-API-Tensorflow-master\Object-Detection-API-Tensorflow-master\CenterNet.py", line 63, in _define_inputs
self.images, self.ground_truth = self.train_iterator.get_next()
File "D:\Anaconda3\lib\site-packages\tensorflow\python\data\ops\iterator_ops.py", line 426, in get_next
output_shapes=self._structure._flat_shapes, name=name)
File "D:\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_dataset_ops.py", line 1974, in iterator_get_next
output_shapes=output_shapes, name=name)
File "D:\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "D:\Anaconda3\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "D:\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3616, in create_op
op_def=op_def)
File "D:\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2005, in init
self._traceback = tf_stack.extract_stack()

Please could you tell my why this error occured?

how to convert xml to tfrecord during training ?

Hi, I am using your centernet code,but there are no comments in the code so I do not know how to perform training.

My data format is standard pascal VOC xml files.However I cannot see xml processing code in testcenternet.py. Can you help me?Thank you!

训练中tensorflow.python.framework.errors_impl.InvalidArgumentError: Reduction axis 1 is empty in shape [8828,0]

您好，
我只用了voc2007数据集，使用项目中的test_voc_utils.py，生成的tfrecord。
在运行SSD300.py时，一开始可以训练，但多一会就会报错退出。先后换了tensorflow1.12和1.13版本，错误相同

------------------------- epoch 2 -------------------------
iters 262/312 loss 4.369279Traceback (most recent call last):
File "/home/lwc/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1334, in _do_call
return fn(*args)
File "/home/lwc/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/home/lwc/.local/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Reduction axis 1 is empty in shape [8828,0]
[[{{node inference/while/ArgMax_1}} = ArgMax[T=DT_FLOAT, Tidx=DT_INT32, output_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:GPU:0"](inference/while/boolean_mask_5/GatherV2, inference/while/ArgMax_1/dimension)]]
[[{{node inference/while/SparseTensor/dense_shape/_221}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_1593_inference/while/SparseTensor/dense_shape", tensor_type=DT_INT64, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

tf.keras and tensorflow 2.0 support?

Does there any plan to support tensorflow 2.0 and tf,keras ? As well as model.fit training procedure?

After training, model has bad detections or lower accuracy.

I am trained models(centernet, refinedet and yolov3) under argumentation configuration...

image_augmentor_config = {
'data_format': 'channels_last',
'output_shape': [320, 320],
# 'zoom_size': [330, 330],
# 'crop_method': 'random',
'flip_prob': [0.0, 0.5],
'fill_mode': 'BILINEAR',
'keep_aspect_ratios': False,
'constant_values': 0.,
'color_jitter_prob': 0.5,
# 'rotate_range': [0.5, -10., 10.],
'pad_truth_to': 60,
}

dataset: VOC 2007

batch size: 16

buffer size :1024

config: code's default except batch size(because gpu's vram size..)

The model's training is working well....but after training, model has bad detections or lower accuracy.

additionally, I don't know why model needs param "pad_truth_to"..

Did I somthing wrong?

train problem,a valid checkpoint????

C:\Users\kzz\AppData\Local\Programs\Python\Python37\python.exe Y:/Models/Object-Detection-API-Tensorflow-master/testrefinedet.py
WARNING:tensorflow:From C:\Users\kzz\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\ops\control_flow_ops.py:423: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.

WARNING: The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:

• https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
• https://github.com/tensorflow/addons
  If you depend on functionality not listed there, please file an issue.

WARNING:tensorflow:From Y:\Models\Object-Detection-API-Tensorflow-master\RefineDet.py:671: max_pooling2d (from tensorflow.python.layers.pooling) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.max_pooling2d instead.
WARNING:tensorflow:From Y:\Models\Object-Detection-API-Tensorflow-master\RefineDet.py:642: conv2d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.conv2d instead.
WARNING:tensorflow:From Y:\Models\Object-Detection-API-Tensorflow-master\RefineDet.py:620: batch_normalization (from tensorflow.python.layers.normalization) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.batch_normalization instead.
WARNING:tensorflow:From Y:\Models\Object-Detection-API-Tensorflow-master\RefineDet.py:657: conv2d_transpose (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.conv2d_transpose instead.
WARNING:tensorflow:From C:\Users\kzz\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\ops\losses\losses_impl.py:209: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
WARNING:tensorflow:From C:\Users\kzz\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\ops\math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
C:\Users\kzz\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\ops\gradients_impl.py:110: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2019-10-08 17:19:51.745878: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
WARNING:tensorflow:From C:\Users\kzz\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\training\saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
Traceback (most recent call last):
File "Y:/Models/Object-Detection-API-Tensorflow-master/testrefinedet.py", line 62, in
refinedet.load_weight('./refinedet320/test-1092')
File "Y:\Models\Object-Detection-API-Tensorflow-master\RefineDet.py", line 613, in load_weight
self.saver.restore(self.sess, path)
File "C:\Users\kzz\AppData\Local\Programs\Python\Python37\lib\site-packages\tensorflow\python\training\saver.py", line 1268, in restore
+ compat.as_text(save_path))
ValueError: The passed save_path is not a valid checkpoint: ./refinedet320/test-1092

FCOS网络

在测试FCOS网络时出现这个问题
ValueError: List argument 'values' to 'ConcatV2' Op with length 0 shorter than minimum length 2.

训练问题

def dataset2tfrecord(xml_dir, img_dir, output_dir, name, total_shards=5):
请问这里的total_shards是什么参数，如果我只训练识别一个东西，这里应该设置多少

custom dataset image size problem

Hello,Thank you for your code.
When I run this code, it works on VOC dataset. When I usu my own custom dataset with only one class, these image are 1000x1400 size，some errors have occurred...
------------------------- epoch 0 -------------------------
Traceback (most recent call last):
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call
return fn(*args)
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 4200000 values, but the requested shape has 1400000
[[{{node Reshape_2}}]]
[[{{node IteratorGetNext}}]]
[[{{node SparseToDense_15}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "H:/deeplearning/Object-Detection-API-Tensorflow-master/testYOLOv3.py", line 77, in
mean_loss = testnet.train_one_epoch(lr)
File "H:\deeplearning\Object-Detection-API-Tensorflow-master\YOLOv3.py", line 451, in train_one_epoch
feed_dict={self.lr: lr})
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
run_metadata_ptr)
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run
run_metadata)
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 4200000 values, but the requested shape has 1400000
[[node Reshape_2 (defined at H:\deeplearning\Object-Detection-API-Tensorflow-master\YOLOv3.py:41) ]]
[[node IteratorGetNext (defined at H:\deeplearning\Object-Detection-API-Tensorflow-master\YOLOv3.py:72) ]]
[[node SparseToDense_15 (defined at H:\deeplearning\Object-Detection-API-Tensorflow-master\YOLOv3.py:127) ]]

Caused by op 'Reshape_2', defined at:
File "H:/deeplearning/Object-Detection-API-Tensorflow-master/testYOLOv3.py", line 70, in
testnet = yolov3.YOLOv3(config, trainset_provider)
File "H:\deeplearning\Object-Detection-API-Tensorflow-master\YOLOv3.py", line 41, in init
self.priors.append(tf.reshape(tf.constant(priors[i], dtype=tf.float32)/self.stride[i], [1, 1, -1, 2]))
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 8462, in reshape
"Reshape", tensor=tensor, shape=shape, name=name)
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\util\deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\framework\ops.py", line 3300, in create_op
op_def=op_def)
File "D:\Software\python\anaconda\lib\site-packages\tensorflow\python\framework\ops.py", line 1801, in init
self._traceback = tf_stack.extract_stack()

InvalidArgumentError (see above for traceback): Input to reshape is a tensor with 4200000 values, but the requested shape has 1400000

centerness-loss

    center_pos_loss = -center_gt*tf.log_sigmoid(center_pred) * loc
    center_neg_loss = -(1.-center_gt)*(-center_pred+tf.log_sigmoid(center_pred)) * (1.-loc)
    center_loss = tf.reduce_sum(center_pos_loss) + tf.reduce_sum(center_neg_loss)

The location not in groundtruth boxes should not be inclued while computing centerness-loss, and the code of binary sigmod entropy may be wrong.
-(y_true*log(y_pred) + (1 - y_truth) log(1 - y_pred))