abdelrahman-gaber/tf2-object-detection-api-tutorial

Tensorflow 2 Object Detection API Tutorial. This tutorial will take you from installation, to running pre-trained detection model, and training your model with a custom dataset, then exporting it for inference.

Hi there 👋, I'm Abdelrahman

I am a Machine Learning Engineer with over 5 years of experience in the fields of Machine Learning, Deep Learning and Computer Vision, along with exposure to Robotics. I currently work as a Machine Learning Engineer at avatarin, trying to enhance robots' human-like capabilities using Imitation Learning and Computer Vision.

Languages and Tools 👨🏻‍💻

     

           

         

     

Connect With Me 🤝

---

tf2-object-detection-api-tutorial's Issues

batch size > 1 in config file for running eval script

I am trying to evaluate the model having batch size more than 1 and it throws me errors. I have checked in many other sources and their suggestion would be to make it 1. But for our application we need a batch size > 1. Following is the error I get.

Traceback (most recent call last):
File "models/research/object_detection/model_main_tf2.py", line 114, in
tf.compat.v1.app.run()
File "/home/fyp3/anaconda3/envs/py3tf2/lib/python3.8/site-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/home/fyp3/anaconda3/envs/py3tf2/lib/python3.8/site-packages/absl/app.py", line 300, in run
_run_main(main, args)
File "/home/fyp3/anaconda3/envs/py3tf2/lib/python3.8/site-packages/absl/app.py", line 251, in _run_main
sys.exit(main(argv))
File "models/research/object_detection/model_main_tf2.py", line 81, in main
model_lib_v2.eval_continuously(
File "/media/fyp3/New Volume/ActivityDetection/ActivityDetection/Custom/models/research/object_detection/model_lib_v2.py", line 990, in eval_continuously
eager_eval_loop(
File "/media/fyp3/New Volume/ActivityDetection/ActivityDetection/Custom/models/research/object_detection/model_lib_v2.py", line 803, in eager_eval_loop
eval_dict, losses_dict, class_agnostic = compute_eval_dict(features, labels)
File "/home/fyp3/anaconda3/envs/py3tf2/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 780, in call
result = self._call(*args, **kwds)
File "/home/fyp3/anaconda3/envs/py3tf2/lib/python3.8/site-packages/tensorflow/python/eager/def_function.py", line 840, in _call
return self._stateless_fn(*args, **kwds)
File "/home/fyp3/anaconda3/envs/py3tf2/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 2829, in call
return graph_function._filtered_call(args, kwargs) # pylint: disable=protected-access
File "/home/fyp3/anaconda3/envs/py3tf2/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1843, in _filtered_call
return self._call_flat(
File "/home/fyp3/anaconda3/envs/py3tf2/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 1923, in _call_flat
return self._build_call_outputs(self._inference_function.call(
File "/home/fyp3/anaconda3/envs/py3tf2/lib/python3.8/site-packages/tensorflow/python/eager/function.py", line 545, in call
outputs = execute.execute(
File "/home/fyp3/anaconda3/envs/py3tf2/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 59, in quick_execute
tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
tensorflow.python.framework.errors_impl.InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: Shapes of all inputs must match: values[0].shape = [1] != values[3].shape = [0]
[[node stack_47 (defined at /media/fyp3/New Volume/ActivityDetection/ActivityDetection/Custom/models/research/object_detection/model_lib.py:158) ]]
[[ToNormalizedCoordinates/Assert/AssertGuard/pivot_f/_828/_321]]
(1) Invalid argument: Shapes of all inputs must match: values[0].shape = [1] != values[3].shape = [0]
[[node stack_47 (defined at /media/fyp3/New Volume/ActivityDetection/ActivityDetection/Custom/models/research/object_detection/model_lib.py:158) ]]
0 successful operations.
0 derived errors ignored. [Op:__inference_compute_eval_dict_28651]

Errors may have originated from an input operation.
Input Source operations connected to node stack_47:
Slice_4 (defined at /media/fyp3/New Volume/ActivityDetection/ActivityDetection/Custom/models/research/object_detection/model_lib.py:276)

Input Source operations connected to node stack_47:
Slice_4 (defined at /media/fyp3/New Volume/ActivityDetection/ActivityDetection/Custom/models/research/object_detection/model_lib.py:276)

Function call stack:
compute_eval_dict -> compute_eval_dict

Can you please let me know what this is and/or whether API can handle only batch_size == 1 for evaluation?

training loss=nan

Thanks for the nice tutorial. I was able to train my custom datasets but got loss=Nan.
I also decreased the learning rate but couldn't fix the problem?

Any suggestions?

Reading images in RGB

The example from the TF2 Model Zoo inference tutorial notebook loads images in RGB format with:

def load_image_into_numpy_array(path):
  """Load an image from file into a numpy array.

  Puts image into numpy array to feed into tensorflow graph.
  Note that by convention we put it into a numpy array with shape
  (height, width, channels), where channels=3 for RGB.

  Args:
    path: the file path to the image

  Returns:
    uint8 numpy array with shape (img_height, img_width, 3)
  """
  img_data = tf.io.gfile.GFile(path, 'rb').read()
  image = Image.open(BytesIO(img_data))
  (im_width, im_height) = image.size
  return np.array(image.getdata()).reshape(
      (im_height, im_width, 3)).astype(np.uint8)

In this tutorial you just use cv2.imread() which will produce numpy arrays in BGR format.

Not sure if this is has been accounted for and I just missed it but if not then it should be fixed. It should be as easy as adding

img = img[...,::-1]

here

tf2-object-detection-api-tutorial/detector.py

Line 22 in 91f6e16

im_height, im_width, _ = img.shape

Error while converting model to tensorflow lite

Thank you, Abdelrahman, so much for the well-organized tutorial, I really spent time searching for good tutorials for object detection and up to date.

I had successfully train and test the model as you explained, but when I tried to convert it to tensorflow lite, I got the following error:
Could you please help me?

tflite_convert \ --saved_model_dir=models/ssd_mobilenet_v2_raccoon/exported_model/saved_model \ --output_file=models/ssd_mobilenet_v2_raccoon/ssd_mobilenet_v2_raccoon.tflite

I got this error:

2020-08-05 10:19:30.217661: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:799] constant_folding: Graph size after: 1394 nodes (0), 2116 edges (0), time = 77.785ms.
Traceback (most recent call last):
File "/opt/anaconda3/envs/v_python3.6/bin/tflite_convert", line 8, in
sys.exit(main())
File "/opt/anaconda3/envs/v_python3.6/lib/python3.6/site-packages/tensorflow/lite/python/tflite_convert.py", line 638, in main
app.run(main=run_main, argv=sys.argv[:1])
File "/opt/anaconda3/envs/v_python3.6/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 40, in run
_run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)
File "/opt/anaconda3/envs/v_python3.6/lib/python3.6/site-packages/absl/app.py", line 299, in run
_run_main(main, args)
File "/opt/anaconda3/envs/v_python3.6/lib/python3.6/site-packages/absl/app.py", line 250, in _run_main
sys.exit(main(argv))
File "/opt/anaconda3/envs/v_python3.6/lib/python3.6/site-packages/tensorflow/lite/python/tflite_convert.py", line 621, in run_main
_convert_tf2_model(tflite_flags)
File "/opt/anaconda3/envs/v_python3.6/lib/python3.6/site-packages/tensorflow/lite/python/tflite_convert.py", line 237, in _convert_tf2_model
tflite_model = converter.convert()
File "/opt/anaconda3/envs/v_python3.6/lib/python3.6/site-packages/tensorflow/lite/python/lite.py", line 483, in convert
_get_tensor_name(tensor), shape_list))
ValueError: None is only supported in the 1st dimension. Tensor 'input_tensor' has invalid shape '[1, None, None, 3]'.

Starting a evaluation after training done but it seems it start training again instead of doing evaluation

Hi, Thank you for creating this tutorial. As the title suggest, I just want to see the metrics by running the evaluation after the training done, but it seems the script start to training again since the number of steps continue to be logged and rising. Can you specify what should I change if I want to do evaluation after the training done.

FYI I do early stopping in the training, means I stop the training before the steps reach num_steps I specify in config, is that why the evaluation script continue to training instead of doing evaluation? Thank you

A way to find Inference speed

Is there a way to find the Inference speed of my .pb file ?
with a modification to the files

generate_tfrecord.py

Hello Abdelrahman
I am getting an error in the file of generate_tfrecord.py in

from object_detection.utils import dataset_util from object_detection.utils import label_map_util

I checked my environment and the TF object detection, everything seems good, can you share with me what you think about this issue?
Thank you

I got An exception has occurred, use %tb to see the full traceback. SystemExit: 2 with generating tfrecord

usage: [-h] test test/_annotations.csv label_map.pbtxt test
: error: the following arguments are required: test, test/_annotations.csv, label_map.pbtxt, test
An exception has occurred, use %tb to see the full traceback.

SystemExit: 2

C:\Users\ferao\anaconda3\envs\mostfgpu\lib\site-packages\IPython\core\interactiveshell.py:3449: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)

Calculations in train_labels.csv

The calculations in your -> tf2-object-detection-api-tutorial/data/raccoon_data/train_labels.csv do not add up.

Take the first line for example

width= 800 
height= 533
xmin= 105 
ymin=10 
xmax= 502 
ymax= 501

If your xmin=105 and your width is 800, then your xmax should be 905 and not 502.

Same for the y-axis,
if your ymin=10 and your height is 533 then your ymax should be 543 and not 501

Please correct me if I missed something.

tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[12] = 12 is not in [0, 0)

I am trying to write an alternative to xml_to_csv.py for TFRecord as I am using VGG Image Annotator and extracted the annotations as JSON.

Here is a sample JSON file

{
  "0.jpg59329": {
    "filename": "0.jpg",
    "size": 59329,
    "regions": [{
      "shape_attributes": {
        "name": "rect",
        "x": 412,
        "y": 130,
        "width": 95,
        "height": 104
      },
      "region_attributes": {}
    }, {
      "shape_attributes": {
        "name": "rect",
        "x": 521,
        "y": 82,
        "width": 126,
        "height": 106
      },
      "region_attributes": {}
    }
}

With VGG Annotator, you have a directory for images and a directory for annotation in separate folders.

Here is my code for creating TF records:

# Ref 1: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/using_your_own_dataset.md
# Ref 2: https://github.com/datitran/raccoon_dataset/blob/master/generate_tfrecord.py


import json
import glob
from object_detection.utils import dataset_util
import tensorflow as tf
from pathlib import Path

flags = tf.compat.v1.app.flags
flags.DEFINE_string('output_path', '', 'Path to output TFRecord')
FLAGS = flags.FLAGS


def json_to_tf(jsonFile, im):
    with open(im, "rb") as image:
        encoded_image_data = image.read()

    with open(jsonFile) as json_file:
        data = json.load(json_file)

        for key, value in data.items():
            width = 1920
            height = 1080
            filename = value["filename"]
            filename = filename.encode('utf8')
            image_format = b'jpeg'
            xmins = []
            xmaxs = []
            ymins = []
            ymaxs = []
            classes_text = []
            classes = []

            for x in value["regions"]:
                xmins.append(x["shape_attributes"]['x'])
                xmaxs.append(x["shape_attributes"]['width'] + x["shape_attributes"]['x'])
                ymins.append(x["shape_attributes"]['y'])
                ymaxs.append(x["shape_attributes"]['height'] + x["shape_attributes"]['y'])
                classes_text.append("car".encode('utf8'))
                classes.append(1)

            tf_example = tf.train.Example(features=tf.train.Features(feature={
                'image/height': dataset_util.int64_feature(height),
                'image/width': dataset_util.int64_feature(width),
                'image/filename': dataset_util.bytes_feature(filename),
                'image/source_id': dataset_util.bytes_feature(filename),
                'image/encoded': dataset_util.bytes_feature(encoded_image_data),
                'image/format': dataset_util.bytes_feature(image_format),
                'image/object/bbox/xmin': dataset_util.float_list_feature(xmins),
                'image/object/bbox/xmax': dataset_util.float_list_feature(xmaxs),
                'image/object/bbox/ymin': dataset_util.float_list_feature(ymins),
                'image/object/bbox/ymax': dataset_util.float_list_feature(ymaxs),
                'image/object/class/text': dataset_util.bytes_list_feature(classes_text),
                'image/object/class/label': dataset_util.int64_list_feature(classes),
            }))
            writer.write(tf_example.SerializeToString())

writer = tf.compat.v1.python_io.TFRecordWriter("train.record")

for fn in glob.glob("..\\annotation_refined\\*.json"):
    for img in glob.glob("..\\images\\*.jpg"):
        if Path(fn).stem == Path(img).stem:
            tf_example_1 = json_to_tf(fn, img)

writer.close()

The error I get:

tensorflow.python.framework.errors_impl.NotFoundError: NewRandomAccessFile failed to Create/Open: : The system cannot find the path specified.
; No such process

I am not sure if it is related to the tf_record or something else, any ideas @abdelrahman-gaber ?