taipingeric/yolo-v4-tf.keras

A simple tf.keras implementation of YOLO v4

yolo-v4-tf.keras

A simple tf.keras implementation of YOLO v4

TODO

• Cosine annealing scheduler
• mAP
• Mosaic augmentation
• DropBlock
• Self-adversarial training (SAT)
• Label smoothing
• Mish
• IoU, GIoU, CIoU loss
• multi-GPU training

Quick Start

1. Download official YOLO v4 pre-trained weights from github/AlexeyAB/darknet

2. Initialize YOLO model and load weights

3. Run prediction

   Example: Inference.ipynb:

from models import Yolov4
model = Yolov4(weight_path='yolov4.weights', 
               class_name_path='class_names/coco_classes.txt')
model.predict('input.jpg')

Training

1. Generate your annotation files (.XML) in VOC format for each images

   HINT: An easily used annotation tool: labelImg

   Example: A 2 object xml file

   <annotation>
       <folder>train_img2</folder>
       <filename>yui.jpg</filename>
       <path>/Users/taipingeric/dataset/train_img2/yui.jpg</path>
       <source>
           <database>Unknown</database>
       </source>
       <size>
           <width>465</width>
           <height>597</height>
           <depth>3</depth>
       </size>
       <segmented>0</segmented>
       <object>
           <name>person</name>
           <pose>Unspecified</pose>
           <truncated>1</truncated>
           <difficult>0</difficult>
           <bndbox>
               <xmin>43</xmin>
               <ymin>41</ymin>
               <xmax>430</xmax>
               <ymax>597</ymax>
           </bndbox>
       </object>
       <object>
           <name>person</name>
           <pose>Unspecified</pose>
           <truncated>1</truncated>
           <difficult>0</difficult>
           <bndbox>
               <xmin>60</xmin>
               <ymin>70</ymin>
               <xmax>20</xmax>
               <ymax>207</ymax>
           </bndbox>
       </object>
   </annotation>
   

2. Convert all XML files to a single .txt file:

   Row format: img_path BOX0 BOX1 BOX2 ...

   BOX format: xmin,ymin,xmax,ymax,class_id

   Example: xml_to_txt.py

   img1.jpg 50,60,70,80,0 70,90,100,180,2
   img2.jpg 10,60,70,80,0
   ...
   

3. Generate class name file, # of lines == # of classes

   Example: coco_classes.txt

   person
   bicycle
   car
   motorbike
   aeroplane
   bus
   ...
   

4. Train with the code below

   Example: train.ipynb

from utils import DataGenerator, read_annotation_lines
from models import Yolov4

train_lines, val_lines = read_annotation_lines('../dataset/txt/anno-test.txt', 
                                               test_size=0.1)
FOLDER_PATH = '../dataset/img'
class_name_path = '../class_names/bccd_classes.txt'
data_gen_train = DataGenerator(train_lines, 
                               class_name_path, 
                               FOLDER_PATH)
data_gen_val = DataGenerator(val_lines, 
                             class_name_path, 
                             FOLDER_PATH)

model = Yolov4(weight_path=None, 
               class_name_path=class_name_path)

model.fit(data_gen_train, 
          initial_epoch=0,
          epochs=10000, 
          val_data_gen=data_gen_val,
          callbacks=[])

Acknowledgements

• qqwweee/keras-yolo3
• AlexeyAB/darknet
• hunglc007/tensorflow-yolov4-tflite
• Cartucho/mAP
• miemie2013/Keras-YOLOv4
• david8862/keras-YOLOv3-model-set
• Ma-Dan/keras-yolo4

yolo-v4-tf.keras's Issues

error while loading weights

i think this error occur when it comes to an output layer
cannot reshape array of size 4243077 into shape (1024,512,3,3)
---> conv_weights = conv_weights.reshape(conv_shape).transpose([2, 3, 1, 0])

Loss and Validation loss is nan

nms iou: 0.413 score: 0.3
Epoch 1/50
1125/1125 [==============================] - 408s 349ms/step - loss: 1367.1504 - val_loss: nan

What can be the possible reason for val_loss becoming nan and after first iteration loss itself also becomes nan

Why model is not saved after training?

I have a few questions

1. Why model is not saved after training
2. How to calculate mAP for each class
3. How to run test on trained model

ValueError: A KerasTensor cannot be used as input to a TensorFlow function

I'm getting the below error when I try to initialize the model.

Code:
from models import Yolov4
model = Yolov4(weight_path='yolov4.weights',
class_name_path='class_names/coco_classes.txt')

Error:
ValueError: A KerasTensor cannot be used as input to a TensorFlow function. A KerasTensor is a symbolic placeholder for a shape and dtype, used when constructing Keras Functional models or Keras Functions. You can only use it as input to a Keras layer or a Keras operation (from the namespaces keras.layers and keras.operations). You are likely doing something like:

x = Input(...)
...
tf_fn(x)  # Invalid.

What you should do instead is wrap tf_fn in a layer:

class MyLayer(Layer):
    def call(self, x):
        return tf_fn(x)

x = MyLayer()(x)

I've tried unwrapping the layer and following the above recommendation but the error still persists.

Train script is not working

It is giving nonetype error

InvalidArgumentError: TypeError: 'NoneType' object is not subscriptable
Traceback (most recent call last):

File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/script_ops.py", line 249, in call
ret = func(*args)

File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/impl/api.py", line 645, in wrapper
return func(*args, **kwargs)

File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/data/ops/dataset_ops.py", line 961, in generator_py_func
values = next(generator_state.get_iterator(iterator_id))

File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/data_adapter.py", line 837, in wrapped_generator
for data in generator_fn():

File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/data_adapter.py", line 963, in generator_fn
yield x[i]

File "../utils.py", line 159, in getitem
X, y_tensor, y_bbox = self.__data_generation(lines)

File "../utils.py", line 179, in __data_generation
img_data, box_data = self.get_data(line)

File "../utils.py", line 190, in get_data
img = cv2.imread(os.path.join(self.folder_path, img_path))[:, :, ::-1]

TypeError: 'NoneType' object is not subscriptable

 [[{{node PyFunc}}]]
 [[IteratorGetNext]] [Op:__inference_train_function_74263]

Function call stack:
train_function

argument must be a string a bytes-like object or a number not 'tensor'

Hi, I run the model and get error:
"argument must be a string a bytes-like object or a number not 'tensor' "
in the file: tensor_shape.py line 193 in init
what could be the problem?
My custom have 4 classes and wasn't be- trained

Issue with Running on Colab

While trying to use the repo in google colab, the command

from models import Yolov4

give an error:

ModuleNotFoundError: No module named 'models'

Please help.

Can't load custom model properly

Good afternoon :)
I train model like You said in train.ipynb. Then predict, it predicts well:

img shape: (324, 432, 3) # of bboxes: 2

Then I save the model: model.save_model('auto_model3.h5')
And when I load it back by model = Yolov4(weight_path='auto_model3.h5', class_name_path=class_name_path) , it does not predict at all! Doesn't show any boxes. I can't understand why and what am I doing wrong:

img shape: (324, 432, 3) # of bboxes: 0

Model size is ok: auto_model3.h5 - 251 181 KB

Any advice? Or can You show the code, when You train the model. then save it, then loads back and it works?
Thank You in advance ))

What are the requirements for this repo?

What version of TensorFlow do you use?
I see that it's using TF2 however which version?
Thanks

Load Training Checkpoints

The saved model after Training all epochs (model.fit(...) -> model.save_model()) works decently as only the yolo_model seems to be saved. Checkpoints created during training cannot be loaded.

Tried to write an own Callback but I cannot only save the current version of the yolo_model.

Any suggestions?

---

`InvalidArgumentError: Index out of range using input dim 0; input has only 0 dims for '{{node strided_slice_4}} = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1](yolo_loss_1/Identity, strided_slice_4/begin, strided_slice_4/end, strided_slice_4/strides)' with input shapes: [], [1], [1], [1] and with computed input tensors: input[3] = <1>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "", line 1, in
model.load_model('/home/omen2/Schreibtisch/OD-Netz_zsm-Tiere/Antilopen_zsm/Training/Kudu_Gelsenkirchen_2-374/Training_nt2/2022-02-07_OD_Kudu_Gelsenkirchen_2-374-ckpt/epoch-225.h5')

File "/home/omen2/60-PromotionJenny/KI_Projekt/KI-Projekt/global/yolo-v4-tf.keras-master/models.py", line 100, in load_model
yolov4_output = yolov4_head(self.yolo_model.output, self.num_classes, self.anchors, self.xyscale)

File "/home/omen2/60-PromotionJenny/KI_Projekt/KI-Projekt/global/yolo-v4-tf.keras-master/custom_layers.py", line 202, in yolov4_head
bbox0, object_probability0, class_probabilities0, pred_box0 = get_boxes(yolo_neck_outputs[0],

File "/home/omen2/anaconda3/envs/ki/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py", line 973, in _slice_helper
return strided_slice(

File "/home/omen2/anaconda3/envs/ki/lib/python3.8/site-packages/tensorflow/python/ops/array_ops.py", line 1140, in strided_slice
op = gen_array_ops.strided_slice(

File "/home/omen2/anaconda3/envs/ki/lib/python3.8/site-packages/tensorflow/python/ops/gen_array_ops.py", line 10174, in strided_slice
_, _, _op, _outputs = _op_def_library._apply_op_helper(

File "/home/omen2/anaconda3/envs/ki/lib/python3.8/site-packages/tensorflow/python/framework/op_def_library.py", line 742, in _apply_op_helper
op = g._create_op_internal(op_type_name, inputs, dtypes=None,

File "/home/omen2/anaconda3/envs/ki/lib/python3.8/site-packages/tensorflow/python/framework/func_graph.py", line 593, in _create_op_internal
return super(FuncGraph, self)._create_op_internal( # pylint: disable=protected-access

File "/home/omen2/anaconda3/envs/ki/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 3319, in _create_op_internal
ret = Operation(

File "/home/omen2/anaconda3/envs/ki/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 1816, in init
self._c_op = _create_c_op(self._graph, node_def, inputs,

File "/home/omen2/anaconda3/envs/ki/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 1657, in _create_c_op
raise ValueError(str(e))

ValueError: Index out of range using input dim 0; input has only 0 dims for '{{node strided_slice_4}} = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1](yolo_loss_1/Identity, strided_slice_4/begin, strided_slice_4/end, strided_slice_4/strides)' with input shapes: [], [1], [1], [1] and with computed input tensors: input[3] = <1>.`

How to load saved model?

How do I load saved model after custom training ?

export prediction

Hello, how to use export prediction function, what is the annotation path ?

Unable to access weights

The code finished training on a set of custom data with custom classes, and had a print statement saying I should have weights in the "Y4.weights/assets" folder. While the folder was created, there is nothing there. In the Y4.weights folder there is a saved_model.pb and a variables folder with files in them, but no *.weights to be found.

The execution of the code is also littered with depreciation warnings. Is it possible to add some proper documentation to this repo that includes the intended versions of tensorflow and keras? I'm using 2.2 for both, but these warnings make me think I should be using older versions.

Most common warning is "arguments object has no attribute posonlyargs", and a change that makes layer updates automatic.

---

Edits: grammar, and more detail/context added

How to save model to TensorRT

Is there a way to save and load this model on TensorRT ?

Continue Interrupted Training from the Best Weights

Hello.

How can you continue training from the last saved best weights?

image_size and anchor size

Hello,

thank you for this repo. On the demo task, it works out of the box. I want to train Yolov4 for my custom dataset.

My custom dataset has three class, and original images are 3072x3072.
I have annotated objects in original image and then I created two further down scaled dataset of

• 2048x2048; also generated new annotations and anchors as per downsampling
• 1024X1024; also generated new annotations and anchors as per downsampling

Now to train the model, all training images will be resized to image_size mentioned in config file. (i.e. 416x416)
Do I also need to change anchor size based on 416 x 416 or the anchors generated for 1024x1024 dataset and 2048x2048 dataset will work fine.

Also is it recommended to train Yolov4 with high resolution images (1024x1024 or 2048x2048 or 3072x3072)? or I should downscale all of them to 512x512 or something smaller and then start training?

Freezing Layers for transfer learning

Hi I intend to use the pre-trained darknet weights to train my own model, is such training possible in the current setup?

I found that you load the weights here:
in Yolov4.build_model()

 def build_model(self, load_pretrained=True):
      if load_pretrained and self.weight_path and self.weight_path.endswith('.weights'):
          if self.weight_path.endswith('.weights'):
              load_weights(self.yolo_model, self.weight_path)
              print(f'load from {self.weight_path}')
          elif self.weight_path.endswith('.h5'):
              self.training_model.load_weights(self.weight_path)
              print(f'load from {self.weight_path}')

As such I found no way to freeze layers in your current code.

Plot loss

How can I plot training and validation loss curves? And in which code should i write it?

Convert Custom Yolov4 model

HI! I want to ask that I want to convert Yolov4 which has trained on 12 classes, hence I change some filter values. Will this repo will change my weights to the h5 Keras model successfully ??

Model Fails to load darknet weights when using custom classes.

Following is the message I recieve:
failed to read all weights, # of unread weights: 0 load from ./darknet_base_model/yolov4.weights

I am training it on the ASL (American Sign Language) dataset and it has 26 classes from A-Z.
The class file is formatted as per standard (1 class per line).

Problem loading the weights

Can anyone help me please? What could be the reason?

---

ValueError Traceback (most recent call last)
in
----> 1 load_weights(model, './model_data/yolov4.weights')

in load_weights(model, weights_file_path)
30 conv_shape = (filters, input_dims, kernel_size, kernel_size)
31 conv_weights = np.fromfile(file, dtype=np.float32, count=np.product(conv_shape))
---> 32 conv_weights = conv_weights.reshape(conv_shape).transpose([2, 3, 1, 0])
33
34 if conv_idx not in conv_output_idxs:

ValueError: cannot reshape array of size 3117998 into shape (1024,512,3,3)

Index out of bound error in custom training

Hello!

I tried to train a custom model with another data set but failed with the following:

  File "/home/.../yolo-v4-tf.keras-master/utils.py", line 290, in preprocess_true_boxes
    y_true[stage][batch_idx, grid_row, grid_col, anchor_idx, :2] = true_boxes_xy[batch_idx, box_idx, :]  # abs xy

IndexError: index 52 is out of bounds for axis 1 with size 52

The differences with the data:

• 7 categories (rows in the class description data set)
• 1024*768 image size

Incorrect Anchors

I realized the anchors used in this repo are the default anchors.
However the default papers used 512 x 512 input size.
This repo is using 416 x 416
with anchors exceeding the image size... config.py

'anchors': [12, 16, 19, 36, 40, 28, 36, 75, 76, 55, 72, 146, 142, 110, 192, 243, 459, 401],

weights

Hello
Where do I find the weights to run the inference?

How to add augmentation to code ?

Hello can you tell where I can add augmentation to the image and labels?