tanakataiki/ssd_kerasv2

BetterNetWork4SSD

ssd_kerasv2's Issues

SSD300MobileNet predicts bad results

Hi there,

I tried to run the code in SSD.ipynb just like you have it: with the model ssd300MobileNet, and picking the weights you provide at https://drive.google.com/drive/u/0/folders/1F8GjD3BFhf_hv9Ipez0twRptYc3P8YwP (MobileNetSSD300weights_voc_2007.hdf5), and I get these results:

you can see they have nothing to do with the results you present.

Could you tell me what can be wrong? Are the weights the correct ones? Is the model working?

Thanks!

save and load the model

Hi, I have keras 2.2.0 TF 1.8 UBUNTU 18.04
I try to save the model using model.save('my_model_trained_ssd300.h5') and then to load it using

net_model = load_model("my_model_trained_ssd300.h5", custom_objects={'PriorBox': PriorBox,
       'DepthwiseConv2D': keras.layers.convolutional.DepthwiseConv2D,
       'relu6': relu6,
        'compute_loss': MultiboxLoss(NUM_CLASSES, neg_pos_ratio=2.0).compute_loss})  

but the following error happen. Any idea how to solve ?

in
'compute_loss': MultiboxLoss(NUM_CLASSES, neg_pos_ratio=2.0).compute_loss})
File "/usr/local/lib/python2.7/dist-packages/keras/engine/saving.py", line 261, in load_model
model = model_from_config(model_config, custom_objects=custom_objects)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/saving.py", line 335, in model_from_config
return deserialize(config, custom_objects=custom_objects)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/init.py", line 55, in deserialize
printable_module_name='layer')
File "/usr/local/lib/python2.7/dist-packages/keras/utils/generic_utils.py", line 145, in deserialize_keras_object
list(custom_objects.items())))
File "/usr/local/lib/python2.7/dist-packages/keras/engine/network.py", line 1036, in from_config
process_layer(layer_data)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/network.py", line 1022, in process_layer
custom_objects=custom_objects)
File "/usr/local/lib/python2.7/dist-packages/keras/layers/init.py", line 55, in deserialize
printable_module_name='layer')
File "/usr/local/lib/python2.7/dist-packages/keras/utils/generic_utils.py", line 147, in deserialize_keras_object
return cls.from_config(config['config'])
File "/usr/local/lib/python2.7/dist-packages/keras/engine/base_layer.py", line 1124, in from_config
return cls(**config)
TypeError: init() takes at least 2 arguments (1 given)

voc_2007.pkl file not found

Can you help me with this file?

l2_normalize() got an unexpected keyword argument 'axis'

i have updated keras from 2.1.2 to 2.1.5,when i try to run the videotest_example.py and occur this error,can u help me to solve this problem。thank u very much！！

Can't load SSD300MobileNet pretrained model

Hello there!
I'm trying to load the SSD300MobileNet using the SSD.ipynb script and the avaible MobileNetSSD300weights_voc_2007.hdf5 weights, but i always get a error regarding dimensions of one of the layers:

ValueError: Layer #42 (named "conv13_mbox_conf"), weight <tf.Variable 'conv13_mbox_conf_1/kernel:0' shape=(1, 1, 512, 126) dtype=float32_ref> has shape (1, 1, 512, 126), but the saved weight has shape (126, 1024, 1, 1).

Am I doing something wrong?

Thanks for your attention.

ssd300MobileNet layer concatenate error

when I use SSD.ipynb or SSD_Train.ipynb, (keras-2.3.1 or 2.1.5 both failed)
model=model = SSD(input_shape, num_classes=NUM_CLASSES) always cause Error as below:

ValueError: A Concatenate layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 1917, 4), (None, 1917, 21), (None, 114432, 8)]

error is caused by merge of the 3 nets at line #186 of ssd300MobileNet.py
net['predictions'] = concatenate([net['mbox_loc'],net['mbox_conf'],net['mbox_priorbox']],axis=2,name='predictions')

how to fix the shape mismatch of these 3 nets ?

You forgot prior_boxes_ssd512vgg16.pkl File

Please give me " prior_boxes_ssd512vgg16.pkl " file because you forgot . My Email ’ [email protected] ‘ Thank you very much

A `Concatenate` layer requires inputs with matching shapes

hello,i've used ssd300 model but it can't work.
the error code is
A Concatenate layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 38, 38, 512), (None, 19, 19, 1024), (None, 10, 10, 512), (None, 5, 5, 256), (None, 3, 3, 256), (None, 1, 1, 256)]

the exception stop at this line:
mbox_priorbox = concatenate([fusion_conv4_5_mbox_priorbox,fc7_mbox_priorbox,conv6_2_mbox_priorbox,conv7_2_mbox_priorbox,conv8_2_mbox_priorbox,pool6_mbox_priorbox],axis=1,name='mbox_priorbox')

and i changed another model but the same error happened.

tf verison is 1.13.1
keras version is 2.3.0
cv2 version is 4.1.1

Issues for taking different input shapes as an input into network.

solved!

Codes for generating the prior_boxes

It would be great to explain how the prior_boxes are generated or share the codes.

As Issue #5 mentioned, there is a need for modifying these boxes. For example, when you have images with different sizes, or the object you want to detect has a particular aspect ratio so you want to restrict to certain prior_boxes.

Also, from the pkl files for prior boxes, I don't see what the last four columns are used for.

preprocess_input

Hi,

in file SSD.ipynb code line
inputs = preprocess_input(np.array(inputs))

is it not mentioned the mode used by the keras library. Looking in keras source file this method need a mode that should be "caffe", "tf" or "torch". What is the correct one you used to train your weights ?

Below the keras library code showing the above

def preprocess_input(x, data_format=None, mode='caffe', **kwargs):

    """Preprocesses a tensor or Numpy array encoding a batch of images.
    # Arguments
        x: Input Numpy or symbolic tensor, 3D or 4D.
            The preprocessed data is written over the input data
            if the data types are compatible. To avoid this
            behaviour, `numpy.copy(x)` can be used.
        data_format: Data format of the image tensor/array.

        mode: One of "caffe", "tf" or "torch".
            - caffe: will convert the images from RGB to BGR,
                then will zero-center each color channel with
                respect to the ImageNet dataset,
                without scaling.
            - tf: will scale pixels between -1 and 1,
                sample-wise.
            - torch: will scale pixels between 0 and 1 and then
                will normalize each channel with respect to the
                ImageNet dataset.

Training

As I customed ssd for relatively log time.
I found it pretty hard to train by means of hyper parameter setting and sensitivity of architecture.
If your have environment to train then Upload your weights is very welcome let's make it better.

Could you share how to create the input dataset .pkl?

what's the definition/structure of input dataset .pkl file?
thanks for any reply.

Got error when training ssd300MobileNet with imagenet

Hi @tanakataiki ,
Thanks a lot for such a great work!
I am trying to train your ssd300MobileNet model with ImageNet dataset and got the following error:

ValueError: Error when checking target: expected predictions to have 3 dimensions, but got array with shape (32, 1000)

Do you have idea how to fix this issue? Is there any chance of missing Flatten layers before predictions layer (or after softmax)?
Best,
Hung

how to generate prior_boxes

Hi, may you explain how to generate the prior files ?
I would train using my own training set

Error in "assign_boxes" on training

I have error when i training (example from SSD_Train.ipynb) on my dataset

history = model.fit_generator(gen.generate(True), gen.train_batches,
                              nb_epoch, verbose=1,
                              #callbacks=callbacks,
                              validation_data=gen.generate(False),
                              nb_val_samples=gen.val_batches,
                              nb_worker=1)

Error

ssd_utils.py in assign_boxes(self, boxes)
    147                                                          :4]
    148         assignment[:, 4][best_iou_mask] = 0
--> 149         assignment[:, 5:-8][best_iou_mask] = boxes[best_iou_idx, 4:]
    150         assignment[:, -8][best_iou_mask] = 1
    151         return assignment

ValueError: shape mismatch: value array of shape (19,4) could not be broadcast to indexing result of shape (19,3)

Background class ID in VOC data

I see from the code that background_class_id is assigned to zero. However in the file https://github.com/tanakataiki/ssd_kerasV2/blob/master/get_data_from_XML.py the zero-th class index is assigned to Aeroplane class. How should I understand this?

Can't train with VOC2007 get error

Please help cant train get error
ValueError: shape mismatch: value array of shape (12,20) could not be broadcast to indexing result of shape (12,3)

training process is slow

i think class Generator runs on cpu ,maybe this is why the training process is so slow .
is there any solution ?
how many time is taken within one epoch on your computer ?
thanks

hello,the 'acc' of model ,it mean what???how compute??

Wrong Layer Choosen for SSD from MobilenetV2 backbone

The line here corresponds to the 12th block output.

The correct replacement for it is
FeatureExtractor = Model(inputs=mobilenetv2.input, outputs=mobilenetv2.get_layer('block_12_add').output)

Prior files for training on new datasets

I am wondering if the prior files are general enough to use with any new datasets, or they are specifically used for a specific dataset?

fit_generator is slow

cpu e5 2620
gpu 1080ti
tensorflow ver 1.8
keras 2.2.0
gpu 10%Usage rate
how to fix thanks~

BatchNormalization axis is probably incorrect.

In the MobileNet based models, only inside the function _depthwise_conv_block, you have

channel_axis = 1 if K.image_data_format() == 'channels_first' else -1

For all the rest functions, you used

channel_axis = 1

As a result, a few BatchNormalization layer has been renormalizing along the wrong direction. This potentially explains the low accuracy problems which others reported on SSD MobileNet

Use alpha parameter in MobileNet

Hi, I see in the code that the model MobileNet is using alpha parameter =1. Do you have any plan to generalize the model for any alpha defined by the user ?

Accuracy

Hi,
may you kindly communicate about the weights you provided what validation accuracy was reached ?

• Restarting the training using MobileNetSSD300 on PASCALVOC2012 loading the weights on DRIVE (PASCAL VOC2007) it start epoch 1 having loss=6,69 - acc=0,1921 - val_loss=3,97 - val_acc=0,062

• Is it correct that FEATURE FUSION after 4 epochs get val_acc =0,04 ?

SSDLite MobileNetV2

I ran into a problem while using your videotest_example.py for the ssd300MobileNetV2Lite model:
model = SSD(input_shape, num_classes=NUM_CLASSES)
leads to an error in ssd300MobileNetV2Lite.py
def SSD(input_shape, num_classes):
in line 118
FeatureExtractor=Model(inputs=mobilenetv2.input, outputs=mobilenetv2.get_layer('res_connect_12').output)
error:
ValueError: No such layer: res_connect_12

It seems like res_connect_12 is not a valid name inside the model. The MobileNetV2 model from Keras has indeed different layer names.
Does the videotest_example.py work for you with MobileNetV2?

Train models on non square images

I tried to adapt them to train them on my own dataset but my images are rectangle. I've got errors about the tensor shape with ssd300VGG16 and ssd300XceptionLite durting the fit_generator call.

I adapted the input shape constant with my own values (1224,370,3) and change the Xception call with xception=Xception(input_shape=(1224,370,3),include_top=False,weights=None) but i still got errors about the tensor shape.

Does anyone no how to modify the models and the train notebook to make the models work ?

Error while training with SSD Mobilenet

When I run the code for training with the SSD Mobilenet model, I get the following error:

ValueError: Shape must be rank 1 but is rank 4 for 'conv1_bn/cond/FusedBatchNorm' (op: 'FusedBatchNorm') with input shapes: [?,150,150,32], [1,150,1,1], [1,150,1,1], [1,150,1,1], [1,150,1,1]

for an input shape of (300,300,3)

Is there any change that would need to be made for rectifying this?

training PASCAL VOC 2012

I try to train PASCAL VOC 2012 (having 20 classes) getting the below error.
I used:

# some constants
NUM_CLASSES = 20
input_shape = (224, 224, 3)

Any idea how to solve ? The main strange think is that the last error line with the size of array change every times I run the code.

model_ssd_train_OLI.py:259: UserWarning: Update your `fit_generator` call to the Keras 2 API: `fit_generator(<generator..., 13700, 30, verbose=1, workers=1, validation_data=<generator..., callbacks=[<keras.ca..., validation_steps=3425)`
  nb_worker=1)
/home/alberto/.local/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py:100: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
Epoch 1/30
Traceback (most recent call last):
  File "model_ssd_train_OLI.py", line 259, in <module>
    nb_worker=1)
  File "/usr/local/lib/python2.7/dist-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1426, in fit_generator
    initial_epoch=initial_epoch)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training_generator.py", line 155, in fit_generator
    generator_output = next(output_generator)
  File "/usr/local/lib/python2.7/dist-packages/keras/utils/data_utils.py", line 793, in get
    six.reraise(value.__class__, value, value.__traceback__)
  File "/usr/local/lib/python2.7/dist-packages/keras/utils/data_utils.py", line 658, in _data_generator_task
    generator_output = next(self._generator)
  File "model_ssd_train_OLI.py", line 236, in generate
    y = self.bbox_util.assign_boxes(y)
  File "/home/alberto/Scrivania/example/ssd_kerasV2-master_OLI/ssd_utils.py", line 149, in assign_boxes
    assignment[:, 5:-8][best_iou_mask] = boxes[best_iou_idx, 4:]
ValueError: shape mismatch: value array of shape (33,22) could not be broadcast to indexing result of shape (33,3)