The drop connect rate (aka survival rate) is incorrect

[xhluca]

The drop connect rate (aka survival rate) is incorrect

[xhluca]

Practically it does perform better than official. -_- .

I've only observed better performance in one case so I'm not sure it generalizes. In that case, the improved performance does indicate that extreme low survival rates (<0.3) might be a good regularization approach.

[darcula1993]

I check the drop rate per block and it looks fine:

block1a_ 1.0
block1b_ 0.9875
block1c_ 0.975
block2a_ 0.9625
block2b_ 0.95
block2c_ 0.9375
block2d_ 0.925
block2e_ 0.9125
block3a_ 0.9
block3b_ 0.8875
block3c_ 0.875
block3d_ 0.8625
block3e_ 0.85
block4a_ 0.8375
block4b_ 0.825
block4c_ 0.8125
block4d_ 0.8
block4e_ 0.7875
block4f_ 0.775
block4g_ 0.7625
block5a_ 0.75
block5b_ 0.7375
block5c_ 0.725
block5d_ 0.7124999999999999
block5e_ 0.7
block5f_ 0.6875
block5g_ 0.675
block6a_ 0.6625
block6b_ 0.6499999999999999
block6c_ 0.6375
block6d_ 0.625
block6e_ 0.6125
block6f_ 0.6
block6g_ 0.5874999999999999
block6h_ 0.575
block6i_ 0.5625
block7a_ 0.55
block7b_ 0.5375
block7c_ 0.5249999999999999

[xhluca]

@darcula1993 I'm confused. Shouldn't the block rate be at ~0.8 for the final block since the drop_connect_rate is 0.2 by default?

[xhluca]

So it turns out I pasted different values. However the problem remains as indicated.

[darcula1993]

        for j in range(round_repeats(args.pop('repeats'))):
            # The first block needs to take care of stride and filter size increase.
            if j > 0:
                args['strides'] = 1
                args['filters_in'] = args['filters_out']
            x = block(x, activation_fn, drop_connect_rate * b / blocks,
                      name='block{}{}_'.format(i + 1, chr(j + 97)), **args)
            b += 1

I check the code and seems that b can be greater than num of blocks,not sure why.

[xhluca]

I've observed the same thing as well.

[fmbahrt]

Qubvel's implementation does not calculate the total number of blocks correctly for configurations larger than B0.

[innat]

Practically it does perform better than official. -_- .

[innat]

Well, I'm not sure, maybe I need to look again properly. In fact, I spent almost a week assuming that there probably some problem with my data loader using the official efficient-net. But when I use non-official implementation, it was just fine.