models 'mobilenetv2_w1_bsconvs' and ''mobilenetv2_w1' are identical

[zeiss-microscopy]

models 'mobilenetv2_w1_bsconvs' and ''mobilenetv2_w1' are identical

[sunzhe09]

me too

[zeiss-mamthor]

Hi,
as already discussed in issue #4 (bullet point 1) and described in the paper (Sect. 4.2), we retain the layer arrangement of the original model.
However, even though for this specific case the layers are the same, the BSConvS variant uses the additional regularization loss which causes the improved results. This is not shown when printing the model.

[sunzhe09]

@zeiss-mamthor I want to replace the MobileNetV2 bottleneck ，should I replace DepthwiseConv with BSConvS ?or just replace invert_residual with BSConvS？I ‘m a little confused

[zeiss-mamthor]

Following the implementation in the paper, the entire linear bottleneck block (not only the DW conv) is replaced by a special form of a BSConvS module, where the layer cascade of PW-PW-DW is shifted to match the original MobileNet design of PW-DW-PW. The reason is that in the MobileNet paper it was shown that residuals in the bottleneck provide a better accuracy/FLOP ratio compared to residuals outside of the bottleneck (as implemented in ResNets). We keep these findings for our models, i.e., converting a MobileNetV2 into its BSConvS variant consists of adding the regularization loss to the last PW conv of each bottleneck block while keeping the overall layer design.
You can simply use the transformer for MobileNetV2/3 which adds the required function to the corresponding modules or just load an already converted model:

import bsconv.pytorch
model = bsconv.pytorch.get_model('cifar_mobilenetv2_w1_bsconvs', num_classes=100)

Btw, do not forget to add the regularization loss during training:

output = model(images)
loss = criterion(output, target)

# THIS LINE MUST BE ADDED, everything else remains unchanged
loss += model.reg_loss(alpha=0.1)