About swish activation

[apple]

About swish activation

[chenbinghui1]

When training classification model, model.activation.name is Swish while then training detection model, it is changed to relu?

ml-cvnets/config/detection/ssd_coco/mobilevit_v2.yaml

Lines 70 to 82 in 84d992f

	classification:
	name: "mobilevit_v2"
	mitv2:
	width_multiplier: 2.0
	attn_norm_layer: "layer_norm_2d"
	activation:
	name: "swish"
	normalization:
	name: "sync_batch_norm"
	momentum: 0.1
	activation:
	name: "relu"
	inplace: false

Is it correct? and why?

[sacmehta]

Hi,

model.activation.name is the default activation function. If we don’t specify anything specific to classification or any backbone, this activation function is used. If we specify a particular activation function for any backbone (Swish in this case for MobileViTv2), we override it while building model. See here:

ml-cvnets/cvnets/models/classification/__init__.py

Line 45 in 84d992f

gen_act_fn = getattr(opts, "model.activation.name", "relu")

Hope this helps.

[chenbinghui1]

@sacmehta Thanks for quick reply~ I got it.