Can model be defined by nn.Sequential or does it need to use nn.ModuleList

[alecda573]

Can model be defined by nn.Sequential or does it need to use nn.ModuleList

[Crazylov3]

You couldn't use shortcut if you use nn.Sequential :)

[alecda573]

@Crazylov3 hey thanks for responding so quickly! Can you explain what is the purpose of the shortcut. So there is no way to replicate shortcut by using the sequential class?

[Crazylov3]

The main purpose of the shortcut (also known as skip connection) in deep neural networks is to help with the flow of information and improve gradient flow during training. In particular, it helps to address the problem of vanishing gradients, which can occur when training very deep neural networks. The idea behind the shortcut is to create a direct connection between the input and output of a block of layers, allowing information to flow directly from one layer to another without having to pass through several intermediate layers. This can help to preserve information and gradients as they propagate through the network, which can lead to more stable and efficient training.

[alecda573]

@Crazylov3 so it seems in the config files the shortcut appears after two consectutive conv layers, can this be replaced in the config files by Bottleneck block seen in yolov7 repo here: https://github.com/WongKinYiu/yolov7/blob/main/models/common.py
and then one could use nn.Sequential inplace of nn.ModuleList?

[Crazylov3]

In general, you can use shortcut everywhere you want. If you have only 1 configs, it will easy to implement shortcut in a block, then you can put these block into nn.Sequential(). However, in this case, there are a lot of configs file, using nn.Sequential() for all may be hard in implementation term. The purpose of their implementation is resuse the code (1 source code for all configs)