About Res5ROIHeads

[RyanXLi]

About Res5ROIHeads

[mandal4]

I have a same question too.

[er-muyue]

In fact, there are no special considerations for this design. I believe that whether to use FPN does not affect the performance improvement of the proposed method, but it may affect the performance upper bound.

I have not done more experiments on FPN features, if you have some interesting conclusions, please let me know.

[Wei-i]

dear author, hi, I have tried to implement the FPN config using gdlayer.
Here is my core code:

# init
      # fpn setting
      self.affine_rpn_dict = {}
      self.affine_rcnn_dict = {}
      for k in backbone.output_shape().keys(): # [p2, p3, p4, p5, p6]
          self.affine_rpn_dict[k] = AffineLayer(num_channels=backbone.output_shape()[k].channels, bias=True)
          self.affine_rcnn_dict[k] = AffineLayer(num_channels=backbone.output_shape()[k].channels, bias=True)

# forward 
features_de_rpn = {k: self.affine_rpn_dict[k](decouple_layer(features[k], self.rpn_backward_scale)) for k in features}
features_de_rcnn = {k: self.affine_rcnn_dict[k](decouple_layer(features[k], self.roi_heads_backward_scale)) for k in features}

However, there is a cuda-cpu-mismatch error in gdl.py .

class AffineLayer(nn.Module):
    def __init__(self, num_channels, bias=False):
        super(AffineLayer, self).__init__()
        weight = torch.FloatTensor(1, num_channels, 1, 1).fill_(1)
        self.weight = nn.Parameter(weight, requires_grad=True)

        self.bias = None
        if bias:
            bias = torch.FloatTensor(1, num_channels, 1, 1).fill_(0)
            self.bias = nn.Parameter(bias, requires_grad=True)
        
    def forward(self, X):
        
        # out = X * self.weight.expand_as(X) 
        out = X * self.weight.expand_as(X).cuda() # if not .cuda(),  X is on cuda(), while self.weight is on cpu().
        if self.bias is not None:
            out = out + self.bias.expand_as(X).cuda() # if not .cuda(),  X is on cuda(), while self.biasis on cpu().
        return out

When I add .cuda(), the code can execute.
However, I found that the training time became much slower about 5 times.

[Wei-i]

This issue has been solved by putting weight and bias to 'cuda' so that it became much faster.
I am sorry for my poor previous code.
Thanks.

class AffineLayer(nn.Module):
    def __init__(self, num_channels, bias=False):
        super(AffineLayer, self).__init__()
        weight = torch.FloatTensor(1, num_channels, 1, 1).fill_(1).to('cuda')
        self.weight = nn.Parameter(weight, requires_grad=True).to('cuda)

        self.bias = None
        if bias:
            bias = torch.FloatTensor(1, num_channels, 1, 1).fill_(0)
            self.bias = nn.Parameter(bias, requires_grad=True)
        
    def forward(self, X):
        
        # out = X * self.weight.expand_as(X) 
        out = X * self.weight.expand_as(X)
        if self.bias is not None:
            out = out + self.bias.expand_as(X)
        return out

[chenf99]

@Wei-i Hi, have you got an improvement with FPN? I'm curious about this, thank you!

[MartinYYYYan]

@chenf99 I have tried to add more affine layers. But there is no significant improvement.

[MartinYYYYan]

@Wei-i How is your network performance? Is it works?

[Ze-Yang]

I tried R101-FPN with either one shared affine layer or 5 independent affine layers each for ['p2', 'p3', 'p4', 'p5', 'p6'] of FPN respectively. I also remove the additional cls_head and use the original box_head. All other settings remain the same as R101-C4. However, the results drop a lot, e.g., from 66.21 to 55.29 (voc split1 10shot). Anybody encountered a similar issue?

[Ze-Yang]

This issue has been solved by putting weight and bias to 'cuda' so that it became much faster. I am sorry for my poor previous code. Thanks.

class AffineLayer(nn.Module):
    def __init__(self, num_channels, bias=False):
        super(AffineLayer, self).__init__()
        weight = torch.FloatTensor(1, num_channels, 1, 1).fill_(1).to('cuda')
        self.weight = nn.Parameter(weight, requires_grad=True).to('cuda)

        self.bias = None
        if bias:
            bias = torch.FloatTensor(1, num_channels, 1, 1).fill_(0)
            self.bias = nn.Parameter(bias, requires_grad=True)
        
    def forward(self, X):
        
        # out = X * self.weight.expand_as(X) 
        out = X * self.weight.expand_as(X)
        if self.bias is not None:
            out = out + self.bias.expand_as(X)
        return out

I have tried with similar implementation with yours. It seems that there is no need to explicitly put the weights to cuda, as the whole model will be moved to cuda device finally with self.to(self.device). I notice significant performance drop with FPN. Do you have the similar issue?

[MartinYYYYan]

I had a similar observation, and I think the performance drop might be caused by the incompatibility between FPN and channel-wise GDL.  Best  DYan  

…

------------------ 原始邮件 ------------------ 发件人: "er-muyue/DeFRCN" ***@***.***>; 发送时间: 2022年7月9日(星期六) 下午2:22 ***@***.***>; ***@***.******@***.***>; 主题: Re: [er-muyue/DeFRCN] About Res5ROIHeads (#5) This issue has been solved by putting weight and bias to 'cuda' so that it became much faster. I am sorry for my poor previous code. Thanks. class AffineLayer(nn.Module): def __init__(self, num_channels, bias=False): super(AffineLayer, self).__init__() weight = torch.FloatTensor(1, num_channels, 1, 1).fill_(1).to('cuda') self.weight = nn.Parameter(weight, requires_grad=True).to('cuda) self.bias = None if bias: bias = torch.FloatTensor(1, num_channels, 1, 1).fill_(0) self.bias = nn.Parameter(bias, requires_grad=True) def forward(self, X): # out = X * self.weight.expand_as(X) out = X * self.weight.expand_as(X) if self.bias is not None: out = out + self.bias.expand_as(X) return out I have tried with similar implementation with yours. It seems that there is no need to explicitly put the weights to cuda, as the whole model will be moved to cuda device finally with self.to(self.device). I notice significant performance drop with FPN. Do you have the similar issue? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>