fnzhan/rabit

Bi-level feature alignment for versatile image translation and manipulation [ECCV 2022]

rabit's Issues

about some bugs in you code

Hi zhan, I appreciate your work so much! However , I met some bugs when I tried to use your code.

1、models/networks/correspondence.py (line 385)

       elif self.opt.warp_mask_losstype == 'cycle':
            # f_div_C_v = F.softmax(f_WTA.transpose(1, 2), dim=-1)
            f_WTA_v = f.transpose(1, 2)
            f_div_C_v = f_WTA_v / f_WTA_v.sum(-1).view(-1, N, 1)

            seg = F.interpolate(seg_map, scale_factor=1 / self.down, mode='nearest')
            channel = seg.shape[1]
            seg = seg.view(batch_size, channel, -1)
            seg = seg.permute(0, 2, 1)
            warp_mask_to_ref = torch.matmul(f_div_C_v, seg)  # 2*1936*channel
            warp_mask = torch.matmul(f_div_C, warp_mask_to_ref)  # 2*1936*channel
            warp_mask = warp_mask.permute(0, 2, 1).contiguous()
            coor_out['warp_mask'] = warp_mask.view(batch_size, channel, feat_height, feat_width)  # 2*3*44*44

'f' and 'f_div_C' is not defined

2、models/networks/generator.py (line 47)

     self.conv_img1 = nn.Conv2d(1, seg, 3, padding=1)
     self.up = nn.Upsample(scale_factor=2)

'seg' is not defined

would you please fix these bugs?

about non-differentiable top-k

Nice Works!

Being very interested in your work，But I found that your top-k ranking removes the differentiable top-k method and uses softmax instead, what is the reason for this ?

关于semantic position encoding(SPE)的问题

您好！
对您的工作非常感兴趣，仔细阅读您的工作后发现您代码中使用的是coordconv全局坐标，而非semantic position encoding(SPE)，暂时没看到SPE相关代码，有代码更新更好，万分感谢！

about bi-level warp function "align_feature" and "align_feature_v"

dear author,
I cannot understand the bi-level warp function "align_feature" and "align_feature_v". The align_feature_v seems like the inverse function of the align_feature. Can you explain more about them, especially about details of bi-level warp and its inverse function. Thank you very much!

about the validation data split

Dear author,
Sorry, I'm a beginner in this field. To my knowledge, data will be splited into train, validation and test set, and we can calculate the FID score on validation data for model selection during training. However, I didn't find the validation set split and related code in the RABIT, as well as CoCosNet, which makes me confused. Thanks in advance.