yoctta/multiple-attention

The code of multi-attention deepfake detection

multiple-attention's Issues

I'm looking forward to your sharing ^ .^

Was the images extracted from the same video organized in a file folder?

我看代码的读取过程，是需要把一个视频内的图像提取出来并放到文件夹内是吗

is the computation of aux_loss in code inconsistent with the paper？

In the code (MAT.py,274 line):
'''
if not jump_aux:
aux_loss,feature_matrix_d=self.auxiliary_loss(feature_maps_d,attention_maps_,y)
'''
it first compute the AttentionPooling between the feature_maps_d and attentions,then get feature_matrix and loss; but I think it should use the parameter feature_maps (not feature_maps_d in code)?
change it to be:
'''
if not jump_aux:
aux_loss,feature_matrix_d=self.auxiliary_loss(feature_maps,attention_maps_,y)
'''
I want to known wether I make a wrong understanding about the code?

dataset directory and requirement.txt

dear Author , whats the requirements of this project? and when we download that dataset and where should we put? should we make some folder? if yes, please guide me whats steps on my questions?

Training stuck for MAT

Hi @yoctta thanks for your contributions.
when I training MAT, The training is stuck, and it's not over. I checked a lot, but I didn't find the relevant information.




CPU shows running, Do you know what the problem is?

Minor issue at efficientnet/utils.py

@yoctta
we get
RuntimeError: expected scalar type Byte but found Float
in the Conv2dStaticSamePadding class' forward method w.r.t self.weights which are of type Parameter

please change the return statement in data/dataset.py getitem method's to

(self.trans(image=image)['image']).float()

to make it work.

I did this in an attempt to make it work, it worked as well, however if you have a better solution please fix it accordingly

meet a problem when running the code

Hi,
Thank you for sharing the code.
I run your code and meet a mistake as follows:

            File "/home/wj/60deepfake/github/multiple-attention-master/models/MAT.py", line 378, in forward
            x=self.pooling(feature_maps)
            ......
            File "/home/wj/anaconda3/envs/video/lib/python3.8/site-packages/torch/nn/functional.py", line 1129, in adaptive_avg_pool2d
            _output_size = _list_with_default(output_size, input.size())
            AttributeError: 'tuple' object has no attribute 'size'

I find that feature_maps is obtained from class 'Texture_Enhance_v2'.

        self.pooling=nn.AdaptiveAvgPool2d(1)
        self.texture_enhance=Texture_Enhance_v2(num_features,1)
        ......
        feature_maps=self.texture_enhance(feature_maps,(0.2,0.2))
        x=self.pooling(feature_maps)

The output of the class 'Texture_Enhance_v2' is a tuple ( "return feature_maps,feature_maps_d" ).
The pooling operation can not deal with tuple data.

Can the author help me and solve this problem?
Thank you!

Pretrained Network

Can the author share the pretrained network file ?

Has anyone obtained the results shown in the paper?

Could you please share the training process and techniques?

about evaluation.

Hi,

When I try to run the code of evaluation, I need to load the "config.pkl" as here shown:

multiple-attention/evaluation.py

Line 13 in bb069cc

with open('runs/%s/config.pkl'%name,'rb') as f:

I didn't find any evaluation config parameters in "config.py" except training parameters.
Could you give me some help?

preprocess lack some files

hi yoctta,
在谷歌盘盘上下载的slave_crop.py缺少一些依赖库，能否麻烦补充下，或者发一份给我([email protected])，多谢～

Example DFDC.json file

Can you share examples of the json files written in data.py?
Or maybe only DFDC.json would be fine.
Thanks in advance.

Anyone successfully run this ?

Has anyone been able to make this program work ?

multiple-attention-modify can run and train

Everyone, I refactored the current project. Although I don’t know how effective it is, the code can run and be trained.
https://github.com/yblir/multiple-attention-modify

模块代码

你好，论文中提到的重要模块的代码有给吗？

What address does the url indicate?

in main.py,
def pretrain():
name='Efb4'
url='tcp://127.0.0.1:27015'
What address does the url indicate?
thank you!

Inconsistent parameters between paper and code

Hi, thank you for your sharing code. I really appriciate your excellent work. I have read both the paper and your released code, but I found that some parameters described in the paper and that in the config setting of your code are inconsistent. For example, the batch size is set to 48 in the paper but 16 in the config code, the learning rate is start from 0.001 in the paper but 0.0001 in the config code, and so on. Since I want to correctly re-implement your method, I wonder which parameter I shoud use? By the way, I started the training from the pretrain() method of main.py, am I right?

dataset

您好，请问训练用的数据集能够提供吗？用官方的脚本下载不是很方便

minor mistake found in code

wrong variable name

multiple-attention/models/MAT.py

Line 175 in f4b3719

inter_calss_loss=inter_class_loss/M/self.alpha

Thanks for your share

Hey, @yoctta
Thanks for pretrained models.
I want to know what is the ckpt_35.pth?

Help

Ask for help using the environment

Running evaluation on FF++,DFDC,Celeb-DF

@yoctta please help us out,

1. I want to run the code in evaluation mode, however there is no celeb.json , i'm guessing this is for Celeb-DF-v1. Can you please outline steps how to evaluate the code?
2. Also could you please the exact file structure of your dataset directory and how to populate it?
3. There is no folder called runs with any config.pkl file.

P.S. : I just wan't to reload the shared weights and run inference on the test sets

Thanks

About the 4 time augment of real video in ffpp

In the training process, we augment the original frames 4 times for real/fake label balance.

Thanks for sharing your work. I noticed that you have augmented the real videos four times. Does it means that you have catch more frames in real videos even there are repeated frames in the final datasets？

About the tricks during the training

After the pretraining of backbone, I froze it and then train the multi-attentional without AGDA loss. But the accuracy is still no more than 97%. Is there any other trick used during the training of attention layer.

关于AGDA的问题

您好，大家都是**人，我就用中文表达了（/尴尬，怕英文表达不准确）
在AGDA.py中有一个 mod_func的函数，最后两行代码不知道代表啥数学意义，有看明白的大佬吗？
bottom=torch.sigmoid((torch.tensor(0.)-thres)*zoom)
return (torch.sigmoid((x-thres)*zoom)-bottom)/(1-bottom)

any guides to show how to use the code to train?

Hi, can you provide a guide to use the code? Thanks!

How to load the pretrained weights?

Thanks for your sharing. But I wanna to ask how to load the pretrained weights, e.g., "ff_c23.pth", as the "FF-checked.pkl" can't be loaded. BTW, could you please provide a simple demo for the evaluation? Thanks again!

Dataset used for training

Hi, thank you for open sourcing your code. I've read your paper and I really appreciate the attention based data augmentation implemented for the deep fake detection.
I do have one question though, in the paper you've mentioned that the FF++ HQ was used for training. And from the FF++ download, I see we have the folders c23 as being the HQ compression mode. While investigating the json file to replace with the folders in my own storage, I've noticed all your references are to the c0 folder, which correspond to the raw files. For the training, have you used the c0 or the c23 folders?

Json Decode Error

关于multi-attention

不好意思打扰一下，我目前正打算按照readme里面那样运行multi-attention，但是报错显示没有这些模块

请问这要怎么解决啊？

requirements

Hey,can you add the reqiurements of this project?I want to run it,Thank you!