yoctta / multiple-attention Goto Github PK
View Code? Open in Web Editor NEWThe code of multi-attention deepfake detection
The code of multi-attention deepfake detection
我看代码的读取过程,是需要把一个视频内的图像提取出来并放到文件夹内是吗
In the code (MAT.py,274 line):
'''
if not jump_aux:
aux_loss,feature_matrix_d=self.auxiliary_loss(feature_maps_d,attention_maps_,y)
'''
it first compute the AttentionPooling between the feature_maps_d and attentions,then get feature_matrix and loss; but I think it should use the parameter feature_maps (not feature_maps_d in code)?
change it to be:
'''
if not jump_aux:
aux_loss,feature_matrix_d=self.auxiliary_loss(feature_maps,attention_maps_,y)
'''
I want to known wether I make a wrong understanding about the code?
dear Author , whats the requirements of this project? and when we download that dataset and where should we put? should we make some folder? if yes, please guide me whats steps on my questions?
Hi @yoctta thanks for your contributions.
when I training MAT, The training is stuck, and it's not over. I checked a lot, but I didn't find the relevant information.
CPU shows running, Do you know what the problem is?
@yoctta
we get
RuntimeError: expected scalar type Byte but found Float
in the Conv2dStaticSamePadding class' forward method w.r.t self.weights which are of type Parameter
please change the return statement in data/dataset.py getitem method's to
(self.trans(image=image)['image']).float()
to make it work.
I did this in an attempt to make it work, it worked as well, however if you have a better solution please fix it accordingly
Hi,
Thank you for sharing the code.
I run your code and meet a mistake as follows:
File "/home/wj/60deepfake/github/multiple-attention-master/models/MAT.py", line 378, in forward
x=self.pooling(feature_maps)
......
File "/home/wj/anaconda3/envs/video/lib/python3.8/site-packages/torch/nn/functional.py", line 1129, in adaptive_avg_pool2d
_output_size = _list_with_default(output_size, input.size())
AttributeError: 'tuple' object has no attribute 'size'
I find that feature_maps is obtained from class 'Texture_Enhance_v2'.
self.pooling=nn.AdaptiveAvgPool2d(1)
self.texture_enhance=Texture_Enhance_v2(num_features,1)
......
feature_maps=self.texture_enhance(feature_maps,(0.2,0.2))
x=self.pooling(feature_maps)
The output of the class 'Texture_Enhance_v2' is a tuple ( "return feature_maps,feature_maps_d" ).
The pooling operation can not deal with tuple data.
Can the author help me and solve this problem?
Thank you!
Can the author share the pretrained network file ?
Could you please share the training process and techniques?
Hi,
When I try to run the code of evaluation, I need to load the "config.pkl" as here shown:
multiple-attention/evaluation.py
Line 13 in bb069cc
hi yoctta,
在谷歌盘盘上下载的slave_crop.py缺少一些依赖库,能否麻烦补充下,或者发一份给我([email protected]),多谢~
Can you share examples of the json files written in data.py
?
Or maybe only DFDC.json
would be fine.
Thanks in advance.
Has anyone been able to make this program work ?
Everyone, I refactored the current project. Although I don’t know how effective it is, the code can run and be trained.
https://github.com/yblir/multiple-attention-modify
你好,论文中提到的重要模块的代码有给吗?
in main.py,
def pretrain():
name='Efb4'
url='tcp://127.0.0.1:27015'
What address does the url indicate?
thank you!
Hi, thank you for your sharing code. I really appriciate your excellent work. I have read both the paper and your released code, but I found that some parameters described in the paper and that in the config setting of your code are inconsistent. For example, the batch size is set to 48 in the paper but 16 in the config code, the learning rate is start from 0.001 in the paper but 0.0001 in the config code, and so on. Since I want to correctly re-implement your method, I wonder which parameter I shoud use? By the way, I started the training from the pretrain()
method of main.py
, am I right?
您好,请问训练用的数据集能够提供吗?用官方的脚本下载不是很方便
wrong variable name
multiple-attention/models/MAT.py
Line 175 in f4b3719
Hey, @yoctta
Thanks for pretrained models.
I want to know what is the ckpt_35.pth?
Ask for help using the environment
@yoctta please help us out,
P.S. : I just wan't to reload the shared weights and run inference on the test sets
Thanks
In the training process, we augment the original frames 4 times for real/fake label balance.
Thanks for sharing your work. I noticed that you have augmented the real videos four times. Does it means that you have catch more frames in real videos even there are repeated frames in the final datasets?
After the pretraining of backbone, I froze it and then train the multi-attentional without AGDA loss. But the accuracy is still no more than 97%. Is there any other trick used during the training of attention layer.
您好,大家都是**人,我就用中文表达了(/尴尬,怕英文表达不准确)
在AGDA.py中有一个 mod_func的函数,最后两行代码不知道代表啥数学意义,有看明白的大佬吗?
bottom=torch.sigmoid((torch.tensor(0.)-thres)*zoom)
return (torch.sigmoid((x-thres)*zoom)-bottom)/(1-bottom)
Hi, can you provide a guide to use the code? Thanks!
Thanks for your sharing. But I wanna to ask how to load the pretrained weights, e.g., "ff_c23.pth", as the "FF-checked.pkl" can't be loaded. BTW, could you please provide a simple demo for the evaluation? Thanks again!
Hi, thank you for open sourcing your code. I've read your paper and I really appreciate the attention based data augmentation implemented for the deep fake detection.
I do have one question though, in the paper you've mentioned that the FF++ HQ was used for training. And from the FF++ download, I see we have the folders c23 as being the HQ compression mode. While investigating the json file to replace with the folders in my own storage, I've noticed all your references are to the c0 folder, which correspond to the raw files. For the training, have you used the c0 or the c23 folders?
Hey,can you add the reqiurements of this project?I want to run it,Thank you!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.