frederickszk/lrnet

Landmark Recurrent Network: An efficient and robust framework for Deepfakes detection

lrnet's Issues

文件路径问题

学长好，下载的特征点检测库文件具体应该放在哪个路径下？
在运行extract_landmarks.py时会出现错误RuntimeError: Unable to open shape_predictor_68_face_landmarks.dat，发现是没有下载shape_predictor_68_face_landmarks.dat导致。
由于不知道具体放到dlib的那个文件下，我尝试更改代码中的路径为绝对路径
predictor = dlib.shape_predictor("C:/Users/wang/Downloads/shape_predictor_68_face_landmarks.dat")依旧显示
RuntimeError: Unable to open C:/Users/wang/Downloads/shape_predictor_68_face_landmarks.dat运行错误。
麻烦学长了。

Error on tracking

with any file except examples I've got error on tracking faze: numpy.linalg.LinAlgError: Singular matrix

The performance problem after adding other FF++ fake dataset.

@meihsuan0301
原来commit引用会把原来的问题关掉 😢
原回答在这里：
#17 (comment)
我新开了一个issue，如果有相关问题欢迎继续在这儿留言~ 😄

地标提取

学长好，为什么运行地表提取的部分时会直接退出，而没有输出到landmark的文件内。
Settings: NOT visualize the extraction results.
Extract landmarks from sample_fake.mp4.
Detecting:
100%|██████████| 371/371 [01:18<00:00, 4.74it/s]
0%| | 0/370 [00:00<?, ?it/s]Tracking
100%|██████████| 370/370 [00:02<00:00, 184.66it/s]
进程已结束,退出代码-1066598274 (0xC06D007E)
麻烦学长了，万分感谢。

地标提取

学长好
我想问一下landmarks里的地标文件和visualize里的文件是运行extract_landmarks.py得到的吗?
我阅读reademe之后用FF++里的一个视频去作为输入运行extract_landmarks.py但是没有得到地表文件，这是为什么呢？
谢谢学长。

training

Hello, I would like to ask, how many landmarks are used for each video during training? Is all the landmarks extracted from each frame of each video used for training? Is it continuous frames? If not, it is discarded Or continue to use/whether to take the first N frames/whether to randomly take N frames. That is to say, during training, what are the rules for selecting landmarks from each video?

About paper

HI, here are some qusetions about paper.

1.The predicted point with a large difference between its original point and backward
LK point will be discarded (dotted arrows). so how can we predict it ? the blue circle.

2.Usually deepfake videos are more noise , so is the denoising step also used for fake videos? Is there influence to classify fake videos?

3.Because you only show the results of part of the best-performance methods.So ff++'s testing data 99.9 is only using (DF)?
It not average of （DF、NT、FS、F2F）? but other methods's acc are average of （DF、NT、FS、F2F）?

Please correct me if I have misunderstood!! thank you so much~~

landmarks

For facilitating the training and evaluation of LRNet, we would release the processed landmark datasets gradually.你好请问这个在哪里下载呢

demo中的landmarks extraction部分是不是有问题

使用extract_landmarks.py提取后分类结果如下：

使用原始的数据分类后结果如下：

questions regarding training

Hello,

Why is ur dataset composed of txt files not image files?
Can you use image files (ex. FF++ c40 DF,F2F..) as an input instead of txt files?

Released FF++ landmark dataset

您好，想請問現在您提供的FF++ landmark dataset是用什麼方法提取的呢?
我在Paper上是看到使用Dlib，在demo跟calibrator資料夾的readme檔中又有看到使用OpenFace, BlazeFace, RetinaFace幾種方法，因此有點不清楚目前最新版本的dataset是以什麼方法做的，希望您能回覆我，感激不盡!

training

Hello, I saw in the article that each video is divided into segments with a fixed length of 60, and when the FPS is 30, the sum of these segments is 2 seconds. How to set this up in code.

训练

1. Hello, may I ask, are all the settings mentioned in the paper? In preprocessing step, we adopt Dlib [14] to carry out face
   and landmark detection (another detector Openface[4] is adopt in the ablation study). In classification procedure, Each RNN in our two-stream network is bidirectional and Consists of GRU (Gated Recurrent Unit) whose number of output units is set to be k = 64. And two fully-connected layers with the number of units to be 64 and 2 are connected to RNN layer's output. A dropout layer with drop rate dr1 = 0.25 is inserted between input and RNN, and another 3 dropout layers with dr2 = 0.5 are used to separate the rest of the layers. These settings are partially based on existing reserach results [26]. In addition, we adopt an 8:2 dataset split, i.e., 80% for training and 20% for testing. Each video is segmented into clips with a fixed length of 60, which sum up to 2 seconds when the fps is 30. We adopt Adam optimizer with lr = 0.001, and batch size is set to be 1024. This classification model will be trained up to 500 epochs.
2. Regarding the question of epoch, the epoch parameters are passed to the model, and in the end is it used in iteration or epoch?Why val_acc fluctuates so much？
3. G1+g2 is to train g1 first and then train g2. After training g1, load the weight of g1 and continue to train g2, or what is the setting of this piece?

关于UADFV数据集获取

学长好！看了你的论文后我受益匪浅。请问你能不能提供一下实验中使用的UADFV数据集呢，我从网上没找到下载方法。

如果方便的话，可以将数据集发到[email protected]中；如果不方便的话，请学长帮忙将申请方法告诉我一下。

再次感谢！

FF++和Celeb-DF数据集问题

学长你好，请问您那还有没有FF++和Celeb-DF数据集视频提取的人脸帧图片，如果有的话能不能给我发一份，我的邮箱是[email protected]

demo/model_weights/ff/

Hello. I don't quite understand how you got LRNet/demo/model_weights/ff/g1.h5/g2.h5
I would like to create my own *.h5 files, but I don't understand how to do it

提取人脸地标

您好，感谢您提供的详尽说明。

我正在按照您提供的方案尝试生成自定义的数据集的landmark，但是发现通过Openface提取landmark得到csv中的frame数目，和原视频的frame数不匹配。请问您有遇到过这个问题吗？应该如何解决呢。

编码错误

你好！我运行代码出现了编码错误的问题，我只查到说是文件读写的方式有问题，但是代码里面也没看到有什么打开文件的命令😂。大佬您能给出一些建议吗？

关于sample-level

学长您好，我想请问下代码中evaluate.py模块的sample-level片段，为什么不是mix/2 和 1 进行比较，而是直接mix 和 1去比较，没怎么理解，可以麻烦您解答一下下吗？

怎么生成自定义的数据集 how to conduct my own dataset

Output not good with example movie

Fllow the reference, i got the output:

they all predict to real.
Is there somewhere I operate wrong?

The performance problem after adding other FF++ fake dataset.

Hi, I would like to ask if you have tried adding other FF++ fake datasets? Because I tried adding other fake datasets to train, but the model performance will hardly exceed 0.7 in AUC during training (both g1 and g2), and it's even worse during testing. Don't know if you have the same situation or not?

Celeb-DF数据集问题

十分感谢您的工作！不论是您的论文、代码还是Issues都给了了我很大的帮助和启示！

我想要询问的是，在论文中的Table2提到，研究使用了celeb-df数据集进行测试，里面包括了raw，c23，c40这几种版本的celeb-df数据集进行测试。我下载了raw的celeb-df数据集并提取人脸关键点进行了测试，但在c23，c40的视频压缩方面存在一些困难，您是使用FF++提供的代码进行数据压缩的嘛，还是运用了什么方法？如果方便的话，您能否帮忙提供提取好的celeb-df数据集？

再次感谢，期待您的答复。

关于demo中的classify.py

您好！请问如何将classify.py所用的模型改为pytorch的

About Kalman Filter

学长您好，首先非常感谢您（们）的工作，我从中学到了很多。卡尔曼滤波器部分我有几个问题，希望得到您的指点。
Q1：LK操作的固有方差Q = 0.3 是如何得到的。

Q2：为什么近似相对方差approximate relative variance 能够用来大致描述探测点的方差？

Q3: 我看论文及代码中的式子，更像是我看网上教程时刚开始的一维向量的例子。但是这里一个点的坐标应该时一个二维的，方程的形式应该不一样吧？

Q4：为什么会想到用Kalman Filter 来做这项工作，灵感来自哪里？

I can't reproduce the paper results.

I use the code in demo and try some fake video from ff++, but the performance I got is not good. Many fake videos will be identified as real.(I use ff g1.h5 and g2.h5 in model_weights.)
Is there somewhere I operate wrong?

数据集相关

大佬您好，请问C23和C40的视频是如何生成的？

关于OpenFace

请问您在用OpenFace提取landmark的时候是在Windows系统下还是在Ubuntu呢？因为我看到您在之前问题里提到的OpenFace图形界面好像只有windows下能用，在ubuntu下是不是需要下载openface源码再进一步处理呢？

About train

Hi，
There is no training code in the source code, can you provide the training code? Thank you very much.

数据集

学长好，想问一下你有没有FF++的数据集 可不可以分享一下 我这里不知道什么原因下载不下来
谢谢学长！

提取人脸地标

您好！我看了您其他问题的回复，你的意思是校准地标的代码现在是在calibrator文件夹下了，不在demo里了是吗。我看了calibrator文件夹下的readme，里面说openface提取人脸地标的代码，要自己实现，您的代码里面没有给，是吗？

编码错误

你好！我运行代码出现了编码错误的问题，我只查到说是文件读写的方式有问题，但是代码里面也没看到有什么打开文件的命令😂。大佬您能给出一些建议吗？

Strange request for you

Hello. I have a rather strange request for you. May I ask you to remove training/model_tf.py and other references to TensorFlow from your repository for one week?
It is quite difficult for me to explain why I need this, but it will help me a lot in my studies.