pengzhou1108/rgb-n

rgb-n's Issues

Where to download NIST16

Hi I am doing an undergraduate project and I'm looking for the NIST16 dataset. Do you know where to download this?

Maybe COVER dataset as well. I have Columbia and CASIA

Thank you!

Zach

per-pixel F1 score

To calculate the per-pixel F1 score, a probability map is needed for generating a binary mask. However, the output of the model is just four bounding boxes and the picture's probability of being tampered with. Which phase during the training pipeline should I use to generate the probability map?

Dataset Description Needed.

Hey,

Please help me out by providing the dataset description particulary for the files.

train_filter.txt
train_filter_single.txt
test_filter.txt
test_filter_single.txt

Description for the files itself and the attributes in the txt file may help.

Thank you.

pretrained model

Hello,
Thank you for your great work!
I could not be able to download the datasets, could you please release the pretrained model (already fine-tuned), or send it to me for study?
Thank you very much!
My email: [email protected]

ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory

I have successfully setup the training script on my local machine and it was running perfectly fine. But when i setup same things on Google Cloud it start giving me this error.
ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory

Even though i have successfully installed cuda 9.0 and cudnn.

Is there anything missing or i didn't compile ?

Please check the details as well.

from nets.vgg16 import vgg16
File "/home/israr/project2/tools/../lib/nets/vgg16.py", line 16, in
from nets.network import Network
File "/home/israr/project2/tools/../lib/nets/network.py", line 18, in
from layer_utils.proposal_layer import proposal_layer
File "/home/israr/project2/tools/../lib/layer_utils/proposal_layer.py", line 13, in
from model.nms_wrapper import nms
File "/home/israr/project2/tools/../lib/model/nms_wrapper.py", line 12, in
from nms.gpu_nms import gpu_nms
ImportError: libcudart.so.8.0: cannot open shared object file: No such file or directory

train

COVER's training data set is only 75 pieces. Is it training together with other training sets or training alone? If it is only training COVER, how many generations does such a small data set train?

CPU->GPU Memcpy failed

hi ：
when i trained the model , after some steps , here is a problem

i have no idea about that can you help me.
thank you

about the synthetic dataset

when i run the code "demo.py" in " coco_synthetic" to creat the synthetic dataset,why i got the error:
AttributeError: 'Image' object has no attribute 'dtype'

Datasets

Hi,
Is there only 11,835 images in the dataset download link provided? I remember that the paper said that there is a 42K tampered and authentic image pairs.

Testing for single image

Hi Peng,
I have two questions for you.
Ques-1
I trained the model on Coco Synthetic dataset provided by you. I got the attached curve.Is it good enough?

Ques-2
While testing for single image , I have set the classes=2 as below.
net.create_architecture(sess, "TEST",2, tag='default',
anchor_scales=ANCHOR_SCALES,anchor_ratios=ANCHOR_RATIOS)

Is that correct??
I am getting the attached scores and boxes. Can you suggest me how to move forward after scores and boxes calculation

Thank you in advance.

raise ValueError("No variables to save")

When i run ./train_faster_rcnn.sh 0 coco res101_fusion EXP_DIR coco_flip_0001_bilinear_new, i meet the problem :
Solving...
/media/junjun/work/ICDE/detection/RGB-N-master/envir/lib/python3.5/site-packages/tensorflow/python/ops/gradients_impl.py:91: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
Loading initial model weights from /media/junjun/work/ICDE/detection/RGB-N-master/imagenet_weights/vgg16.ckpt
Traceback (most recent call last):
File "./tools/trainval_net.py", line 173, in
max_iters=args.max_iters)
File "/media/junjun/work/ICDE/detection/RGB-N-master/tools/../lib/model/train_val.py", line 356, in train_net
sw.train_model(sess, max_iters)
File "/media/junjun/work/ICDE/detection/RGB-N-master/tools/../lib/model/train_val.py", line 170, in train_model
restorer = tf.train.Saver(variables_to_restore)
File "/media/junjun/work/ICDE/detection/RGB-N-master/envir/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1000, in init
self.build()
File "/media/junjun/work/ICDE/detection/RGB-N-master/envir/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1021, in build
raise ValueError("No variables to save")

for help

Sorry to disturb you again，How to solve this problem？

unknown error while training

I got repeated unknown errors while running the RGBN training using the ‘train_faster_rcnn.sh’ patch file.
The machine is a google cloud instance with Ubuntu 16 as OS and here are the specs
Machine type :n1-standard-4 (4 vCPUs, 15 GB memory)
CPU platform: Intel Haswell
GPUs : 1 x NVIDIA Tesla T4
Zone :us-central1-a

After following the instructions in the readme file,
Cuda 8.0.44 cudnn 5.1 have been installed from Nvidia official website and Conda environment has been created with python 3.5, and the requirements that include TensorFlow GPU 0.12.1 have been installed.

The pre-trained resnet101 has been installed from this site
http://download.tensorflow.org/models/resnet_v1_101_2016_08_28.tar.gz

The dataset (train & test) mentioned in the readme file has been used also https://drive.google.com/open?id=1vIAFsftjmHg2J5lJgO92C1Xmyw539p_B

And here is the command we used to start training after modifying the train_faster_rcnn.sh ,factory.py
./train_faster_rcnn.sh 0 coco res101 EXP_DIR coco_flip_0001_bilinear_new

And here are the logs
unknown error
unknown error
unknown error
iter: 20 / 110000, total loss: 0.448014

rpn_loss_cls: 0.072341
rpn_loss_box: 0.004598
loss_cls: 0.371075
loss_box: 0.000000
lr: 0.001000
speed: 49.008s / iter
unknown error
unknown error
unknown error
logs.log

Training/testing split

Hi,

Is it possible to share the training/testing split txt files of nist and casia database? if not possible, just a few lines to understand the protocol.

Thank you in advance.

Training on datasets from the paper (NIST)

Hey peng,

I'm been able to train a model using the synthetic dataset you provided. I would like to use this pretrained model and train further using NIST2016. I believe this is the "Use synthetic pre-trained model for fine tuning" step. I'm using the NC2016_Test0601 folder which was provided in the download of NIS2016. I'm assuming we have to create the txt files similar to how they are in the provided synthetic dataset? Since it says I am missing them...

I can generate these files easily enough using the ground truth masks, but I'm confused about the format. For example...

In train_filter_single.txt you have:

Tp_75466_471421_31.84_62.29_640.0_251.24_airplane.png
bbox: [12, 66, 640, 368] (red)

Based on the code in split_train_test.py, it seems train_filter.txt is created by cropping the original image from coco and rescaling it to the aspect ratio of the new image. So this blue roi is padded to create the green bbox. Then the green bbox is rescaled to create the red one from above?

So I'm asking, which files are required to train using a dataset like NIST? Just the tampered images found in train_filter_single.txt? I'm assuming I would just use the masks and create bboxes for each image from NIST.

How to generate local noise feature using RGB input?

Sorry but I can't find the implementation code of the noise input using SRM filter?

I write one using pytorch but the output seem different from paper, here's the code.

import torch
from torch.nn.functional import conv2d
from torch.nn import Conv2d
import numpy as np


def srm_filter(x):
    q = [4.0, 12.0, 2.0]
    filter1 = [[0, 0, 0, 0, 0],
               [0, -1, 2, -1, 0],
               [0, 2, -4, 2, 0],
               [0, -1, 2, -1, 0],
               [0, 0, 0, 0, 0]]
    filter2 = [[-1, 2, -2, 2, -1],
               [2, -6, 8, -6, 2],
               [-2, 8, -12, 8, -2],
               [2, -6, 8, -6, 2],
               [-1, 2, -2, 2, -1]]
    filter3 = [[0, 0, 0, 0, 0],
               [0, 0, 0, 0, 0],
               [0, 1, -2, 1, 0],
               [0, 0, 0, 0, 0],
               [0, 0, 0, 0, 0]]
    filter1 = np.asarray(filter1, dtype=np.float32) / q[0]
    filter2 = np.asarray(filter2, dtype=np.float32) / q[1]
    filter3 = np.asarray(filter3, dtype=np.float32) / q[2]
    filters = np.asarray(
        [
            [filter1, filter2, filter3],    # R
            [filter1, filter2, filter3],    # G
            [filter1, filter2, filter3],    # B
        ], dtype=np.float32)

    output = conv2d(torch.from_numpy(x.astype(np.float32)), torch.from_numpy(filters), stride=1, padding=2)

    return output


if __name__ == '__main__':
    from PIL import Image

    raw_img = Image.open(
        r'/project/scene_classify/test_images/截屏2020-07-07 下午9.52.22.png').convert('RGB')

    noisemap = srm_filter((np.asarray(raw_img) / 255.0).transpose(2, 0, 1)[np.newaxis, ...])

    noisemap = Image.fromarray((noisemap.squeeze(dim=0).numpy() * 255.0).transpose(1, 2, 0).astype(np.uint8))
    noisemap.save('noise_map.png')
    raw_img.save('raw_map.png')

and here's the output:

the output from paper:

How to calculate the F1 score？

Hello, peng!
Sorry to disturb you.
In the paper you use F1 score to evaluate the model. I know the formula for calculating the F1 score. But what I get is the coordinates and score of the bounding box. How do I calculate the F1 score at the pixel level? I tried to calculate F1 score with the precision and recall from voc_eval.py. But I feel that this is not correct.
Please tell me how to calculate F1 score .
Thank you very much.

How to calculate the pixel-level AUC?

Hello, peng!
I don't understand the pixel-level AUC in your paper. Can you tell me how to calculate this metric? Thank you~

issue

I'd like to ask whether the use of sum pooled to compress spatial features in this paper will affect the accuracy without compression?

Using Resnet_fusion_noise model.

@pengzhou1108
I was trying to use the resnet_fusion_noise model to include the noise network as well. However I am facing Nan in summary histogram error after about 160 iterations. These are the training logs that come up on terminal:

I tensorflow/core/common_runtime/gpu/pool_allocator.cc:259] Raising pool_size_limit_ from 596 to 655
iter: 40 / 30000, total loss: nan
 >>> rpn_loss_cls: nan
 >>> rpn_loss_box: nan
 >>> loss_cls: 0.439838
 >>> loss_box: 0.000000
 >>> lr: 0.001000
speed: 8.585s / iter

What am I doing wrong? Am I using the wrong network?
All I did was change res101_fusion to res101_fusion_noise inside the train_dis_faster.sh script. Is there anything else I need to do?
please help. thank you.

The CASIA Dataset

Hi,

I'm currently working on reproducing the work of your paper. I pretrained the model with the synthetic datasets and I want to fine tune it with either CASIA or NIST. However, I generated the annotation of CASIA 2.0 dataset by comparing the Authentic and the Tampered images to label the differences. I reviewed most of them but still think there should be masks or ground truth labels for this datasets. Do you have the masks or the annotations of the dataset or related download links for that. Thank you so much.

Generate visual results

Hi，
when i trained and tested the synthetic dataset(as follows)

How can I generate visual results like this?
thank you

res101_fusion ckpt

HI @pengzhou1108 , May I ask where's the download link of the res101_fusion model?
I am following
'Train on synthetic dataset'
.....
'Step 3: Specify the dataset, gpu, and network in train_dist_faster.sh as below as run the file:
./train_faster_rcnn.sh 0 coco res101_fusion EXP_DIR coco_flip_0001_bilinear_new

build/sequential_batch_fft.so: undefined symbol _ZTIN10tensorflow8OpKernelE

How to solve this issue ?
tensorflow.python.framework.errors_impl.NotFoundError: RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/build/sequential_batch_fft.so: undefined symbol: _ZTIN10tensorflow8OpKernelE

Error:sequential_batch_fft.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv

Hi, when I run sh train_dist_faster.sh on Ubuntu 16.04&Python 3.6&tensorflow1.8, I got this:

+ set -e
+ export PYTHONUNBUFFERED=True
+ PYTHONUNBUFFERED=True
+ GPU_ID=0
+ DATASET=coco
+ NET=res101
+ array='0 coco res101 EXP_DIR coco_flip_0001_bilinear_new'
+ len=1
+ EXTRA_ARGS=
+ EXTRA_ARGS_SLUG=
+ case ${DATASET} in
+ TRAIN_IMDB=train_filter_single.txt
+ TEST_IMDB=test_filter_single.txt
+ STEPSIZE=40000
+ ITERS=110000
+ ANCHORS='[8,16,32,64]'
+ RATIOS='[0.5,1,2]'
++ date +%Y-%m-%d_%H-%M-%S
+ LOG=./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26
+ exec
++ tee -a ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26
+ echo Logging output to ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26
Logging output to ./logs/res101_train_filter_single.txt__res101.txt.2019-02-22_16-50-26
+ set +x
+ '[' '!' -f output/res101/train_filter_single.txt/default/res101_faster_rcnn_iter_110000.ckpt.index ']'
+ [[ ! -z '' ]]
+ python ./tools/trainval_net.py --weight /home/yihang/detect_ps/RGB-N/models/res101.ckpt --imdb /home/yihang/detect_ps/RGB-N/coco_synthetic/train_filter_single.txt --imdbval /home/yihang/detect_ps/RGB-N/coco_synthetic/test_filter_single.txt --iters 110000 --cfg cfgs/res101.yml --net res101 --set ANCHOR_SCALES '[8,16,32,64]' ANCHOR_RATIOS '[0.5,1,2]' TRAIN.STEPSIZE 40000
Traceback (most recent call last):
  File "./tools/trainval_net.py", line 25, in <module>
    from nets.resnet_v1_noise import resnet_noise
  File "/home/yihang/detect_ps/RGB-N/tools/../lib/nets/resnet_v1_noise.py", line 25, in <module>
    from compact_bilinear_pooling.compact_bilinear_pooling import compact_bilinear_pooling_layer
  File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/compact_bilinear_pooling.py", line 6, in <module>
    from .sequential_fft import sequential_batch_fft, sequential_batch_ifft
  File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/__init__.py", line 1, in <module>
    from .sequential_batch_fft_ops import sequential_batch_fft, sequential_batch_ifft
  File "/home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/sequential_batch_fft_ops.py", line 10, in <module>
    'build/sequential_batch_fft.so'))
  File "/home/yihang/anaconda3/envs/py36tf13/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
    lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: /home/yihang/detect_ps/RGB-N/tools/../lib/compact_bilinear_pooling/sequential_fft/build/sequential_batch_fft.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv

Anyone can help?

Checkpoints for pretrained model

Hi,

I was wondering if anyone has checkpoints for pretrained models of this repo by any chance and wants to share it?

Filtered roidb entries to 0

when I train the dataset，gives the error tips

And I fix the problem, because roidb is filtered.

Why the roidb is filtered? Thanks

no file:sequential_batch_fft_kernel.cu.o

where is this file: g++: error: sequential_batch_fft_kernel.cu.o

COVER's training data set is only 75 pieces. Is it training together with other training sets or training alone? If it is only training COVER, how many generations does such a small data set train?

Code will not run AWS

Hello, has anyone here been able to get the code to run on AWS? I have tried symlinks from newer versions to older versions but no success.

issue when I train on synthetic dataset

Hi!
I got a problem when I run the order "./train_faster_rcnn.sh 0 coco res101_fusion EXP_DIR coco_flip_0001_bilinear_new"
`
Traceback (most recent call last):
File "./tools/trainval_net.py", line 111, in
imdb, roidb = combined_roidb(args.imdb_name)
File "./tools/trainval_net.py", line 82, in combined_roidb
roidbs = [get_roidb(s) for s in imdb_names.split('+')]
File "./tools/trainval_net.py", line 82, in
roidbs = [get_roidb(s) for s in imdb_names.split('+')]
File "./tools/trainval_net.py", line 75, in get_roidb
imdb = get_imdb(imdb_name)
File "./RGB-N/tools/../lib/datasets/factory.py", line 77, in get_imdb
return __setsname
File "./RGB-N/tools/../lib/datasets/factory.py", line 26, in
__sets[name] = (lambda split=split, year=year: pascal_voc(split, year))
File "./RGB-N/tools/../lib/datasets/pascal_voc.py", line 42, in init
self._image_index = self._load_image_set_index()
File "./RGB-N/tools/../lib/datasets/pascal_voc.py", line 85, in _load_image_set_index
'Path does not exist: {}'.format(image_set_file)

AssertionError: Path does not exist: ./RGB-N/data/VOCdevkit2007/VOC2007/ImageSets/Main/trainval.txt
`
Do you have any idea how to fix this? THANKS

Columbia dataset mask

Hi ,thank you for your useful repository!
May I ask how to process the color mask of the Columbia dataset into a black and white binary mask?
Best Wishes!

the FP of authentic image

hello, peng
I have some questions while I learn your model. I tested fifty authentic coco images with your model , the confidence of the highest score box of twenty-four images is above 0.5, I think the situation as the FP of authentic image . In our application scenario, we hope the probability of authentic image detected as tampered image can be as low as possible. I have tried many methods to drop the FP, however all failed .I don't know if you have considered this problem，and how can I handle with the problem?
Looking forward to your reply ， thank you!