tancik/stegastamp

Invisible Hyperlinks in Physical Photographs

stegastamp's Issues

How to fine tune the model?

How to fine tune the model through the trained CKPT model

decode valueError issues

When I run decode python code on both images and videos, it comes to this problem.
Traceback (most recent call last):
File "decode_image.py", line 64, in
main()
File "decode_image.py", line 57, in main
bitflips = bch.decode_inplace(data, ecc)
ValueError: invalid parameters
I am not familar with bch, so how can I solve this?

The encoded image can only be 400*400？

ValueError: Cannot feed value of shape (1, 200, 200, 3) for Tensor 'input_hide:0', which has shape '(?, 400, 400, 3)'
How to achieve the encoded image pixels and the encoded pixels consistent？

How to convert encoded_image from tensor to numpy ？

The encoded_image in the network has always been operated in the form of a tensor. When I want to make some changes to the noise layer, I find that I cannot directly assign a value to the tensor, so I want to convert the encoded_image to the form of an array first. I use sess.run and .eval() both failed. Is there any solution? Thank you very much! ! ! ! ! !

feeding a new natural image without secret

I am wondering when you feed a natural image without residual secret into the decoder network.

what result will the decoder produce?

Output size

The ouuout images are 400x400. IS possible to change this output to a higher quality?

Encode messages of different lengths.

Where can i change the length of the message.

A new question

Have you implemented this algorithm in other deep learning frameworks, such as pytorch, caffe ?

Need Guidance to implemented on camera scanning

Hi,

Thanks for this awesome library but can you please guide me how you have implemented this on the camera scanning
If possible can you show some example code as when we are taking the images from the camera it doesn't work

lpips_tf missing?

In trying to train I got this error (tensorflow1.15)

models.py", line 4, in
import lpips_tf
ModuleNotFoundError: No module named 'lpips_tf'

Tried to fix with
https://github.com/alexlee-gk/lpips-tensorflow but didn't solve the problem.

What am I missing?

Training problems

Time problem of training model

hello,if I have 10 pictures of 400X400, How long does it take for you to train the model. It took me a long time to train the model.

Is the loss of the discriminator negative?

Hello, I have been studying your paper recently. I don't quite understand the role of discriminator, because I find that G_loss is used in total loss, so what is the role of D_loss? And is the loss of discriminator negative? Thank you very much!

How high can ‘bit_acc’ reach without BCH code？

Hello, tancik!

1. Excuse me, if you don't use BCH code, what is the decoding accuracy (bit_acc)?

2. Every 12 bytes (96 bits), only 5 bits can be corrected, right?

• In encode_image.py and decode_image.py, The BCH parameters you use are:

  BCH_POLYNOMIAL = 137
  BCH_BITS = 5
  

• According to the bchlib source code, my understanding is,
  The entire packet can only be corrected by 5 bits, right?
  In your case, packet is 12 bytes (96 bits), data is 7 bytes (56 bits, embedded link length)

Detector dataset

Thank you for your amazing work.

What is the dataset used in training the detector?

Problem with parameter setting .

What does YUV_scales stand for? Why did you set y,u, and v to 1,100,100, respectively?
How do I adjust these three parameters?

Many thanks

could you share training parameters?

I try to train model, bug could not get the same performance like the released model.
Could you share training parameters or told me how to adjust parameters.

Thanks.

How many images in your training dataset?

How many images in your training dataset?

Why StegaStamp can only encode 56bits (7 characters) with ECC?

Dear tancik:

I got the following error when running encode_image.py:

Error: Can only encode 56 bits (7 characters) with ECC.

I'm confused about why it can only encode 56 bits (7 characters) with ECC.

Thanks in advance! 😄

Detecting and decoding an image

Hi everyone,
I am trying to add a image switch to decode.py file to detect and decode the hidden physical images. Anyone has done it before or any suggestions ?

Hello, author, I want to train a model with the height of 1200 and the width of 800

Do you need to change some parameters？I changed part of the code, but it is still not right, I wonder if it is a parameter problem。

what's the parameter 'borders' means?

Hello!
in build_model, borders control the generation of encoded image. what's the 'borders' mean? what's the different between 'no_edge' 'black' 'random' 'white' and 'image'

The model size is extremely large T_T

I trained the original model without any modification, and got a model which can work well. But the model size is too big, about 1.1GB, while the supplied pretrained model is 217MB.
I just wonder if the pretrained model has discard some weights? Like the discriminator network's weights?

Some questions about making detector dataset

Hello, tancik !
I am keen on your outstanding work. But when I was traing a detector for distiguishing the encoded images from complex background (not totally white), I have some trouble about making dataset. Here is my questions:

1. Are the encoded images placing on totally white background in the traing dataset for detector?

2. If the encoded image's size is small (maybe 1/100 of an A4 paper), how can I make a training dataset to get great precision ?

   Finally, I have written a documentation to guide chinese novices to build an environment and turn your code from tensorflow 1.x to tensorflow 2.x. I hope you will like it and it can help more githubers to run your work.

The full detector model

Could you guys give the original detector model please?
I mean the one you released is not robust.

Error code < TypeError: join() argument must be str or bytes, not 'NoneType' > while training

I have cloned the project from github and installed the requirements after than i have specified the "TRAIN_PATH" but when i try to launch the train.py i am facing with error code:

Traceback (most recent call last):
  File "C:/Users/yavuz/PycharmProjects/Tensorflowpytorch/StegaStamp/train.py", line 224, in <module>
    main()
  File "C:/Users/yavuz/PycharmProjects/Tensorflowpytorch/StegaStamp/train.py", line 151, in main
    writer = tf.summary.FileWriter(join(LOGS_Path,EXP_NAME),sess.graph)
  File "C:\Users\yavuz\AppData\Local\Programs\Python\Python37\lib\ntpath.py", line 115, in join
    genericpath._check_arg_types('join', path, *paths)
  File "C:\Users\yavuz\AppData\Local\Programs\Python\Python37\lib\genericpath.py", line 149, in _check_arg_types
    (funcname, s.__class__.__name__)) from None
TypeError: join() argument must be str or bytes, not 'NoneType'

Process finished with exit code 1

Note:
I have changed the /// parser.add_argument('exp_name', type=str) /// to /// parser.add_argument('--exp_name', type=str) ///
Because i was also having an error in other way.

I have read all the paper and wanted to try it but having hard time on debugging. Because i do not have much experience on python.

Waiting for your help.

What is the function of the "border" parameter？

Hello, I don't quite understand the function of the parameter "border", can you explain it？

Contract work

Hi, are you available for contract work ? I need this tech for a commercial project.

does tensorflow 1.13.2 have problem?

my test environment:

docker pull tensorflow/tensorflow:1.13.2-gpu-py3

and I found the lastest saved model

root@3ec62ce01961:~/StegaStamp# du -sm saved_models/7/*
208     saved_models/7/saved_model.pb
0       saved_models/7/variables

but the released model is

(tf1.13) [phoenixkiller@ultrapower stegastamp_pretrained]$ du -sm *
1	saved_model.pb
207	variables

Does I got pb model?
And I still could not get the performance same as release model.
Should I change tensorflow version to 1.13.1?

About low-frequency pictures and shot

Watermark steganography is bad for low-frequency pictures. Is it because my parameters have not been adjusted? When I deal with black-and-white document images, the visual effect of watermarks is very poor. How to adjust? The watermark cannot be extracted from the picture taken by the mobile phone.

can this model handle random size image input

as title

how to improve str_acc

hi, can you give me some advise about:
1 how to improve str acc?
I use win10 + tf cpu to train a model with the same image dataset you offered.

perhaps the iterate does not reach its maximum num(14w, now is about 1.9w)

2 and are the parameters in parser.add_argument(in train.py) correct? I need some details about these paras, to make sure whether I am on the right way training model.

3 can you tell me the ideal about these tf encode/decode/discriminator models, why do we need these 10 or 5 CNN layers? and why can these models achieve a good performance?

4 and can you tell me the BCH code? in code, you use BCH_POLYNOMIAL = 137, BCH_BITS = 5. this BCH code need 127bits(92bits are valid bits that we can hide data). But in parper, you use 100bits(96bits+4bits) which can recover 56 bits data.

many thanks!

Which parameter or loss is the most important for the decodability of the printed image?

Hello, @tancik
Did you find out which parameter affects decodability of the printed image the most when doing experiments?
I tried to train a new model with some image which is not natural, but the some of the printed images can't be decoded.

according to bash scripts/base.sh EXP_NAME，Model test after training

After training the saved PB model, when a picture is encrypted, printed and photographed, it cannot be decrypted

Problem with parameter

Is the picture distortion just controlled by lpips_loss_scale ？
G_loss_scale,l2_loss_scale, What does G,l2 stand for?

Many thinks

save_video

Hello guys. could anyone tell me what kind of file form should I create for save_video FILENAME?

Error: the following arguments are required: exp_name

When I run this project, spyder told me: train.py: error: the following arguments are required: exp_name,I am new for this, so I how to deal with it? Thanks(#^.^#)

if this is an unexpected mistake in writing？

Hello, tancik
I wonder if this is an unexpected mistake in writing？
"Accounting for error correcting, using only 50 total message bits would drastically reduce the number of possible encoded hyperlinks to under one billion"

The paper said that ”use 100 bits to hide 56 bits“,
so if we use 50 bits, it can hide ” under one billion“ ？ really？

look forward to your reply. Many Thanks！

Project robust to photos taken on the screen?

import lpips.lpips_tf failed.

Failed in models.py line 3: import lpips.lpips_tf.
No module named 'lpips_tf'.

test_vid.mp4

Where is test_vid.mp4？

The problem of str_acc compution.

Hello, when I first read the source code, I had this problem.
The source code of computing str_acc and bit_acc is the follows:
''
def get_secret_acc(secret_true,secret_pred):
with tf.variable_scope("acc"):
secret_pred = tf.round(secret_pred)
correct_pred = tf.count_nonzero(secret_pred - secret_true, axis=1)
str_acc = 1.0 - tf.count_nonzero(correct_pred - tf.to_int64(tf.shape(secret_pred)[1])) / tf.size(correct_pred, out_type=tf.int64)
bit_acc = tf.reduce_sum(correct_pred) / tf.size(secret_pred, out_type=tf.int64)
return bit_acc, str_acc
''
why the line 4 is tf.count_nonzero(secret_pred - secret_true, axis=1)？ This seems to compute the error bit number of each batch. However, the str_acc and bit_acc use it as the correct number. I think it maybe a typo, but the traning log of tensorborad shows a reasonable result.

dataset

Could you send the dataset to me ? I cannot get the data.

how to reducer the params for fc layer in decoder?

hi @tancik
since the fc layer in decoder is too larger (13*13 * 128 * 500 = 10,816,000), so i want use a conv(dim=1024) to increase the dimension and then use tf.reduce_mean(x, axis=[1, 2]), but in the first 1w iterations, the bit_acc always gets around 0.5, and the str_acc gets zero. so how to reduce the params for the fc layer, at the same time, it can get good results?

How could I transform my picture like what displayed in the paper?

bitflips always has value -1

When I run the code to my own video
python detector.py
--detector_model detector_models/stegastamp_detector
--decoder_model saved_models/stegastamp_pretrained
--video test_vid.mp4
I found bitflips always has value -1
The following is what I did：
step 1: Hide message into image lena and verified the message can be decoded
step 2: Print the image with message inside
step 3: Record a video of the printed image
And then, I found the bitflips always has value -1, could you help me with this problem. Thank you.
The image and video can be find here.[https://drive.google.com/open?id=1XGXWCN78kDvr6IEtTyIQKB6JJ9jXvTok]

Savedmodel generate empty variables directory

Excuse me,sir.Can you help me? I try to use the natural images (100000 images) from Set and retrain the model,but don’t know why the generated model's variables directory is empty and saved_model.pb file is really big ,and I try a lot ,many say it's the reason that the graph is "frozen" (i.e., all variables are converted to constant nodes in the graph), so there are no "variables" left in the computation and hence the directory is empty.I don't know how to solve this,and I really appreciate for your help.
the code to generate model is below:
with tf.Session(graph=tf.Graph()) as session: tf.import_graph_def(constant_graph_def, name='') tf.saved_model.simple_save(session, SAVED_MODELS + '/' + EXP_NAME, inputs={'secret': secret_pl, 'image': image_pl}, outputs={'stegastamp': deploy_hide_image_op, 'residual': residual_op, 'decoded': deploy_decoder_op})

problem about the STN module

Hi,did you do the ablation study of the STN module of the decoder? I find it doesn't play an important role for improving decoding accuracy. Furthermore, the parameters of the STN cost too much space. The model size decrease from 200M to 50M after removing this module.

Retraining detector

I did some testing and found that it requires a white border around the image containing the hidden code for the detector to work. For my use-case, I need the detector to work even without the white border. So I will have to retrain the detector.

For that I am following the other github repo but it seems to be causing some issues with the input size of model.

Has anyone tried this?

The theory of the random_blur_kernel function?

Hi, I can't understand the motion blur kernel, which is implemented in random_blur_kernel function.

StegaStamp/utils.py

Line 8 in 2e852ee

def random_blur_kernel(probs, N_blur, sigrange_gauss, sigrange_line, wmin_line):

Is there any paper or other materials which can help me understand it? It seems that it uses gauss function and makes a direction to simulate motion blur. I search Linear Motion Blur, and it's hard to find something related to this. Hope someone who know this can give me some references.