if i only train the model 3 ,can i realize the effect in the paper?

[knazeri]

if i only train the model 3 ,can i realize the effect in the paper?

[knazeri]

@Lvhhhh
mode=4 is what we initially did and our paper and initial results are based on the joint optimization of G1 and G2. However, in our later experiments, we really didn't find a significant improvement over fixing G1 parameters (mode=3) and that's why when we released the code, we mode=3 the final stage. The other reason is that mode=4 slows down training drastically compared to mode=3.

If you don't want to train the edge model, then you can directly jump to stage 3, that is mode=3.

[knazeri]

@Lvhhhh The problem is your mask dataset. Almost entire input space is masked in your mask dataset. Honestly, I'm surprised that the network can create even the blurry result!

Please use this Irregular Mask Dataset for training. Also make sure you change the loss values to match the values we defined in our paper:

STYLE_LOSS_WEIGHT: 250 
CONTENT_LOSS_WEIGHT: 0.1
INPAINT_ADV_LOSS_WEIGHT: 0.1

[Lvhhhh]

thank you for your answer! i train this mode=3 but i found the color of result may be wrong. here is the result .

[Lvhhhh]

the training picture is here. why the picture is so blur?

[Lvhhhh]

here is the config.yml
MODE: 1 # 1: train, 2: test, 3: eval
MODEL: 3 # 1: edge model, 2: inpaint model, 3: edge-inpaint model, 4: joint model
MASK: 3 # 1: random block, 2: half, 3: external, 4: (external, random block), 5: (external, random block, half)
EDGE: 1 # 1: canny, 2: external
NMS: 1 # 0: no non-max-suppression, 1: applies non-max-suppression on the external edges by multiplying by Canny
SEED: 10 # random seed
GPU: [0] # list of gpu ids
DEBUG: 0 # turns on debugging mode
VERBOSE: 0 # turns on verbose mode in the output console

TRAIN_FLIST: ./place_train.list
VAL_FLIST: ./place_val.list
TEST_FLIST: ./place_val.list

TRAIN_EDGE_FLIST: ./datasets/places2_edges_train.flist
VAL_EDGE_FLIST: ./datasets/places2_edges_val.flist
TEST_EDGE_FLIST: ./datasets/places2_edges_test.flist

TRAIN_MASK_FLIST: ./qd_train.list
VAL_MASK_FLIST: ./qd_test.list
TEST_MASK_FLIST: ./qd_test.list

LR: 0.0001 # learning rate
D2G_LR: 0.1 # discriminator/generator learning rate ratio
BETA1: 0.0 # adam optimizer beta1
BETA2: 0.9 # adam optimizer beta2
BATCH_SIZE: 1 # input batch size for training
INPUT_SIZE: 256 # input image size for training 0 for original size
SIGMA: 2 # standard deviation of the Gaussian filter used in Canny edge detector (0: random, -1: no edge)
MAX_ITERS: 2e6 # maximum number of iterations to train the model

EDGE_THRESHOLD: 0.5 # edge detection threshold
L1_LOSS_WEIGHT: 1 # l1 loss weight
FM_LOSS_WEIGHT: 10 # feature-matching loss weight
STYLE_LOSS_WEIGHT: 1 # style loss weight
CONTENT_LOSS_WEIGHT: 1 # perceptual loss weight
INPAINT_ADV_LOSS_WEIGHT: 0.01 # adversarial loss weight

GAN_LOSS: nsgan # nsgan | lsgan | hinge
GAN_POOL_SIZE: 0 # fake images pool size

SAVE_INTERVAL: 1000 # how many iterations to wait before saving model (0: never)
SAMPLE_INTERVAL: 1000 # how many iterations to wait before sampling (0: never)
SAMPLE_SIZE: 12 # number of images to sample
EVAL_INTERVAL: 0 # how many iterations to wait before model evaluation (0: never)
LOG_INTERVAL: 10 # how many iterations to wait before logging training status (0: never)

[Lvhhhh]

the reason is no sufficient training or some other problem? thank you for your time

[knazeri]

@Lvhhhh
Your training sample is not loading completely! However, judging by the partial image, it looks like you are using a very sparse training mask dataset! We used this Testing Irregular Mask Dataset for our training which contains 12,000 irregular masks. Make sure you have the right mask dataset.
Also, in your configuration file, make sure to change the value of STYLE_LOSS_WEIGHT from 1 to 250. That's the value we used and mentioned in our paper.
Let us know the results after these changes.

[cmyyy]

@knazeri ,the value of CONTENT_LOSS_WEIGHT and INPAINT_ADV_LOSS_WEIGHT also need to change to 0.1,right?

[Lvhhhh]

@knazeri here is the training sample . the picture is blurred .what is the problem?

[knazeri]

Thanks @cmyyy just fixed it 826f2b8

[shumile66]

hello，I have a problem to ask to you .What is " random block, 2: half, 3: external, 4: (external, random block), 5: (external, random block, half) " meaning?

[shumile66]

@knazeri I have a question about mask (1:random block, 2: half, 3: external, 4: (external, random Block), 5: (External, random block, half)), why there are so many options, but I don't understand each meaning.