Conditioning on y_cond

[PouriaRouzrokh]

Conditioning on y_cond

[Janspiry]

Hi, thanks for this great question, and I think there are two potential considerations for this:

1. Keep the consistency between training and inference over all tasks. The model samples from random noise and y_cond in the inference stage.
2. y_cond can distinguish between the mask and unmasked areas since y_t may not be straightforward enough when t is small.

[PouriaRouzrokh]

Hi, thanks for this great question, and I think there are two potential considerations for this:

1. Keep the consistency between training and inference over all tasks. The model samples from random noise and y_cond in the inference stage.
2. y_cond can distinguish between the mask and unmasked areas since y_t may not be straightforward enough when t is small.

Thanks for the kind reply. This makes sense, though it is worth trying the second reason. I will post here if I realized something different.

[Janspiry]

Feel free to reopen the issue if there is any question.

[vinodrajendran001]

@PouriaRouzrokh I have opened a separate issue on it. But I am in urgent need of a solution, so I just wanted to check with you .

In my inpainting case, during the inference only the y_cond and mask images are given. In that case, may I know how to do a inference?

In the network.py script, for the inpainting task the below line will be executed as part of the restoration function. As y_0 is None for me, I am not sure how to deal with this line. If I skip the below line then the results are very bad (just only some whitish kind of image is generated). Also, in the Process.png image I can notice that for each step the noise level is increasing rather than decreasing.

if mask is not None:
    y_t = y_0*(1.-mask) + mask*y_t

Any idea on how to proceed?

[yc-cui]

@Janspiry Why not just set the mask as y_cond? for consistency among all tasks?