Computing forward&backward cycle loss twice

[vanhuyz]

Computing forward&backward cycle loss twice

[vanhuyz]

Actually, there are 2 generators G and F which are trained separately. Each generator loss is a sum of GAN loss and cycle loss.

[theflashsean1]

Yes, but isn't cycle loss for G (which takes x as input, and produces G(x) as output) "forward_loss = tf.reduce_mean(tf.abs(F(G(x))-x))"

And cycle loss for F "tf.reduce_mean(tf.abs(G(F(y))-y))"

When you defined "cycle_loss = self.cycle_consistency_loss(self.G, self.F, x, y)" on line 72, I thought you've already written this loss as sum of two cycle losses for each generator

[vanhuyz]

The original paper and PyTorch implementation optimize F and G simultaneously, so there is only 1 objective function for 2 generators:

L(G, F) = L_gan(G) + L_gan(F) + L_cycle

# optimize
AdamOptimizer(L(G, F))

But my implementation is based on the Torch implementation. If I understand correctly, loss of G and loss of F are computed separately as follows:

# compute losses
L_forward: x → G(x) → F(G(x)) ≈ x
L_backward: y → F(y) → G(F(y)) ≈ y
L_cycle = L_forward +  L_backward

L(G) = L_gan(G) + L_cycle
L(F) = L_gan(F) + L_cycle

# optimize separately
AdamOptimizer(L(G))
AdamOptimizer(L(F))

[theflashsean1]

I see what you mean now. Both cycles need to be taken into account for each generator