RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.

[davda54]

RuntimeError: Trying to backward through the graph a second time, but the saved intermediate results have already been freed. Specify retain_graph=True when calling backward the first time.

[davda54]

Hi, you should call the forward pass output = model(images) again before criterion(output, target).backward()

[XieZixiUSTC]

@davda54 thanks,it work

[1215481871]

Sorry, I got the similar question. I actually did the full second forward pass but also reported "Trying to backward through the graph a second time (or directly access saved variables after they have already been freed). Saved intermediate values of the graph are freed when you call .backward() or autograd.grad(). Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved variables after calling backward."

Here is my code:
output = backbone(imgs)
loss = self.criterion(output,labs)
backbone_opt.zero_grad()
loss.backward()
backbone_opt.first_step(zero_grad=True)

        output2 = backbone(imgs)
        loss2 = self.criterion(output2,labs)
        loss2.backward()
        backbone_opt.second_step(zero_grad=True)

But if I substitute "output2 = backbone(imgs)" with "output2 = backbone(imgs.detach())", it works.
Does this implementation make sense or I also made some mistakes here?