Out of memory issue

[allenai]

Out of memory issue

[ibeltagy]

As far as I can tell, this is expected PyTorch behavior and is not specific to Longformer. Every time you call output = model(input_ids, attention_mask=attention_mask), PyTorch stores more of the activations computed during the forward pass to use them in the backward pass.
To avoid that, you need to encode your code in with torch.no_grad(): your_code. You might also want to do model.eval() if you are only doing inference. Check here for the difference between the two.

[taherehmarvdashti]

Thank you. Solved my problem