Out of memory... how to increase memory?

[sniklaus]

Out of memory... how to increase memory?

[JohnTravolski]

A small modification you could make to your approach: have the quadrants overlap and blend the overlapping regions. Still not the best solution but it should hide obvious seams between the quadrants.

My code did something very similar to your suggestion. The quadrants do overlap, but they aren't blended together at the end; the overlapping region is just cut off. This prevented the seams, but I didn't try it on any videos with fast motion.
Also, I shouldn't have said quadrants either, because the code can divide the source frames any integer number of times (into fourths, ninths, sixteenths, etc.).

[sniklaus]

Yes, you would have to get a graphics card with more memory. Alternatively, you could choose to not use the graphics card at all by removing the .cuda() calls. Keep in mind though, that not using a graphics card will increase the inference time.

[deama]

Ah, I see. What should I replace the .cuda() calls to, to get the CPU to work instead? Also, would 8GB of VRAM be enough for 1080p images or how much would you need roughly?

[sniklaus]

You can just remove them. However, I just remembered that you will have to provide a CPU implementation for the adaptive convolution as well.

https://github.com/sniklaus/pytorch-sepconv/blob/26b411e7776c444303e667d128ab0ad14a8439c2/sepconv/sepconv.py#L115

[JohnTravolski]

You can just remove them. However, I just remembered that you will have to provide a CPU implementation for the adaptive convolution as well.

https://github.com/sniklaus/pytorch-sepconv/blob/26b411e7776c444303e667d128ab0ad14a8439c2/sepconv/sepconv.py#L115

I don't understand the code well enough to write my own cpu implementation for this. The only way I could get this to work for higher resolution images was to slice the images into quadrants and upscale each slice separately and then stitch them back together. See this code.
https://pastebin.com/SBPxnQd0
I'm sure there's a better way, but that was the only way I could interpolate a 4k image sequence with 4GB of VRAM. This method probably won't work if there's too much motion but it works fine when there's not. Does anybody have a better recommendation?

[sniklaus]

A small modification you could make to your approach: have the quadrants overlap and blend the overlapping regions. Still not the best solution but it should hide obvious seams between the quadrants.