Issues about the patch embedding.

[jingyunliang]

Issues about the patch embedding.

[JingyunLiang]

Your idea deserves a try. To be honest, I didn't try it because I believe image restoration is a very local problem. We should process the image pixel by pixel (attention between pixels). From my intuition, there should have a performance drop if we process them patch by patch (attention between patches such as what IPT does). Imagine that the patch embedding process (e.g., using a strided convolution layer) is not invertible. The spatial information within a patch is lost, which may have extreme burden for the model to reconstruct pixels (pixel is our final goal) given a patch feature.

However, it can reduce the computation burden if we do so. You can try to use it and see the results.

[JingyunLiang]

Feel free to open it if you have more questions.