Is torchtext mandatory？

[zdyshine]

Is torchtext mandatory？

[0x3f3f3f3fun]

Can you please provide a more detailed explanation of your question? 😺

[zdyshine]

No description provided.

Thanks for the reply!
There is torchtext in my local environment. When using DiffBIR and adapting to the inference environment, I found that pytorch_lightning has a dependency on torchtext, /opt/conda/lib/python3.8/site-packages/pytorch_lightning/utilities/apply_func.py. You can also run the code without torchtext installed. I uninstalled torchtext, and finally used the provided sample code to successfully run the results, but the quality of the recovered images was very poor, and I don't know why.

The degradation of the input is not very complex, but the results are strange.

[0x3f3f3f3fun]

This issue is unrelated to torchtext. Your result is reasonable and falls under DiffBIR's bad cases. Currently, the stage1 model produces over-smoothed results, causing details like text and small faces to be erased in the first stage, making it impossible for the second stage to perform generation. Additionally, the image you've shown appears to be from a low-resolution video with video compression artifacts. In our experiments, we have also observed that DiffBIR tends to generate some noisy textures when dealing with such compression noise. These are known issues with DiffBIR at the moment, and we are actively working on improvements to enhance its performance. Thank you for bringing this issue to our attention!

[0x3f3f3f3fun]

You can improve DiffBIR's performance by tuning inference parameters, such as:

• set sr_scale to a value larger than 1 to alleviate the "over-smoothed" problem.
• set prompt guidance scale to a value larger than 1 to enable the negative prompt, which can reduce bad cases and improve the image quality.

Due to the stochastic nature of the diffusion model, you can also change the random seed to select a good result for yourself (we will never perform this action in our experiments!).

We hope these suggestions are helpful for you. Here are two examples with video compression artifacts:

[zdyshine]

You can improve DiffBIR's performance by tuning inference parameters, such as:

• set sr_scale to a value larger than 1 to alleviate the "over-smoothed" problem.
• set prompt guidance scale to a value larger than 1 to enable the negative prompt, which can reduce bad cases and improve the image quality.

Due to the stochastic nature of the diffusion model, you can also change the random seed to select a good result for yourself (we will never perform this action in our experiments!).

We hope these suggestions are helpful for you. Here are two examples with video compression artifacts:

Thanks for the reply, I will try to adjust the parameters

[zdyshine]

When I installed xformers==0.0.16, it prompted that torch requires 1.13. Have you encountered such a problem?

[0x3f3f3f3fun]

Yes, I've encountered this problem as well. xformers has strict requirements for both PyTorch and CUDA versions. I recommend you to create a separate environment as shown in the README.md and then install the PyTorch version that matches with xformers.