chisenzhang/phantom

repository for dreamoving-phantom https://www.modelscope.cn/studios/vigen/DreaMoving_Phantom/summary. DreaMoving-Phantom is a general and automatic image enhancement and super resolution framework.

Phantom

DreaMoving-Phantom is a general and automatic image enhancement and super-resolution framework, which can be applied to images of various scenes and qualities. With the demo we provide, users just need to upload a low-quality image to generate an enhanced image with one click. No need to choose the sr model or adjust parameters. Now you can try our demo at modelscope.

The quality of enhancement and functions of this project are being continuously optimized and expanded. We also welcome developers to continue to develop and contribute to this Repo.

Examples

input (left) and output (right)

News

[2024.01.12] First release code
[2024.01.19] Add text super-resolution module (improved from MARCONet). This module will still be updated iteratively

⏰TODOs

• [✅] Add text super-resolution module to improve the effect of text scenes
• Release a model specifically for AIGC image enhancement
• Release a model specifically for old photo enhancement

Installation

Now we suggest using the image provided by modelscope, simply run the following code:

# docker pull  
docker pull registry.cn-hangzhou.aliyuncs.com/modelscope-repo/modelscope:ubuntu20.04-cuda11.8.0-py38-torch2.0.1-tf2.13.0-1.9.3

run the docker and then:

git clone https://github.com/dreamoving/Phantom.git
# install python package
pip install -r requirements.txt

or you can install modelscope manually

git clone https://github.com/dreamoving/Phantom.git
pip install modelscope==1.9.3
pip install -r requirements.txt

Besides, you may need to download the following checkpoints before usage.
Download PASD and unzip it in runs/
Download SD1.5 models from huggingface and put them into checkpoints/stable-diffusion-v1-5
Dwonload RealESRGAN_x4plus and RealESRGAN_x2plus and put them in realesrgan/weights
Download SwinIR and put it in SwinIR/weights
Download iqa_model and put it in synthesis_vqa/weights
Download text_sr_model and put it in MARCONet/checkpoints, please refer to MARCONet/checkpoints/download_github.py
Download cnstd and put it in MARCONet/checkpoints/db_resnet34/1.2/db_resnet34
When inferencing with PASD, you can use personalized_models instead of SD1.5. Download the majicMIX and put it in checkpoints/personalized_models to run the demo.

Usage

As mentioned in the introduction, this is a fully automatic image enhancement super-resolution framework. Generally, you don’t need to select a model or adjust parameters based on image input. Simply run the following code, you can build a gradio demo locally. You can also try our online demo at modelscope.

cd Phantom
# python gradio1.py
python gradio2.py # gradio2 is for the newest version, which includes text super-resolution module 

Reference and Acknowledgments

We gratefully acknowledge the following projects and contributors for their work, which has greatly contributed to this program.
PASD
SwinIR
Real-ESRGAN
MARCONet
CLIP
LAVIS