Inpaint Anything can inpaint anything in images, videos and 3D scenes!
- Authors: Tao Yu, Runseng Feng, Ruoyu Feng, Jinming Liu, Xin Jin, Wenjun Zeng and Zhibo Chen.
- Institutes: University of Science and Technology of China; Eastern Institute for Advanced Study.
- [Paper] [Website] [Hugging Face Homepage]
TL; DR: Users can select any object in an image by clicking on it. With powerful vision models, e.g., SAM, LaMa and Stable Diffusion (SD), Inpaint Anything is able to remove the object smoothly (i.e., Remove Anything). Further, prompted by user input text, Inpaint Anything can fill the object with any desired content (i.e., Fill Anything) or replace the background of it arbitrarily (i.e., Replace Anything).
[2023/4/30] Remove Anything Video available! You can remove any object from a video!
[2023/4/24] Local web UI supported! You can run the demo website locally!
[2023/4/22] Website available! You can experience Inpaint Anything through the interface!
[2023/4/22] Remove Anything 3D available! You can remove any 3D object from a 3D scene!
[2023/4/13] Technical report on arXiv available!
- Remove Anything
- Fill Anything
- Replace Anything
- Remove Anything 3D (🔥NEW)
- Fill Anything 3D
- Replace Anything 3D
- Remove Anything Video (🔥NEW)
- Fill Anything Video
- Replace Anything Video
- Any aspect ratio supported
- 2K resolution supported
- Technical report on arXiv available (🔥NEW)
- Website available (🔥NEW)
- Local web UI available (🔥NEW)
- Multiple modalities (i.e., image, video and 3D scene) supported (🔥NEW)
Click on an object in the image, and Inpainting Anything will remove it instantly!
- Click on an object;
- Segment Anything Model (SAM) segments the object out;
- Inpainting models (e.g., LaMa) fill the "hole".
Requires python>=3.8
python -m pip install torch torchvision torchaudio
python -m pip install -e segment_anything
python -m pip install -r lama/requirements.txt In Windows, we recommend you to first install miniconda and
open Anaconda Powershell Prompt (miniconda3) as administrator.
Then pip install ./lama_requirements_windows.txt instead of
./lama/requirements.txt.
Download the model checkpoints provided in segment_anything
and lama (e.g. sam_vit_h_4b8939.pth
and big-lama), and put them into ./pretrained_models.
Specify an image and a point, and Inpaint-Anything will remove the object at the point.
python remove_anything.py \
--input_img ./example/remove-anything/dog.jpg \
--coords_type key_in \
--point_coords 200 450 \
--point_labels 1 \
--dilate_kernel_size 15 \
--output_dir ./results \
--sam_model_type "vit_h" \
--sam_ckpt ./pretrained_models/sam_vit_h_4b8939.pth \
--lama_config ./lama/configs/prediction/default.yaml \
--lama_ckpt ./pretrained_models/big-lamaYou can change --coords_type key_in to --coords_type click if your machine has a display device. If click is set, after running the above command, the image will be displayed. (1) Use left-click to record the coordinates of the click. It supports modifying points, and only last point coordinates are recorded. (2) Use right-click to finish the selection.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Text prompt: "a teddy bear on a bench"
Click on an object, type in what you want to fill, and Inpaint Anything will fill it!
- Click on an object;
- SAM segments the object out;
- Input a text prompt;
- Text-prompt-guided inpainting models (e.g., Stable Diffusion) fill the "hole" according to the text.
Requires python>=3.8
python -m pip install torch torchvision torchaudio
python -m pip install -e segment_anything
python -m pip install diffusers transformers accelerate scipy safetensorsDownload the model checkpoints provided in segment_anything
(e.g. sam_vit_h_4b8939.pth).
, and put them into ./pretrained_models.
Specify an image, a point and text prompt, and run:
python fill_anything.py \
--input_img ./example/fill-anything/sample1.png \
--coords_type key_in \
--point_coords 750 500 \
--point_labels 1 \
--text_prompt "a teddy bear on a bench" \
--dilate_kernel_size 50 \
--output_dir ./results \
--sam_model_type "vit_h" \
--sam_ckpt ./pretrained_models/sam_vit_h_4b8939.pth |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Text prompt: "a man in office"
Click on an object, type in what background you want to replace, and Inpaint Anything will replace it!
- Click on an object;
- SAM segments the object out;
- Input a text prompt;
- Text-prompt-guided inpainting models (e.g., Stable Diffusion) replace the background according to the text.
Requires python>=3.8
python -m pip install torch torchvision torchaudio
python -m pip install -e segment_anything
python -m pip install diffusers transformers accelerate scipy safetensorsDownload the model checkpoints provided in segment_anything
(e.g. sam_vit_h_4b8939.pth)
, and put them into ./pretrained_models.
Specify an image, a point and text prompt, and run:
python replace_anything.py \
--input_img ./example/replace-anything/dog.png \
--coords_type key_in \
--point_coords 750 500 \
--point_labels 1 \
--text_prompt "sit on the swing" \
--output_dir ./results \
--sam_model_type "vit_h" \
--sam_ckpt ./pretrained_models/sam_vit_h_4b8939.pth |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Remove Anything 3D can remove any object from a 3D scene! We release some results below. (Code and implementation details will be released soon.)
 |
 |
 |
 |
 |
 |
With a single click on an object in the first video frame, Remove Anything Video can remove the object from the whole video! We release some results below. (Code and implementation details will be released soon.)
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
If you find this work useful for your research, please cite us:
@article{yu2023inpaint,
title={Inpaint Anything: Segment Anything Meets Image Inpainting},
author={Yu, Tao and Feng, Runseng and Feng, Ruoyu and Liu, Jinming and Jin, Xin and Zeng, Wenjun and Chen, Zhibo},
journal={arXiv preprint arXiv:2304.06790},
year={2023}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent