abyildirim/inst-inpaint

A novel inpainting framework that can remove objects from images based on the instructions given as text prompts.

inst-inpaint's Issues

Upload dataset to huggingface

Could you upload the dataset to huggingface as well ? The google drive always complains "there are too many users ..." and we could not download it.

Can you provide your train_instructions.json?

I found that only the instructions of test are provided, can you further provide the larger data from training stage?

where can we access to the dataset

Request for Vocabulary in "Selecting Objects from the Scene Graphs" Section

Hi,

I am currently researching the topic covered in your paper, specifically the "Selecting Objects from the Scene Graphs" section. I am interested in the complete vocabulary for the categories "Implicit parts of an object" and "Items worn by an object" as outlined in your "Parts vs. Objects" classification. Your assistance would greatly benefit my ongoing research!

Thank you for your time and consideration.

How many training steps do you use when training on GQA-Inpaint?

I cannot find the total training steps you used in the config or the code.
There seems to be no description about the training steps or epochs in the paper either.
Could you tell me how many training steps do you use in the GQA-Inpaint dataset?
And does training longer or shorter influence the results? What is the influence?

Thanks!

when will the code be released?

Image quality isn't retained

Great work,I noticed that the output looks really bad in comparison to the input when using it on replicate. It also resizes the image.

Bad results

Hey, i am very appreciated of your work, but when i try to inpaint on kite-boy image in your example, the result is very bad

Can you explain this for me, please?

About the scale of the GQA-inpainting dataset

In the paper, it said "Because from source images we pick different objects and generate instructions, training includes 173715 unique source-target-prompt pairs. We test the models on 4811 unique source images, 9485 unique target images, and 18883 unique source-target-prompt pairs.". When I downloaded the data from github, I got 38w+ masks, which means I have 38w+ source-target-mask pairs. Did I process the data in a wrong way or something?

dataset

Hello, why can't I download the dataset? It says "no permission" halfway through the download.

Any plans to do some research on masking based removing methods?

It's a great work!
I wonder if this method can be transferred to mask based removing? As instruction sometimes cannot convey the user's accurate intention! I have tried SD-Inpaint, but its removing ability is not that good, it usually generate some unwanted objects.
Do you have some ideas about diffusion based masking removing strategy?

How to indicate which object should be removed?

Hi all,

I read through the instructions for Inst-Inpaint. However, I am not sure where/how we can indicate which option should be removed from the picture?
I cannot find any parameter in the instructions that explains how to indicate the object.
Can someone please let me know how to specify exactly which object needs to be removed?

Thanks

Great work

Thanks for your great works! I cannot wait to try out your code and models. I'm looking forward to it!

Demo ? input image -> result ?

when will the code to be released?

It's been a long wait since the last update to this repo.