ogkalu2/sketch-guided-stable-diffusion

Unofficial Implementation of the Google Paper - https://sketch-guided-diffusion.github.io/

sketch-guided-stable-diffusion's Issues

You seem to only share the pt model, I would like to consult where to put this model to work

I downloaded your pt model and put it in the embeddings folder, but it doesn't work, is it my operation wrong?

LGP Inference

Thanks for posting an amazing implementation.

I managed to train the LGP model, but having trouble in inferance, where we are supposed to guide reverse process of diffusion model using the gradient of the LGP.

Could you please provide the complete inferance code ? Or at least some crude outlines ?

issue about text_encoder

Hello, I encountered bugs when I run the code. "text_encoder = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14").to(device)"
something like that:
..size mismatch for text_model.encoder.layers.11.mlp.fc2.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for text_model.encoder.layers.11.layer_norm2.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for text_model.encoder.layers.11.layer_norm2.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for text_model.final_layer_norm.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([512]).
size mismatch for text_model.final_layer_norm.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([512]).

Could you help me fix it?