What is Midjourney's secret? Anyone knows?

[xavierxiao]

What is Midjourney's secret? Anyone knows?

[float-trip]

Emad discusses that here: https://old.reddit.com/r/StableDiffusion/comments/x9xqap/ama_emad_here_hello/inqj7dy/

They do prompt editing on the way in and post processing on the way out basically.

[TingTingin]

Using the same prompt between images isn't a good idea as they generate images in different ways so its not a like for like test regardless one of the forms of post processing we know mj does is clip guidance which SD doesn't do mj also has rich data from users and ratings and can use that to improve things even further Emad also mentioned in a Center Stage that also do "something else" but didnt say exactly what

In short stable diffusion is a raw unprocessed output whereas mj does alot of pre and post processing which is why you can have different results in this way

[janekm]

My theory (without much evidence) is that they built a pre-processor model that works on the embedding of the prompt. So rather than just appending "digital art, Greg Rutkowski" it takes the tensor of your prompt and tries to steer it towards prompt embeddings that got highly rated by their community.
In addition Emad did say they're using CLIP guidance and he probably knows. You could also have other guidance models alongside CLIP guiding composition and style, and that could achieve a lot.
If that's not what they're doing then this could be an interesting idea for someone else to try... 😅

[DrakenZA]

Its just some well made custom embeddings, along with things like GFP to fix faces etc.

Its like a very well done google colab notebook, but you pay for it.