Comments (1)
Hi,
This DDPM code was created replicating the blocks from the official version for stable diffusion with the goal to create a minimalistic diffusion model purely for understanding purposes(and then progress to Stable Diffusion from here).
As far as I saw in the official repo, the authors don't add any positional encoding information prior to attention in their transformer block.
That was the only reason for me also not adding in here and without adding it, I was getting decent results so then didn't end up adding it.
I don't think in the stable diffusion repo, the authors anywhere mention why that is the case though. The same is asked here as well. So to answer why its not added, I honestly don't know. Maybe after adding positional information the result might indeed end up being significantly better. Or maybe the presence of padding in the resnet convolutions already give the network enough indication of positional information in its feature map representations, enough that it manages to do fine without any explicitly provided positional information.
If you want to experiment with adding the positional information to the attention block , you can just replicate what is done in diffusers library here, where they simply add sinusoidal embeddings prior to every attention block .
Do let me know what results you end up with, in case you decide to try it out or if you end up finding more information on this.
from ddpm-pytorch.
Related Issues (6)
- what changes would we need to do if we used our own dataset? HOT 76
- what parameter changes would I need to make sure it runs on our dataset? HOT 12
- Noise on all generated images HOT 4
- RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu) HOT 5
- Question on the training processo of a Diffusion Model HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ddpm-pytorch.