Comments (21)
Regarding convert_videos_to_frames.py, is there a significant performance/speed increase associated with that approach over extracting frames via opencv in python?
There should be no noticeable speed or performance gains.
from latte.
训练loss打印出来,降到大概0.03上下,这正常吗
from latte.
trained and sample result very strange (我自己训练复现的效果很奇怪,我在ucf数据集上面从头训练XL/2,训练到100000step,然后sample一些视频出现发现非常丑陋,根本没有规律)
sample.mp4
This is not a normal result. Did you notice a sudden increase in gradient during your training?How many Gpus did you train on?Can you provide a detailed training configuration?Thanks~
from latte.
Thank you so much, you're so nice. My training configuration is as follows:
My batchsize change to 1, ddp training on eight v100 32g. Because all the other parameters are completely unchanged and it's fine for me to sample the video using the checkpoint you provided. So I suspect it's because I changed the batchsize?
from latte.
Thank you so much, you're so nice. My training configuration is as follows:
My batchsize change to 1, ddp training on eight v100 32g. Because all the other parameters are completely unchanged and it's fine for me to sample the video using the checkpoint you provided. So I suspect it's because I changed the batchsize?
Check your training log for any sudden gradient increases. I suspect there may be something wrong with the training process.
from latte.
At 100step Gradient Norm: 1.1843
Gradual decrease, no mutation
At 50000step Gradient Norm: 0.03
Is that normal?
from latte.
At 100step Gradient Norm: 1.1843 Gradual decrease, no mutation At 50000step Gradient Norm: 0.03 Is that normal?
It is normal. How long have you been training?
from latte.
50000step, about 17 hours
from latte.
50000step, about 17 hours
I think it has not converged, please train for a day or two
from latte.
How many steps did you train before the sampled video was normal?
from latte.
How many steps did you train before the sampled video was normal?
Because we use different training equipment, I can't give you an exact number. It takes about 2 days on the training equipment I use. You can also refer to Fig. 8 in our paper.
from latte.
May I ask what GPU's you were using, I'm training with 8 A100 80G now, roughly how many steps do I need to train? I see your paper converged at 150k.
from latte.
May I ask what GPU's you were using, I'm training with 8 A100 80G now, roughly how many steps do I need to train? I see your paper converged at 150k.
I have confirmed with someone who uses the same training equipment as you to repeat latte on ucf101 recently, and it will take about 10w iterations to get a normal video.
from latte.
非常感谢您,我找到原因了,其实是因为我的数据集文件夹格式和你dataset代码的读取默认格式不一样,所以我训成了无条件生成,但是推理又用了类别条件。(Thank you very much, I found the reason, actually it's because my dataset folder format is not the same as the read default format of your dataset code, so I trained it to unconditional generation, but then used the category condition for inference.)
from latte.
顺便一问,您的Taichi数据集是从哪里下载的,为啥我下载的全是mp4文件,可我看你dataset是按照图像frames来读取的?
from latte.
顺便一问,您的Taichi数据集是从哪里下载的,为啥我下载的全是mp4文件,可我看你dataset是按照图像frames来读取的?
I used the Taichi dataset after converting the videos into images.
from latte.
Could you provide your code to convert the videos into images?
from latte.
https://github.com/universome/stylegan-v/blob/master/src/scripts/convert_videos_to_frames.py
Are you using this code?
from latte.
https://github.com/universome/stylegan-v/blob/master/src/scripts/convert_videos_to_frames.py
Are you using this code?
You can use it.
from latte.
Regarding convert_videos_to_frames.py, is there a significant performance/speed increase associated with that approach over extracting frames via opencv in python?
from latte.
trained and sample result very strange (我自己训练复现的效果很奇怪,我在ucf数据集上面从头训练XL/2,训练到100000step,然后sample一些视频出现发现非常丑陋,根本没有规律)
sample.mp4This is not a normal result. Did you notice a sudden increase in gradient during your training?How many Gpus did you train on?Can you provide a detailed training configuration?Thanks~
Hi maxin~ I noticed that you mentioned "the sudden increase in gradient". I've met the same problem. Did you know the reason why the gradient explosion happens? Would you be kind to tell how you solved this? Thanks very much!
from latte.
Related Issues (20)
- 视频帧率 HOT 2
- Question: model code and design choices HOT 2
- how to place and preprocess these datasets HOT 7
- the code of variant 4 HOT 1
- Question: evaluate the FVD HOT 6
- Error once speed up training HOT 2
- How to get preprocessed_ffs HOT 4
- Any plan to implement Latte in HuggingFace's diffusers library? HOT 3
- 模型在ucf101上无法收敛 HOT 5
- Can Latte train for I2V tasks? HOT 2
- Batch Size Ablations HOT 1
- what the param <input_sq_size> stands for? HOT 2
- Can we use batch_size>1 in sample_t2x.py HOT 7
- Evaluate the `FVD` on FFS HOT 4
- How can I utilize the weights of pre-trained PixArt-α to initialize the parameters of the spatial Transformer block in the Latte T2V model? HOT 2
- FVD on UCF-101 HOT 6
- Is there tutorial on transfering t2i to t2v model? HOT 4
- inference memory with torch.set_grad_enabled(True) HOT 1
- T2V training and evaluation HOT 6
- Results & ckpts of different sized Latte on UCF-101 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from latte.