camenduru/text-to-video-finetuning-colab

🐣 Please follow me for new updates https://twitter.com/camenduru
🔥 Please join our discord server https://discord.gg/k5BwmmvJJU
🥳 Please join my patreon community https://patreon.com/camenduru

🧊 3D ML Papers

🆕 https://github.com/camenduru/HumanGaussian-colab
🆕 https://github.com/camenduru/GeoDream-colab
🆕 https://github.com/camenduru/4dfy-colab
🆕 https://github.com/camenduru/LucidDreamer-colab
https://github.com/camenduru/Wonder3D-colab
https://github.com/camenduru/zero123plus-colab
https://github.com/camenduru/GaussianDreamer-colab
https://github.com/camenduru/MVDream-colab
https://github.com/camenduru/dreamgaussian-colab
https://github.com/camenduru/Text2Tex-colab
https://github.com/camenduru/SyncDreamer-colab
https://github.com/camenduru/SyncDreamer-docker
https://github.com/camenduru/threestudio-colab
https://github.com/camenduru/IT3D-text-to-3D-colab
https://github.com/camenduru/cotracker-colab
https://github.com/camenduru/ZoeDepth-colab
https://github.com/camenduru/zero123-colab
https://github.com/camenduru/PanoHead-colab
https://github.com/camenduru/bite-colab
https://github.com/camenduru/ECON-colab
https://github.com/camenduru/shap-e-colab

💃 3D Motion Papers

🆕 https://github.com/camenduru/SMPLer-X-colab
🆕 https://github.com/camenduru/MotionDiffuse-colab
https://github.com/camenduru/NIKI-colab
https://github.com/camenduru/PHALP-colab
https://github.com/camenduru/DWPose-colab
https://github.com/camenduru/4D-Humans-colab
https://github.com/camenduru/vid2avatar-colab
https://github.com/camenduru/PARE-colab
https://github.com/camenduru/VIBE-colab
https://github.com/camenduru/ViTPose-colab

📸 NeRF

🆕 https://github.com/camenduru/LucidDreamer-Gaussian-colab
🆕 https://github.com/camenduru/PeRF-colab
https://github.com/camenduru/4DGaussians-colab
https://github.com/camenduru/neuralangelo-colab
https://github.com/camenduru/gaussian-splatting-colab
https://github.com/camenduru/instant-ngp-colab

📽 Video ML Papers

🆕 https://github.com/camenduru/FreeInit-colab
🆕 https://github.com/camenduru/dynamiCrafter-colab
🆕 https://github.com/camenduru/MotionDirector-colab
🆕 https://github.com/camenduru/SEINE-colab
🆕 https://github.com/camenduru/LaVie-colab
🆕 https://github.com/camenduru/stable-video-diffusion-colab
https://github.com/camenduru/SadTalker-colab
https://github.com/camenduru/Show-1-colab
https://github.com/camenduru/VideoCrafter-colab
https://github.com/camenduru/Hotshot-XL-colab
https://github.com/camenduru/video-retalking-colab
https://github.com/camenduru/ProPainter-colab
https://github.com/camenduru/TokenFlow-colab
https://github.com/camenduru/I2VGen-XL-colab
https://github.com/camenduru/CoDeF-colab
https://github.com/camenduru/AnimateDiff-colab
https://github.com/camenduru/Rerender-colab
https://github.com/camenduru/3d-photo-inpainting-colab
https://github.com/camenduru/text2video-zero-colab
https://github.com/camenduru/text-to-video-synthesis-colab
https://github.com/camenduru/one-shot-talking-face-colab
https://github.com/camenduru/wav2lip-colab
https://github.com/camenduru/pix2pix-video-colab

🎙 Audio ML Papers

🆕 https://github.com/camenduru/singing-voice-conversion-colab
🆕 https://github.com/camenduru/styletts-colab
🆕 https://github.com/camenduru/HierSpeech_TTS-colab
🆕 https://github.com/camenduru/AudioSep-colab
🆕 https://github.com/camenduru/coqui-XTTS-colab
https://github.com/camenduru/VALL-E-X-colab
🆕 https://github.com/camenduru/seamless-m4t-colab
https://github.com/camenduru/audiogen-colab
https://github.com/camenduru/LP-Music-Caps-colab
https://github.com/camenduru/vampnet-colab
https://github.com/camenduru/tortoise-tts-colab
https://github.com/camenduru/MusicGen-colab
https://github.com/camenduru/elevenlabs-colab
https://github.com/camenduru/Retrieval-based-Voice-Conversion-WebUI-colab
https://github.com/camenduru/whisper-jax-colab
https://github.com/camenduru/bark-colab
https://github.com/camenduru/audioldm-colab

🧨 Diffusers

🆕 https://github.com/camenduru/UDiffText-colab
🆕 https://github.com/camenduru/PatchFusion-colab
🆕 https://github.com/camenduru/Mix-of-Show-colab
🆕 https://github.com/camenduru/SyncDiffusion-colab
🆕 https://github.com/camenduru/DemoFusion-colab
🆕 https://github.com/camenduru/playground-colab
🆕 https://github.com/camenduru/DeepCache-colab
🆕 https://github.com/camenduru/style-aligned-colab
🆕 https://github.com/camenduru/MagicAnimate-colab
🆕 https://github.com/camenduru/sdxl-turbo-colab
🆕 https://github.com/camenduru/cross-image-attention-colab
🆕 https://github.com/camenduru/sliders-colab
https://github.com/camenduru/SD-T2I-360PanoImage-colab
https://github.com/camenduru/SSD-1B-colab
https://github.com/camenduru/latent-consistency-model-colab
https://github.com/camenduru/DiffSketcher-colab
https://github.com/camenduru/FreeU-colab
https://github.com/camenduru/stable-fast-colab
https://github.com/camenduru/trainer
https://github.com/camenduru/litelama-colab
https://github.com/camenduru/background-replacement-colab
https://github.com/camenduru/IllusionDiffusion-colab
https://github.com/camenduru/Wuerstchen-colab
https://github.com/camenduru/T2I-Adapter-SDXL-colab
https://github.com/camenduru/facechain-colab
https://github.com/camenduru/StableVideo-colab
https://github.com/camenduru/StableSR-colab
https://github.com/camenduru/ldm3d-colab
https://github.com/camenduru/PixelFusion-colab
https://github.com/camenduru/UniControl-colab
https://github.com/camenduru/tiny-stable-diffusion-colab
https://github.com/camenduru/fabric-colab
https://github.com/camenduru/kohya_ss-colab
https://github.com/camenduru/PSLD-colab
https://github.com/camenduru/control-a-video-colab
https://github.com/camenduru/Matting-Anything-colab
https://github.com/camenduru/TextDiffuser-colab
https://github.com/camenduru/StableStudio-colab
https://github.com/camenduru/Radiata-colab
https://github.com/camenduru/DeepFloyd-IF-colab
https://github.com/camenduru/ControlNet-v1-1-nightly-colab
https://github.com/camenduru/grounded-segment-anything-colab
https://github.com/camenduru/kandinsky-colab
https://github.com/camenduru/stable-diffusion-dreambooth-colab
https://github.com/camenduru/stable-diffusion-diffusers-colab
https://github.com/camenduru/converter-colab

🧨 ppDiffusers

https://github.com/camenduru/paddle-ppdiffusers-webui-aistudio-colab
https://github.com/camenduru/paddle-converter-colab
https://github.com/camenduru/paddle-ppdiffusers-webui-aistudio

👀 Vision LLM

🆕 https://github.com/camenduru/ShareGPT4V-colab
https://github.com/camenduru/MiniGPT-v2-colab
https://github.com/camenduru/LLaVA-colab
https://github.com/camenduru/Qwen-VL-Chat-colab
https://github.com/camenduru/kosmos-2-colab
https://github.com/camenduru/Video-LLaMA-colab
https://github.com/camenduru/MiniGPT-4-colab

🦙 LLM

🆕 https://github.com/camenduru/Qwen-Audio-Chat-colab
🆕 https://github.com/camenduru/Mistral-colab
https://github.com/camenduru/Llama-2-Onnx-colab
https://github.com/camenduru/japanese-text-generation-webui-colab
https://github.com/camenduru/DoctorGPT-colab
https://github.com/camenduru/Replit-v1-CodeInstruct-3B-colab
https://github.com/camenduru/guanaco-colab
https://github.com/camenduru/nvidia-llm-colab
🆕 https://github.com/camenduru/text-generation-webui-colab
https://github.com/camenduru/gpt4all-colab
https://github.com/camenduru/alpaca-lora-colab

🎈 ML Papers

🆕 https://github.com/camenduru/EdgeSAM-colab
🆕 https://github.com/camenduru/SlimSAM-colab
🆕 https://github.com/camenduru/disco-colab
https://github.com/camenduru/daclip-uir-colab
https://github.com/camenduru/FastSAM-colab
https://github.com/camenduru/DiffBIR-colab
https://github.com/camenduru/inst-inpaint-colab
https://github.com/camenduru/anime-face-detector-colab
https://github.com/camenduru/insightface-person-detection-colab
https://github.com/camenduru/insightface-face-detection-colab
https://github.com/camenduru/FreeDrag-colab
https://github.com/camenduru/DragDiffusion-colab
https://github.com/camenduru/sam-hq-colab
https://github.com/camenduru/DragGAN-colab
https://github.com/camenduru/UserControllableLT-colab
https://github.com/camenduru/controlnet-colab
https://github.com/camenduru/notebooks

📦 Models

https://github.com/camenduru/xenmon-xl-model-colab
https://github.com/camenduru/ios-emoji-xl-model-colab
https://github.com/camenduru/text-to-video-model

📚 Tutorials

https://github.com/camenduru/Text-To-Video-Finetuning-colab
https://github.com/camenduru/train-text-to-image-tpu-tutorial

🧿 Web UI

https://github.com/camenduru/stable-diffusion-webui-colab
https://github.com/camenduru/sdxl-colab
https://github.com/camenduru/stable-diffusion-webui-sagemaker
https://github.com/camenduru/stable-diffusion-webui-paperspace
https://github.com/camenduru/stable-diffusion-webui-colab/tree/drive
https://github.com/camenduru/stable-diffusion-webui-colab/tree/training
https://github.com/camenduru/stable-diffusion-webui-docker
https://github.com/camenduru/stable-diffusion-webui-huggingface
https://github.com/camenduru/stable-diffusion-webui-kaggle
https://github.com/camenduru/stable-diffusion-webui-runpod
https://github.com/camenduru/stable-diffusion-webui-vultr
https://github.com/camenduru/stable-diffusion-webui-portable
https://github.com/camenduru/stable-diffusion-webui-artists-to-study
https://github.com/camenduru/stable-diffusion-webui-scripts
https://github.com/camenduru/ai-creator-archive
https://github.com/camenduru/aica

🍥 Comfy UI

https://github.com/camenduru/comfyui-colab

🍨 InvokeAI

https://github.com/camenduru/InvokeAI-colab

🍭 Fooocus

https://github.com/camenduru/Fooocus-docker
https://github.com/camenduru/Fooocus-colab

🔋 Lambda Labs Demos

https://github.com/camenduru/fabric-lambda
https://github.com/camenduru/text-generation-webui-lambda
https://github.com/camenduru/llama-2-70b-chat-lambda
https://github.com/camenduru/comfyui-lambda
https://github.com/camenduru/MusicGen-lambda
https://github.com/camenduru/whisper-jax-lambda
https://github.com/camenduru/stable-diffusion-webui-lambda
https://github.com/camenduru/stable-diffusion-webui-api-lambda
https://github.com/camenduru/Radiata-lambda
https://github.com/camenduru/DeepFloyd-IF-lambda
https://github.com/camenduru/falcon-40b-instruct-lambda
https://github.com/camenduru/MiniGPT-4-lambda
https://github.com/camenduru/guanaco-lambda
https://github.com/camenduru/pygmalion-7b-text-generation-webui-lambda
https://github.com/camenduru/one-shot-talking-face-lambda
https://github.com/camenduru/guanaco-13b-lambda
https://github.com/camenduru/guanaco-33b-4bit-lambda
https://github.com/camenduru/converter-lambda

🪐 Saturn Cloud Demos

https://github.com/camenduru/stable-diffusion-webui-saturncloud
https://github.com/camenduru/text-generation-webui-saturncloud
https://github.com/camenduru/comfyui-saturncloud

🥼 Open X Lab Demos

🆕 https://github.com/camenduru/stable-video-diffusion-openxlab
https://github.com/camenduru/DiffBIR-openxlab
https://github.com/camenduru/I2VGen-XL-openxlab
https://github.com/camenduru/DoctorGPT-openxlab
https://github.com/camenduru/stable-diffusion-webui-openxlab

🦗 Modal Demos

https://github.com/camenduru/jupyter-modal
https://github.com/camenduru/DiffBIR-modal
https://github.com/camenduru/stable-diffusion-webui-modal
https://github.com/camenduru/One-2-3-45-modal

🕸 Replicate

https://github.com/camenduru/MotionDirector-replicate
https://github.com/camenduru/DynamiCrafter-replicate

🕹 Unreal Engine

https://github.com/camenduru/unreal-engine-puzzle-collection-blueprint
https://www.unrealengine.com/marketplace/en-US/profile/camenduru

🎮 Unity

https://github.com/camenduru/seamless

🏆 Diffusion Awards

https://github.com/camenduru/DiffusionAwards

🥽 VR

Full Body Motion Capture With HTC Vive and Unity
https://www.youtube.com/watch?v=gE3E-AN7Qiw

👾 Study

https://github.com/camenduru/HoudiniStudy
https://github.com/camenduru/BlenderStudy
https://github.com/camenduru/GrasshopperStudy
https://github.com/camenduru/UEStudy

text-to-video-finetuning-colab's Issues

Readme Assets

Fruits.Strawberries.Flying.Falling.In.Air.Tabletop.Shot.High.Speed.Slow.Motion.5cf78a08.mp4

Training Issue

Hello Sir while I am using train_single colab file for 100 max train steps I am getting fallowing error
Steps: 0% 0/100 [00:00<?, ?it/s]2200 params have been unfrozen for training.
/usr/local/lib/python3.10/dist-packages/diffusers/configuration_utils.py:134: FutureWarning: Accessing config attribute num_train_timesteps directly via 'DDPMScheduler' object attribute is deprecated. Please access 'num_train_timesteps' over 'DDPMScheduler's config object instead, e.g. 'scheduler.config.num_train_timesteps'.
deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False)
/usr/local/lib/python3.10/dist-packages/diffusers/configuration_utils.py:134: FutureWarning: Accessing config attribute prediction_type directly via 'DDPMScheduler' object attribute is deprecated. Please access 'prediction_type' over 'DDPMScheduler's config object instead, e.g. 'scheduler.config.prediction_type'.
deprecate("direct config name access", "1.0.0", deprecation_message, standard_warn=False)
Steps: 100% 100/100 [01:12<00:00, 1.84it/s, lr=1e-6, step_loss=0.00567]Traceback (most recent call last):
File "/content/Text-To-Video-Finetuning/train.py", line 999, in
main(**OmegaConf.load(args.config))
File "/content/Text-To-Video-Finetuning/train.py", line 978, in main
save_pipe(
File "/content/Text-To-Video-Finetuning/train.py", line 488, in save_pipe
unet_out = copy.deepcopy(accelerator.unwrap_model(unet, keep_fp32_wrapper=False))
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.10/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.10/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 297, in _reconstruct
value = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.10/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.10/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 297, in _reconstruct
value = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.10/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.10/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 297, in _reconstruct
value = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.10/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.10/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 297, in _reconstruct
value = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.10/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.10/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 297, in _reconstruct
value = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.10/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.10/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 297, in _reconstruct
value = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.10/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.10/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 297, in _reconstruct
value = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 271, in _reconstruct
state = deepcopy(state, memo)
File "/usr/lib/python3.10/copy.py", line 146, in deepcopy
y = copier(x, memo)
File "/usr/lib/python3.10/copy.py", line 231, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 172, in deepcopy
y = _reconstruct(x, memo, *rv)
File "/usr/lib/python3.10/copy.py", line 297, in _reconstruct
value = deepcopy(value, memo)
File "/usr/lib/python3.10/copy.py", line 153, in deepcopy
y = copier(memo)
File "/usr/local/lib/python3.10/dist-packages/torch/nn/parameter.py", line 55, in deepcopy
result = type(self)(self.data.clone(memory_format=torch.preserve_format), self.requires_grad)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 14.75 GiB total capacity; 13.22 GiB already allocated; 10.81 MiB free; 13.37 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
wandb: Waiting for W&B process to finish... (failed 1). Press Control-C to abort syncing.
wandb:
wandb: Run history:
wandb: train_loss ▂█▁▁▅▁▅▃▁▆▂▃▃▅█▄▄▂▁▇▁▂▃▂▅▁▃▄▂▅▁▇▄▄▂▁▁▁▃▁
wandb:
wandb: Run summary:
wandb: train_loss 0.00567
wandb:
wandb: 🚀 View run legendary-wood-38 at: https://wandb.ai/genrative/text2video-fine-tune/runs/uwo0ozwf
wandb: Synced 5 W&B file(s), 0 media file(s), 0 artifact file(s) and 0 other file(s)
wandb: Find logs at: ./wandb/run-20230715_161213-uwo0ozwf/logs please help in this issuse sir