yingtiandt Goto Github PK

ask-anything

[CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

auditory-slow-fast

Implementation of "Slow-Fast Auditory Streams for Audio Recognition, ICASSP, 2021" in PyTorch

avid-cma

Audio Visual Instance Discrimination with Cross-Modal Agreement

brain-score

A framework for evaluating models on their alignment to brain and behavioral measurements (50+ benchmarks)

brainio

Data management for quantitative comparison of brains and brain-inspired systems

cav-mae

Code and Pretrained Models for ICLR 2023 Paper "Contrastive Audio-Visual Masked Autoencoder".

contrastive2021

Implementation for paper "Towards the Generalization of Contrastive Self-Supervised Learning" (https://arxiv.org/abs/2111.00743)

debug_tools

dino

PyTorch code for Vision Transformers training with the Self-Supervised learning method DINO

exp_tools

gdt

We present a framework for training multi-modal deep learning models on unlabelled video data by forcing the network to learn invariances to transformations applied to both the audio and video streams.

ijepa

Official codebase for I-JEPA, the Image-based Joint-Embedding Predictive Architecture. First outlined in the CVPR paper, "Self-supervised learning from images with a joint-embedding predictive architecture."

llava

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

mae_st

Official Open Source code for "Masked Autoencoders As Spatiotemporal Learners"

mamba.py

A simple and efficient Mamba implementation in PyTorch and MLX.

mastering-cpp-multithreading

mmaction2

OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark

model-tools

Helper functions to extract model activations and translate from Machine Learning to Neuroscience

modelzoo_continual

Model Zoos for Continual Learning (ICLR 22)

neuroparc

openstl

OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learning

pdbpp

pdb++, a drop-in replacement for pdb (the Python debugger)

result_caching

Store results of function calls with respect to the call parameters

s3d_howto100m

S3D Text-Video model trained on HowTo100M using MIL-NCE

scenic

Scenic: A Jax Library for Computer Vision Research and Beyond

segformer-pytorch

Implementation of Segformer, Attention + MLP neural network for segmentation, in Pytorch

selavi

This repo covers the implementation for Labelling unlabelled videos from scratch with multi-modal self-supervision, which learns clusters from multi-modal data in a self-supervised way.

simclr-cifar10-master

videomae

[NeurIPS 2022 Spotlight] VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training

videomaev2

[CVPR 2023] VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking