gabeorlanski/adversarial-robustness-toolbox

Adversarial Robustness Toolbox (ART) - Python Library for Machine Learning Security - Evasion, Poisoning, Extraction, Inference

adversarial-robustness-toolbox's Issues

Kinetics 400 Exploration

• File formats
• Simple Examples
• Complex Examples
• Data Types
• (?) Memory Issue
• Provided Preprocessing
• ART Preprocessing

Kinetics 400 Dataset

Video Preprocessing

https://github.com/deepmind/kinetics-i3d#sample-data-and-preprocessing

https://github.com/google/mediapipe

I3D Exploration

• Get it to run
• See what their preprocessing is and if it is open source & summarize
• Run Evaluation and get baseline results
• See if we can make a confusion matrix

I3D git

Downloading Videos

• Split the data links into classes and select ~ 10 Classes
• Randomly select 6 examples from train, 3 from validate, and 3 from test for each class (120 total videos)
• Record size for 12 videos of a single class
• For loop download

https://github.com/open-mmlab/mmaction2

Their Demo

Their Config

MMAction2 Model Implementation

Since the original I3D repository is very out of date, we need to implement the model with the MMAction2 framework. This colab notebook has all we need since they test the classifier on the Kinetics 400 dataset.

We need to take that notebook and get it into a script (or our own google colab) that can:

• Train a model
• Save the trained model*
• Load the trained model*
• Evaluate the trained model on a given set of videos

* If we find that training these models on our small amount of data takes a trivial amount of time, then we can exclude this.

Location of our code

We need to figure out where we want to put our code for the parts that are not going to be combined with ART.

They are:

• The MMAction2 Model
• Retrieving and loading the dataset
• Any preprocessing
• Evaluations(?)

We could create a new directory at the top level or maybe in examples.