amplab/keystone

Simplifying robust end-to-end machine learning on Apache Spark.

keystone's Issues

Grayscale

With proper NTSC standard weights

Tokenization

Contrast Normalization

Create Keystone Repo

That is this repo!

BSD Licensed. Permissions, labels, and milestones set up.

ImageNet Pipeline

Should have accuracy >60% with 4k features.

MNIST Pipeline

Simplest image pipeline - should run single-node with no issues.

Max Classifier

Accuracy

Compute accuracy of a classifier.

TIMIT Features -> Solve

No public features - but we can achieve the solve result!

mAP

Mean average precision.

Add Jenkins Build to Pipelines Repo

Jenkins should build the repo, and run all unit tests. It should also automatically build/test PRs.

POS Tagging

May be unnecessary for Release 0.1

SIFT

Should call VLFeat dense SIFT via JNI.

Standard VOC Pipeline

VOC Pipeline with 59% MAP

Requires

1. SIFT
2. PCA
3. GMM
4. FisherVector

As well as a dataloader for VOC2007.

Pooling

Clean up launch scripts

Bash scripts should be put in a uniform format, and we should settle on some common spark options.

e.g. setting eventLogger to "on" and locality.wait to some reasonable default.

We should also have instructions about how to launch a cluster with spark-ec2 scripts and execute pipelines with this repo.

Fisher Vector

Should call VGG fisher vector C++ code via JNI.

TIMIT Full Pipeline

Including featurization. Stretch goal - no milestone.

N-Grams

MFCCs

Multi-frequency Cepstral Coefficients - a common set of speech features.

HOG Featurizer

C build script

Create a Makefile for the C components, and integrate with sbt.

Clean up build script

First commit contains a reasonable clean build script.

HMM

Used for speech pipeline. May be unncessary for release 0.1.

VOC Data Loader

Convolver

Text Classification Pipeline

Stop-word removal -> N-Grams -> TFIDF -> Naive Bayes

Should work on 20 Newsgroups and RCV1.

Feature Normalization

Subtract feature means and optionally divide by SD.

Standardize main() format and use scopt

Pipelines executed from the command line should have a uniform CLI. We should use scopt for argument parsing and be consistent with option names across pipelines.

LDA

Linear Discriminant Analysis - as opposed to the topic modeling kind.

Linear Rectifier

Signed Hellinger Mapper

See: https://github.com/etrain/pipelines/blob/fishertesting2/src/main/scala/pipelines/FVVOC2007.scala#L128

LCS

Local Color Statistic

ZCA Whitening

KMeans

Call into MLlib

Kneser-Ney Language Model Pipeline

Zongheng can fill in details - I think this is a stretch for release 0.1, so I'm not going to assign a milestone.

CIFAR Pipelines from AMP Camp 5

These should also run single node

1. Pixel version
2. Random Patch/Convolutions Version

Lemmatization

May be unnecessary for Release 0.1

Image Scaling

Random Features

Should support gaussian and cauchy random maps. Useful if there's a lazy interface as well.

GMM

Should fit a GMM on a sample of 1m features locally - call out to VGG GMM library with JNI.

Language Model Pipeline

Implemented with Stupid Backoff. Should have a small demo that shows what the language model can do - e.g. evaluate P(text comes from modeled langague).

PCA

Transformer should downsample with PCA matrix.

Linear Solver/Mapper

First cut may or may not do label centering.

Event Extraction Pipeline

Ideally on the ACE2005 dataset.

Sampling

Downsample a collection.

NER

Named Entity Recognition. May be unnecessary for 0.1.

Daisy Featurizer

Windowing

FFT Transformer

Whether this belongs in stats or elsewhere is certainly up for debate.