itsank/cs7545

Course Information

• Course Info: CS7545, Fall 2019
• Instructor: Jacob Abernethy
  • Office: Klaus 2134
  • Email: prof_at_gatech_dot_edu
  • Office Hours: TBA
• Course Time&Place: MW 4:30-5:45pm, Weber SST III (Lecture Hall 2)
• Teaching Assistants:
  • Bhuvesh Kumar
    • Email: bhuvesh_at_gatech.edu
    • Office Hours: TBA
  • Zihao Hu
    • Email: zihaohu_at_gatech.edu
    • Office Hours: TBA

Course Description

This course will study theoretical aspects of prediction and decision-making probelms, where our goal is to understand the mathematical underpinnings of machine learning. A primary objective of the class is to bring students to the frontiers of research and to prepare students to publish in this area. The course will cover, among other things, concentration inequalities, uniform deviation bounds, Vapnik-Chervonenkis Theory, Rademacher Complexity, margin bounds, boosting, some theoretical aspects of deep learning, online learning theory, regret minimization, multi-armed bandit algorithms, and connections to convex optimization. Along the way, we may dive into several related topics, including minimax equilibrium in games, calibration, sequential portfolio selection, option pricing, and differential privacy.

Prerequisites: Familiarity with the analysis of algorithms, probabilistic analysis, and several similar topics. CS7641 (Machine Learning) will be quite helpful but not strictly necessary. The material is going to be about 90% "theory" and thus potential students must have a strong mathematical background. We shall rely heavily on techniques from calculus, probability, and convex analysis, but many tools will be presented in lecture.

Coursework: There will be 5 problem sets throughout the semester.

Grade Breakdown:

• 50% - Homeworks
• 40% - Final Exam
• 10% - Participation

Note: The final exam will be held on Wednesday, December 11, from 2:40-5:30pm.

References:

Roughly half of the course will follow material from the following text:

• "Foundations of Machine Learning" by Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar

Much of the material in online learning (aka regret minimization) is of my own taste, and I will present these topics how I enjoy. But for students that want reading material on this topic, there are several surveys released in the last several years that explore several many that we shall cover. I will link to them here, and will mention them in various lectures when appropriate:

• The Multiplicative Weights Update Method by Sanjeev Arora, Elad Hazan, and Satyen Kale.
• Online Learning and Online Convex Optimization survey by Shai Shalev-Shwartz.
• The convex optimization approach to regret minimization survey by Elad Hazan.
• Sasha Rakhlin's Lecture Notes.

Scribe Notes

Lecture	Date	Topic
1	19 Aug 2019	Introduction and and Linear Algebra Review
2	21 Aug 2019	Convex Analysis

The Latex template for scribes is available here.

Homeworks

Homework	Due Date
1	Sep 4 2019, 2:00 pm

The Latex template for HW submissions is available here.

Previous offerings of the course: Fall 2018