tauraiucb/sgd_tail_index

The codebase for the paper "A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks"

SGD Tail-Index Analysis

on the importance of the role of the noise in SGD

Please use the run.py as a template to run bathces of experiments to reproduce the results in the paper. Make sure to replace the command that is specific to your server.

If you find this code useful, thank you for citing the following work:

@inproceedings{Simsekli2019Tail, author = {U. Simsekli and L. Sagun and M. Gurbuzbalaban} title = {A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks}, booktitle = {Proceedings of the 36th International Conference on Machine Learning, (ICML) 2019}, address = {Long Beach, CA, USA}, month = jun, year = {2019}, }