umutsimsekli/sgd_tail_index

The codebase for the paper "A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks"

sgd_tail_index's Issues

Utils package

Hello,

It seems like you did not upload utils package. I cannot import these follow functions:

from utils import get_id, get_data, accuracy
from utils import get_grads, alpha_estimator, alpha_estimator2
from utils import linear_hinge_loss, get_layerWise_norms

Could you just upload the rest or did I miss anything?

Thx.

Server setup

Hi, this is amazing work. I have a quick question. What do you mean by " Make sure to replace the command that is specific to your server."

In the code, I saw the line below. How do I change it? I am new to this. THANKS.
launcher = "srun --nodes=1 --gres=gpu:1 --time=40:00:00 --mem=60G" # THIS IS AN EXAMPLE!!!