davidcpage/cifar10-fast

cifar10-fast's Issues

why write your code in this way?

The way you write code is really hard to understand... why write in this way... is your project in work written in this way too? what's the benifit~, just curious~~

classifier bias mismatch

basic_net classifier uses bias=False, but blog diagrams show bias=True. Also, if it is indeed set to False, why so?

about the weight decay

in your original code opt = SGD(trainable_params(model), momentum=0.9, weight_decay=5e-4*batch_size, nesterov=True)
multiply 5e-4 by batch_size is redundant.

How to draw a 'net.svg' like you?

I really like the picture you draw about your net because it can make parameters Visual.I want to know how you do it?Thank you.

Not so fast

Hi,

I ran your code on a single GPU, which is NVIDIA GeForce GTX 1080Ti, but training 24 epochs took me nearly 4 minutes.

I did not change anything in your code except two paths.

Is that we have to configure something before we run your model ?

Thank you.

Nesterov update question

Hi there, I have a question about the nesterov update

def nesterov_update(w, dw, v, lr, weight_decay, momentum):
    dw.add_(weight_decay, w).mul_(-lr)
    v.mul_(momentum).add_(dw)
    w.add_(dw.add_(momentum, v))

If I had to sketch the logic of this, I see it as saying

dw = -lr*(dw + weight_decay*w)
v = v*momentum + dw
w = w + dw + momentum*v

due to the in-place use of add_ etc.

Q: Can you explain how this is implementing nesterov? I don't see how it is calculating the gradient at the shifted weight.

How to cite the ResNet9?

self.cache[n] = getattr(self, n)(*[self.cache[x] for x in i])，run it python3.5,it make a mistake.

File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(input, **kwargs)
File "/home/clarence/cifar10-fast/torch_backend.py", line 126, in forward
self.cache[n] = getattr(self, n)([self.cache[x] for x in i])
File "/home/clarence/cifar10-fast/torch_backend.py", line 126, in
self.cache[n] = getattr(self, n)(*[self.cache[x] for x in i])
KeyError: 'classifier'