sgugger/deep-learning

A few notebooks about deep learning in pytorch

deep-learning's Issues

Thanks for this incredible work

I wish more people published notebooks like this. Thanks a lot.

DeepPainterlyHarmonization notebook error: IndexError: too many indices for tensor of dimension 1

Hi Sylvain, thanks for the awesome exposition in your DeepPainterlyHarmonization notebook. I am however running into an issue during the second training phase. Not changing anything, was able to reproduce the code up until In[84].
With the following warning:

IndexError: too many indices for tensor of dimension 1

Here's the detailed warning:

---

IndexError Traceback (most recent call last)
in ()
1 n_iter=0
----> 2 while n_iter <= max_iter: optimizer.step(partial(step,final_loss))

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/torch/optim/lbfgs.py in step(self, closure)
101
102 # evaluate initial f(x) and df/dx
--> 103 orig_loss = closure()
104 loss = float(orig_loss)
105 current_evals = 1

in step(loss_fn)
2 global n_iter
3 optimizer.zero_grad()
----> 4 loss = loss_fn(opt_img_v)
5 loss.backward()
6 n_iter += 1

in final_loss(opt_img_v)
4 c_loss = content_loss(out_ftrs[-1])
5 s_loss = style_loss(out_ftrs)
----> 6 h_loss = hist_loss([out_ftrs[0], out_ftrs[3]])
7 t_loss = tv_loss(opt_img_v[0])
8 return c_loss + w_s * s_loss + w_h * h_loss + w_tv * t_loss

in hist_loss(out_ftrs)
6 mask = V(torch.Tensor(mf).contiguous().view(1, -1), requires_grad=False)
7 of_masked = of * mask
----> 8 of_masked = torch.cat([of_masked[i][mask>=0.1].unsqueeze(0) for i in range(of_masked.size(0))])
9 loss += F.mse_loss(of_masked, V(remap_hist(of_masked, sh), requires_grad=False))
10 return loss / 2

in (.0)
6 mask = V(torch.Tensor(mf).contiguous().view(1, -1), requires_grad=False)
7 of_masked = of * mask
----> 8 of_masked = torch.cat([of_masked[i][mask>=0.1].unsqueeze(0) for i in range(of_masked.size(0))])
9 loss += F.mse_loss(of_masked, V(remap_hist(of_masked, sh), requires_grad=False))
10 return loss / 2

IndexError: too many indices for tensor of dimension 1

ImageClassifierDataset no longer exists?

when I try to fun your notebook I get:

NameError: name 'ImageClassifierData' is not defined

what is the cause of this issue?

1cycle Policy. Unfamiliar results

Hey,

I was implementing 1 cycle policy as an exercise. And I have a few observations from my experiments.
I have a
Model : Resnet18.
Batch size for training = 128
Batch size for testing = 100

Optimser : optim.SGD(net.parameters(), lr=0.1, momentum=0.9, weight_decay=5e-4)
Total number of epochs 26

1 cycle policy : Learning rate goes from 0.01 to 0.1 and back till 24 epochs

Then model is trained for 2 epochs at 0.001 learning rate.

No cyclic momentum used or adamw.

I achieved a test set accuracy of 93.4%in 26 epochs.

This seems like a big difference from the 70 epochs at 512 batch size that is quoted in your blog post.

Am I doing something wrong ? Is the number of epochs a good metric to base your results on, as those are dependant on the batch size ? .

The whole point of using super convergence is using high learning rates to converge quicker , but it seems like using low learning rates (0.01- 0.1 < 0.8-3) is faster to train.

to_gpu(x, *args, **kwargs) 'numpy.ndarray' object has no attribute 'cuda'

Hi, I try to run the code of the website (https://github.com/sgugger/Deep-Learning/blob/master/DeepPainterlyHarmonization.ipynb).

When in the cell 22, I met a error.
In [22]: m_vgg(VV(input_tfm[None]))
input_ftrs = [s.features for s in sfs]
[sf.shape for sf in input_ftrs]

d:\python3.6.5\lib\site-packages\fastai\core.py in VV(x)
20 def V(x): return [V_(o) for o in x] if isinstance(x,list) else V_(x)
21 def VV_(x): return to_gpu(x, async=True) if isinstance(x, Variable) else Variable(to_gpu(x, async=True), volatile=True)
---> 22 def VV(x): return [VV_(o) for o in x] if isinstance(x,list) else VV_(x)
23
24 def to_np(v):

d:\python3.6.5\lib\site-packages\fastai\core.py in VV_(x)
19 def V_(x): return to_gpu(x, async=True) if isinstance(x, Variable) else Variable(to_gpu(x, async=True))
20 def V(x): return [V_(o) for o in x] if isinstance(x,list) else V_(x)
---> 21 def VV_(x): return to_gpu(x, async=True) if isinstance(x, Variable) else Variable(to_gpu(x, async=True), volatile=True)
22 def VV(x): return [VV_(o) for o in x] if isinstance(x,list) else VV_(x)
23

d:\python3.6.5\lib\site-packages\fastai\core.py in to_gpu(x, *args, **kwargs)
28 def to_gpu(x, *args, **kwargs):
29 if torch.cuda.is_available():
---> 30 return x.cuda(*args, **kwargs)
31 else:
32 return x

AttributeError: 'numpy.ndarray' object has no attribute 'cuda'

the programe is runing in the windows 10 with GTX Titan X.
The software environment is : fastai 0.6 troch 0.4.0 cuda 9.1 + v7.1

I would like to know why ?
Did I installed the software with the wrong version?
How can I fix it ?

Use with pytorch model

Can you give me an example of how to use with with my own dataset function. For instance your getdata() function uses ImageClassifierData module of fastai. I have created a pytorch dataset but I'm not sure how to wrap it for this function