yunjey / pytorch-tutorial Goto Github PK
View Code? Open in Web Editor NEWPyTorch Tutorial for Deep Learning Researchers
License: MIT License
PyTorch Tutorial for Deep Learning Researchers
License: MIT License
Hi,
I am running the 09 image captioning model. When I used the pytorch 0.10, I can set the batchsize to be 512 (takes around 8G for the GPU memory). But when I upgrade to pytorch 0.11, the maximum batchsize I can set is only around 30 (takes around 11G GPU memory). I am confused since the codebase is mainly the basic CNN and lstm models. Do you know where the problem comes from? Have you tried to upgrade to pytorch 0.11 to run it?
By the way, I am using Tesla K40c GPU.
pytorch 0.10: 0.1.10+ac9245a
pytorch 0.11: same problem with (0.1.11_5 and 0.1.11+b13b701)
In the folder "09 - Image Captioning", you provided the pretrained model in the form of a .egg file. May i ask how do i use the downloaded file?
I tried extracting it by renaming the file to a .zip file, but it did not work.
Executing the egg file as Python package also gave the error:
SyntaxError: Non-UTF-8 code starting with '\xeb' in file trained_model.egg on line 2, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
Hi,
for the image caption model (chapter 09):
There is a minor syntax error, a : is missing here
https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/09%20-%20Image%20Captioning/train.py#L42
The Dropbox link to the pretrained model in the README does not work anymore (file not found at Dropbox). Last week it still worked.
hello, i noticed that you have saved the model in .pkl form : torch.save(file,'model.pkl'),then how to using this model?
This repository provides tutorial code for deep learning researchers to learn PyTorch. In the tutorial, most of the models were implemented with less than 30 lines of code. Before starting this tutorial, it is recommended to finish Official Pytorch Tutorial.
The link of Official Pytorch Tutorial
is failed.
When I follow the tutorials to train this network, I can not get effective image results. What should I do??
I made the following modifications
fake_images = fake_images.view(fake_images.size(0), 3, 32, 32)
torchvision.utils.save_image(fake_images.data, './sample/fake_samples_%d.png'
Traceback (most recent call last):
File "main.py", line 69, in
torchvision.utils.save_image(fixed_x.data.cpu(), './data/real_images.png')
AttributeError: 'torch.FloatTensor' object has no attribute 'data'
just move to_var above save,here is what it should be:
--- a/tutorials/03-advanced/variational_auto_encoder/main.py
+++ b/tutorials/03-advanced/variational_auto_encoder/main.py
@@ -65,8 +65,8 @@ data_iter = iter(data_loader)
# fixed inputs for debugging
fixed_z = to_var(torch.randn(100, 20))
fixed_x, _ = next(data_iter)
-torchvision.utils.save_image(fixed_x.data.cpu(), './data/real_images.png')
fixed_x = to_var(fixed_x.view(fixed_x.size(0), -1))
+torchvision.utils.save_image(fixed_x.data.cpu(), './data/real_images.png')
Hello,
I'm confused about the codes of training generator in GAN. Anyone could help me?
Codes here:
#=============== Train the generator ===============#
# Compute loss with fake images
z = to_var(torch.randn(batch_size, 64))
fake_images = G(z)
outputs = D(fake_images)
g_loss = criterion(outputs, real_labels)
# Backprop + Optimize
D.zero_grad()
G.zero_grad()
g_loss.backward()
g_optimizer.step()
Images generated from generator are fake, but why the second parameter of criterion is real_labels not fake_labels? Thank you all.
Hi , nice work !
I also implement image caption model reference from your project
But same config like your's , after 5 epochs loss converge at 2.5
And can't not do well on captioning job
Can you give me some advice?
Hi, I'm a bit new to PyTorch / Python so I apologize if I made a poor diagnosis in advance. My error is of the following form when running main.py
python main.py
Step [100/50000], Loss: 2.1832, Acc: 0.51
Traceback (most recent call last):
File "main.py", line 94, in <module>
for tag, value in net.named_parameters():
File "/home/yenson/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 237, in __getattr__
return object.__getattr__(self, name)
AttributeError: type object 'object' has no attribute '__getattr__'```
Hi,
I want to use the pre-trained models (without download MS-COCO Dataset). when i run the
python2 sample.py --image='png/example.png'
I got this error:
Traceback (most recent call last):
File "sample.py", line 97, in <module>
main(args)
File "sample.py", line 37, in main
vocab = pickle.load(f)
File "/home/karami/anaconda2/lib/python2.7/pickle.py", line 1384, in load
return Unpickler(file).load()
File "/home/karami/anaconda2/lib/python2.7/pickle.py", line 864, in load
dispatch[key](self)
File "/home/karami/anaconda2/lib/python2.7/pickle.py", line 1096, in load_global
klass = self.find_class(module, name)
File "/home/karami/anaconda2/lib/python2.7/pickle.py", line 1132, in find_class
klass = getattr(mod, name)
AttributeError: 'module' object has no attribute 'Vocabulary'
Hi, Is there a code version with beam search? currently the lstm uses greedy output
should main.gpu.py have cpu() calls in it?
https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/07%20-%20Bidirectional%20Recurrent%20Neural%20Network/main-gpu.py#L91
Hello I try implement this into video classification. Let say I have 3 feature per frame then I have 10 frames to consider. I am using this example, I think sequence_length=10
and input_shape=3
am I right? this is my code for the architecture.
# Hyper Parameters
sequence_length = 10
input_size = 3
hidden_size = 128
num_layers = 2
num_classes = 2
batch_size = 100
num_epochs = 2
learning_rate = 0.003
train_loader = torch.utils.data.DataLoader(dataset=spatio_dataset,
batch_size=batch_size,
shuffle=True)
print(train_loader)
# BiRNN Model (Many-to-One)
class BiRNN(nn.Module):
def __init__(self, input_size, hidden_size, num_layers, num_classes):
super(BiRNN, self).__init__()
self.hidden_size = hidden_size
self.num_layers = num_layers
self.lstm = nn.LSTM(input_size, hidden_size, num_layers,
batch_first=True, bidirectional=True)
self.fc = nn.Linear(hidden_size*2, num_classes) # 2 for bidirection
def forward(self, x):
# Set initial states
h0 = Variable(torch.zeros(self.num_layers*2, x.size(0), self.hidden_size)).cuda() # 2 for bidirection
c0 = Variable(torch.zeros(self.num_layers*2, x.size(0), self.hidden_size)).cuda()
# Forward propagate RNN
out, _ = self.lstm(x, (h0, c0))
# Decode hidden state of last time step
out = self.fc(out[:, -1, :])
return out
rnn = BiRNN(input_size, hidden_size, num_layers, num_classes)
rnn.cuda()
# Loss and Optimizer
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(rnn.parameters(), lr=learning_rate)
Here is my train code:
# Train the Model
for epoch in range(num_epochs):
for i, (spatio, label) in enumerate(train_loader):
#print(spatio)
#print(label)
spatio = Variable(spatio.view(-1, sequence_length, input_size)).cuda()
label = Variable(label).cuda()
print(spatio.size())
print(label.size())
# Forward + Backward + Optimize
optimizer.zero_grad()
outputs = rnn(spatio)
loss = criterion(outputs, label)
loss.backward()
optimizer.step()
if (i+1) % 100 == 0:
print ('Epoch [%d/%d], Step [%d/%d], Loss: %.4f'
%(epoch+1, num_epochs, i+1, len(train_dataset)//batch_size, loss.data[0]))
However I got This error:
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-54-5beb6b9da8ce> in <module>()
11 # Forward + Backward + Optimize
12 optimizer.zero_grad()
---> 13 outputs = rnn(spatio)
14 loss = criterion(outputs, label)
15 loss.backward()
/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.pyc in __call__(self, *input, **kwargs)
204
205 def __call__(self, *input, **kwargs):
--> 206 result = self.forward(*input, **kwargs)
207 for hook in self._forward_hooks.values():
208 hook_result = hook(self, input, result)
<ipython-input-51-474be96e77da> in forward(self, x)
29
30 # Forward propagate RNN
---> 31 out, _ = self.lstm(x, (h0, c0))
32
33 # Decode hidden state of last time step
/usr/local/lib/python2.7/dist-packages/torch/nn/modules/module.pyc in __call__(self, *input, **kwargs)
204
205 def __call__(self, *input, **kwargs):
--> 206 result = self.forward(*input, **kwargs)
207 for hook in self._forward_hooks.values():
208 hook_result = hook(self, input, result)
/usr/local/lib/python2.7/dist-packages/torch/nn/modules/rnn.pyc in forward(self, input, hx)
89 dropout_state=self.dropout_state
90 )
---> 91 output, hidden = func(input, self.all_weights, hx)
92 if is_packed:
93 output = PackedSequence(output, batch_sizes)
/usr/local/lib/python2.7/dist-packages/torch/nn/_functions/rnn.pyc in forward(input, *fargs, **fkwargs)
341 else:
342 func = AutogradRNN(*args, **kwargs)
--> 343 return func(input, *fargs, **fkwargs)
344
345 return forward
/usr/local/lib/python2.7/dist-packages/torch/autograd/function.pyc in _do_forward(self, *input)
200 self._nested_input = input
201 flat_input = tuple(_iter_variables(input))
--> 202 flat_output = super(NestedIOFunction, self)._do_forward(*flat_input)
203 nested_output = self._nested_output
204 nested_variables = _unflatten(flat_output, self._nested_output)
/usr/local/lib/python2.7/dist-packages/torch/autograd/function.pyc in forward(self, *args)
222 def forward(self, *args):
223 nested_tensors = _map_variable_tensor(self._nested_input)
--> 224 result = self.forward_extended(*nested_tensors)
225 del self._nested_input
226 self._nested_output = result
/usr/local/lib/python2.7/dist-packages/torch/nn/_functions/rnn.pyc in forward_extended(self, input, weight, hx)
283 hy = tuple(h.new() for h in hx)
284
--> 285 cudnn.rnn.forward(self, input, hx, weight, output, hy)
286
287 self.save_for_backward(input, hx, weight, output)
/usr/local/lib/python2.7/dist-packages/torch/backends/cudnn/rnn.pyc in forward(fn, input, hx, weight, output, hy)
253 w.zero_()
254 params = get_parameters(fn, handle, w)
--> 255 _copyParams(weight, params)
256
257 if tuple(hx.size()) != hidden_size:
/usr/local/lib/python2.7/dist-packages/torch/backends/cudnn/rnn.pyc in _copyParams(params_from, params_to)
181 for layer_params_from, layer_params_to in zip(params_from, params_to):
182 for param_from, param_to in zip(layer_params_from, layer_params_to):
--> 183 assert param_from.type() == param_to.type()
184 param_to.copy_(param_from)
185
AssertionError:
After I check the input shape, the example and my code have same input shape but it has different output shape.
Example MNIST shape:
torch.Size([100, 28, 28])
torch.Size([100])
While my shape is:
torch.Size([100, 10, 3])
torch.Size([100, 1])
Anybody know how to solved this error?, if my assumpion is right because of shape, do you know how to shape from torch.Size([100, 1])
into torch.Size([100])
?
-Thank You-
@yunjey ,Hi, when I read the code of "pytorch-tutorial/tutorials/02-intermediate/deep_residual_network/main.py", I find your implemention of the "_make_layer" function usese the function "conv3_3" to do the downsample, but whether it should use the "1_1 conv", it makes me confused.
Hello,
Thanks for the nice tutorial. I have a question, which confuses me a lot, but NOT an issue of your codes.
In [06-RNN line:55], you compute the output of RNN as "out = self.fc(out[:, -1, :]) ". In other words, you take the hidden values of the last element of all sequences.
My question is: if we need to take different element from each sequence, i.e. take the hidden values from different positions of the sequences, can the backpropagation work correctly?
For example:
x = Variable(torch.randn(3, 5, 6))
lstm = nn.LSTM(6, 3, batch_first=True)
fc = nn.Linear(3, 1)
a, (b, c) = lstm(x) # a is a 3 x 5 x 3 tensor
out = torch.stack([a[i, j] for i, j in enumerate([1,2,1])])
out = fc(out)
criterion = nn.MSELoss()
loss = criterion(out, targets) #targets is a 3x1 tensor
does loss.backward() computes the right grad for all parameters of fc and lstm?
Thank you so much!
When I Run the evaluation command
python sample.py --image='png/example.png'
I got this error
Traceback (most recent call last):
File "sample.py", line 97, in
main(args)
File "sample.py", line 61, in main
sampled_ids = decoder.sample(feature)
File "/users/PAS1273/osu8235/pytorch/pytorch-tutorial/tutorials/03-advanced/image_captioning/model.py", line 62, in sample
hiddens, states = self.lstm(inputs, states) # (batch_size, 1, hidden_size),
File "/users/PAS1273/osu8235/.local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 224, in call
result = self.forward(*input, **kwargs)
File "/users/PAS1273/osu8235/.local/lib/python2.7/site-packages/torch/nn/modules/rnn.py", line 162, in forward
output, hidden = func(input, self.all_weights, hx)
File "/users/PAS1273/osu8235/.local/lib/python2.7/site-packages/torch/nn/_functions/rnn.py", line 351, in forward
return func(input, *fargs, **fkwargs)
File "/users/PAS1273/osu8235/.local/lib/python2.7/site-packages/torch/autograd/function.py", line 284, in _do_forward
flat_output = super(NestedIOFunction, self)._do_forward(*flat_input)
File "/users/PAS1273/osu8235/.local/lib/python2.7/site-packages/torch/autograd/function.py", line 306, in forward
result = self.forward_extended(*nested_tensors)
File "/users/PAS1273/osu8235/.local/lib/python2.7/site-packages/torch/nn/_functions/rnn.py", line 293, in forward_extended
cudnn.rnn.forward(self, input, hx, weight, output, hy)
File "/users/PAS1273/osu8235/.local/lib/python2.7/site-packages/torch/backends/cudnn/rnn.py", line 208, in forward
'input must have 3 dimensions, got {}'.format(input.dim()))
RuntimeError: input must have 3 dimensions, got 2
Hi,
I have tested your train.py script in captioning tutorial to run over my custom dataset. After some debugging it seems the script is not looping back after the first iteration. This is last line executed by the script and then I don't see any output for hours. Below is the output of the script:
python train.py
Epoch [0/5], Step [0/22], Loss: 4.7755, Perplexity: 118.5671
Have you tested this on custom dataset? Can I get some pointers what might be wrong here.
Thanks
Hi guys,
Image captioning doesn't contain a main.py / main-gpu.py file as suggested by your main Readme file. It would be really helpful if you can add some instructions how to run inference on input images using existing models or start training on some custom dataset.
Thanks
i am testing out different lengths of text for the caption of an image and I always get out of memory if I go over 200 characters. I am using Titan X 12 GB and I have 64 GB of ram. it seems like this shouldnt die with out of memory issues on 200 characters. How can I get a longer sequence outputted? If I implement beam search, would that resolve this? I also tried cpu mode for sampling and get the same error:
python sample.py --num_layers=3 --length=300 --vocab_path=./models/v2/vocab.pkl --image=/home/jtoy/sandbox/06.jpg der=./models/v2/encoder-60-532.pkl --num_layers=3 THCudaCheck FAIL file=/py/conda-bld/pytorch_1493680494901/work/torch/lib/THC/generic/THCStorage.cu line=66 error=2 : out of memory Traceback (most recent call last): File "sample.py", line 95, in <module> main(args) File "sample.py", line 54, in main sampled_ids = decoder.sample(feature, state,args.length) File "/home/jtoy/sandbox/pytorch/model.py", line 60, in sample hiddens, states = self.lstm(inputs, states) # (batch_size, 1, hidden_size) File "/home/jtoy/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 206, in __call__ result = self.forward(*input, **kwargs) File "/home/jtoy/anaconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 91, in forward output, hidden = func(input, self.all_weights, hx) File "/home/jtoy/anaconda3/lib/python3.6/site-packages/torch/nn/_functions/rnn.py", line 343, in forward return func(input, *fargs, **fkwargs) File "/home/jtoy/anaconda3/lib/python3.6/site-packages/torch/autograd/function.py", line 202, in _do_forward flat_output = super(NestedIOFunction, self)._do_forward(*flat_input) File "/home/jtoy/anaconda3/lib/python3.6/site-packages/torch/autograd/function.py", line 224, in forward result = self.forward_extended(*nested_tensors) File "/home/jtoy/anaconda3/lib/python3.6/site-packages/torch/nn/_functions/rnn.py", line 285, in forward_extended cudnn.rnn.forward(self, input, hx, weight, output, hy) File "/home/jtoy/anaconda3/lib/python3.6/site-packages/torch/backends/cudnn/rnn.py", line 272, in forward fn.workspace = torch.cuda.ByteTensor(workspace_size.value) RuntimeError: cuda runtime error (2) : out of memory at /py/conda-bld/pytorch_1493680494901/work/torch/lib/THC/generic/THCStorage.cu:66
Hi, thanks a lot for this wonderful repo and recent work on tensorboard!
I just wonder when would be graph visualization of tensorboard for pytorch could be available.
any idea?
Thanks again for your great work!
@yunjey How we can train your image captioning code on another dataset? If we prepare a JSON file like (http://msvocds.blob.core.windows.net/annotations-1-0-3/captions_train-val2014.zip) the code can be trained on a new dataset?
I want to train the model with my own dataset, but I have a question.
Why is default image size in resize.py 256? I think it must be 224 as same as resnet's input size.
I try to train model by using my own datasets, but i have a problem of character encoding, i try to change the caption into Chinese, but the vab.pkl has many ' S'\xe7\xba\xa6' ' and when i use my trained model,it always gives /</start/>/</unk/>/</end/>/ ,so, could you please tell me how to solve this problem?Thank you very much.
when i reach that link,it show 404 not found.
hi there,
First of all, This is really good tutorial, thank you for your work!
I am having some troubles understanding Logistic Regression example here. What bothers me is lack of clear activation function at the very end of the graph. From what I see I conclude this is in fact softmax classifier rather than logistic regression. Am I missing smth?
There are many libraries needed to run these tutorials. lets add the requirements in a file to make it easy to run these.
Hi,
I have been wondering this for a while; in https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/02-intermediate/recurrent_neural_network/main.py, why are you setting new hidden and cell states for each forward propagation? Why are h0 and c0 not being passed in as arguments to the forward function and returned after the forward prop. Example:
h0 = nn.Parameter(torch.randn(num_layers*2, 100, hidden_size)) # 2 for bidirection
c0 = nn.Parameter(torch.randn(num_layers*2, 100, hidden_size))
for epoch in range(num_epochs):
for i, (images, labels) in enumerate(train_loader):
images = Variable(images.view(-1, sequence_length, input_size))
labels = Variable(labels)
optimizer.zero_grad()
outputs, h0, c0 = rnn.forward(images, h0, c0)
Dont the partial derivatives of the hidden and cell states also effect the accuracy of the model?
training /test/validation data? I saw in image captioning and the other examples that there is no training/test/validation splits. Should the examples have this to promote best practices? If I wrote code to add it, would it be merged in?
When I followed the command: python sample.py --image='png/example.png', it returns an error:
<start> guest building storing guest next gazelle 6 drawing guest applied pate <end>
Traceback (most recent call last):
File "sample.py", line 96, in <module>
main(args)
File "sample.py", line 75, in main
plt.imshow(np.asarray(image))
File "/home/marsha/anaconda2/lib/python2.7/site-packages/matplotlib/pyplot.py", line 3029, in imshow
**kwargs)
File "/home/marsha/anaconda2/lib/python2.7/site-packages/matplotlib/__init__.py", line 1819, in inner
return func(ax, *args, **kwargs)
File "/home/marsha/anaconda2/lib/python2.7/site-packages/matplotlib/axes/_axes.py", line 4922, in imshow
im.set_data(X)
File "/home/marsha/anaconda2/lib/python2.7/site-packages/matplotlib/image.py", line 449, in set_data
raise TypeError("Image data can not convert to float")
TypeError: Image data can not convert to float
Hi,
Thank you for your great tutorial.
Could you please let me know what you meant from "# Softmax is internally computed." in the this file?
fixed_x, _ = next(data_iter) torchvision.utils.save_image(fixed_x.data.cpu(), './data/real_images.png')
The code in VAE: line 68: fixed_x.data.cpu()
change to fixed_x
(the fixed_x is FloatTensor type. Not a Variable)
Hi!
First of all, really nice resource for learning pytorch and neural nets!
I am following your tutorial for preparing a short hands-on tutorial
During this process, I have identified a very small understandability issue here https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/00%20-%20PyTorch%20Basics/main.py#L74
If understand correctly, your intention is to show how the loss goes down after the first opt step, but you do not make a new forward pass with the new weights, thus the loss is the same as previously. Making linear(x) either in the call or more explicitly reassigning pred = linear(x) would help to show how the loss is reduced. As I said, it is a minor issue but thought it might help!
Thank you!
Nvm, I found my error. Sorry
I have a question about the evaluation mode. I found that in the resnet tutorial, the network is not set to evaluation through resnet.eval(). Will this affect the testing accuracy? Thanks!
Love the tutorials.
I see that for the RNNs you use variable length data. Some of the tutorials use pack_padded_Sequence and not, for example this RNN uses one:
https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/03-advanced/image_captioning/model.py#L52
but this one doesnt:
https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/02-intermediate/language_model/main-gpu.py#L32
Why do you do it like this? Is pack_padded_sequence needed? Could both of those models be written with and without pack_padded_sequence
project 11 - linux python3.5
Files already downloaded and verified
Epoch [0/50], Step[100/500], d_loss: 1.1626, g_loss: 0.9049, D(x): 0.70, D(G(z)): 0.54
Traceback (most recent call last):
File "main.py", line 130, in
'./data/fake_samples_%d_%d.png' %(epoch+1, i+1))
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/torchvision-0.1.8-py3.5.egg/torchvision/utils.py", line 95, in save_image
File "/home/andrewcz/miniconda3/lib/python3.5/site-packages/PIL/Image.py", line 1682, in save
fp = builtins.open(filename, "wb")
FileNotFoundError: [Errno 2] No such file or directory: './data/fake_samples_1_100.png'
Having trouble with this example.
Thanks for a fantastic tutorial. Really clear and easy to follow!
I'd like to run the pretrained models on a CPU, and am trying to convert the models as follows:
encoder.load_state_dict(torch.load(args.encoder_path, map_location=lambda storage, loc: storage))
decoder.load_state_dict(torch.load(args.decoder_path, map_location=lambda storage, loc: storage))
I also tried:
encoder = torch.load(args.encoder_path, map_location=lambda storage, loc: storage)
decoder = torch.load(args.decoder_path, map_location=lambda storage, loc: storage)
as advised by the pytorch developers (https://discuss.pytorch.org/t/on-a-cpu-device-how-to-load-checkpoint-saved-on-gpu-device/349/8)
Based on the discussions it seems that it might be a pytorch versioning issue. Could you let me know what version of pytorch you used to train and save the models? Or if it's something else, would really appreciate any guidance in getting it fixed.
Many thanks!
Hello,
In train.py of Image Captioning tutorial, you import get_loader as:
"from data_loader import get_loader "
But I didn't find the corresponding file in the repos. Do I miss anything here? Thanks.
Hello,
I have a quick question on calling of the lstm() in model.py.
In the training step, you simply run lstm layer as: "hiddens, _ = self.lstm(packed)", without assigning the states.
Differently, in the sampling step, you send the features of EncoderRNN to the lstm layer as: "hiddens, states = self.lstm(inputs, states)".
I wonder that why we send the state parameters in sampling step. Because it seems to me that the returned states are not used in sample() function, can we just do "hiddens, _ = self.lstm(inputs)"?
Thanks.
Hi,
First thank you for sharing the tutorials!
I have a quick question on the GAN example, where 'detaching' the variable should be used in my opinion like fake_images = G(z).detach()
, to avoid the training of Discriminator influences the Generator's behaviour at this stage. Or you have specific considerations here?
Thank you!
Hello,
I am reading your codes of image captioning, and I have a question here:
In model.py you attach the features from EncoderCNN to the captions as:
"embeddings = torch.cat( (features.unsqueeze(1), embeddings), 1)" and this is the input of the lstm layer.
In train.py, you define the targets as:
"targets = pack_padded_sequence(captions, lengths, batch_first=True)[0]" .
My question is, these two tensors are of different sizes (I think the former is "one word" longer on each caption than the later). How can they be compared? Thanks!
Hi,
Thanks for the great set of tutorials.
I see that you have packed both the output or Decoder and targets into a PackedSequence
and applied CrossEntropy
criterion on them here.
But by directly applying the loss, don't you end up taking the mean loss over all samples? Since pack_padded_sequence
rearranges your sequences in batches, doesn't this mess things up? i.e. shouldn't you sum over the sequence loss and take the mean over batch?
Hope I'm not being confusing
Hi, @jtoy @hunkim @Kongsea @DingKe @jayparks
I met this error in deep_convolution_gan.py :
usage: ipykernel_launcher.py [-h] [--image_size IMAGE_SIZE] [--z_dim Z_DIM]
[--g_conv_dim G_CONV_DIM]
[--d_conv_dim D_CONV_DIM]
[--num_epochs NUM_EPOCHS]
[--batch_size BATCH_SIZE]
[--sample_size SAMPLE_SIZE]
[--num_workers NUM_WORKERS] [--lr LR]
[--beta1 BETA1] [--beta2 BETA2] [--mode MODE]
[--model_path MODEL_PATH]
[--sample_path SAMPLE_PATH]
[--image_path IMAGE_PATH] [--log_step LOG_STEP]
[--sample_step SAMPLE_STEP]
ipykernel_launcher.py: error: unrecognized arguments: -f /Users/dti/Library/Jupyter/runtime/kernel-0a3c5ccc-79ba-4805-8668-2c85aa0e488d.json
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
I used my images(128x128, RGB, 24files).
What's wrong with me?
Hi,I retrain Image Captioning model.It works well in training,but failed in testing.Here is my error.
xiang@xiang-Z270X-Gaming-K7:~/pytorch-tutorial/tutorials/03-advanced/image_captioning$ python sample.py --image='png/example.png'
showing info https://raw.githubusercontent.com/nltk/nltk_data/gh-pages/index.xml
Traceback (most recent call last):
File "sample.py", line 97, in
main(args)
File "sample.py", line 61, in main
sampled_ids = decoder.sample(feature)
File "/home/xiang/pytorch-tutorial/tutorials/03-advanced/image_captioning/model.py", line 62, in sample
hiddens, states = self.lstm(inputs, states) # (batch_size, 1, hidden_size),
File "/home/xiang/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 225, in call
result = self.forward(*input, **kwargs)
File "/home/xiang/anaconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py", line 91, in forward
output, hidden = func(input, self.all_weights, hx)
File "/home/xiang/anaconda3/lib/python3.6/site-packages/torch/nn/_functions/rnn.py", line 343, in forward
return func(input, *fargs, **fkwargs)
File "/home/xiang/anaconda3/lib/python3.6/site-packages/torch/autograd/function.py", line 284, in _do_forward
flat_output = super(NestedIOFunction, self)._do_forward(*flat_input)
File "/home/xiang/anaconda3/lib/python3.6/site-packages/torch/autograd/function.py", line 306, in forward
result = self.forward_extended(*nested_tensors)
File "/home/xiang/anaconda3/lib/python3.6/site-packages/torch/nn/_functions/rnn.py", line 285, in forward_extended
cudnn.rnn.forward(self, input, hx, weight, output, hy)
File "/home/xiang/anaconda3/lib/python3.6/site-packages/torch/backends/cudnn/rnn.py", line 205, in forward
'input must have 3 dimensions, got {}'.format(input.dim()))
RuntimeError: input must have 3 dimensions, got 2
What size cards are these networks tested and trained on? I just tried running "09 - Image Captioning" and I immediately get errors. I am testing this on a Titan X with 12 GB of memory:
Namespace(batch_size=128, caption_path='./data/annotations/captions_train2014.json', crop_size=224, embed_size=256, hidden_size=512, image_dir='./data/resized2014', learning_rate=0.001, log_step=10, model_path='./models/', num_epochs=5, num_layers=1, num_workers=2, save_step=1000, vocab_path='./data/vocab.pkl')
loading annotations into memory...
Done (t=0.89s)
creating index...
index created!
THCudaCheck FAIL file=/b/wheel/pytorch-src/torch/lib/THC/generic/THCStorage.cu line=66 error=2 : out of memory
Traceback (most recent call last):
File "train.py", line 119, in <module>
main(args)
File "train.py", line 66, in main
features = encoder(images)
File "/root/.venv/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/root/pytorch/pytorch-tutorial/tutorials/09 - Image Captioning/model.py", line 26, in forward
features = self.resnet(images)
File "/root/.venv/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/root/.venv/local/lib/python2.7/site-packages/torchvision/models/resnet.py", line 146, in forward
x = self.layer3(x)
File "/root/.venv/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/root/.venv/local/lib/python2.7/site-packages/torch/nn/modules/container.py", line 64, in forward
input = module(input)
File "/root/.venv/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/root/.venv/local/lib/python2.7/site-packages/torchvision/models/resnet.py", line 85, in forward
out = self.bn3(out)
File "/root/.venv/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
result = self.forward(*input, **kwargs)
File "/root/.venv/local/lib/python2.7/site-packages/torch/nn/modules/batchnorm.py", line 43, in forward
self.training, self.momentum, self.eps)
File "/root/.venv/local/lib/python2.7/site-packages/torch/nn/functional.py", line 463, in batch_norm
return f(input, weight, bias)
RuntimeError: cuda runtime error (2) : out of memory at /b/wheel/pytorch-src/torch/lib/THC/generic/THCStorage.cu:66
Command exited with non-zero status 1
can you provide links to the datasets you use?
in particular whihc treebank dataset are you using here: https://github.com/yunjey/pytorch-tutorial/blob/master/tutorials/02-intermediate/language_model/main.py#L20 ?
Im trying to replicate your tutorials down to exact data.
Hi,
Looks like the weights starting from the second epoch, gives an issue while testing with sample image.
The output is only sequence of start ...
When i debugged, I see that the "outputs" tensor array in model.py contains all "nan"
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.