roebius/deeplearning_keras2

Modification of fast.ai deep learning course notebooks for usage with Keras 2 and Python 3.

deeplearning_keras2's Issues

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

Environment Python 3 within Anaconda running on Mac OS X (v10.12.6)

The code in lesson5.ipynb for Pre-trained vectors, also found in wordvectors.ipynb doesn't work for me.

def load_vectors(loc):
    return (load_array(loc+'.dat'),
        pickle.load(open(loc+'_words.pkl','rb')),
        pickle.load(open(loc+'_idx.pkl','rb')))

vecs, words, wordidx = load_vectors('data/glove/results/6B.50d')

---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)
<ipython-input-14-f3d9d0357284> in <module>()
----> 1 vecs, words, wordidx = load_glove(res_path+'6B.50d')
      2 vecs.shape

<ipython-input-13-7df1842b2042> in load_glove(loc)
      1 def load_glove(loc):
      2     return (load_array(loc+'.dat'),
----> 3         pickle.load(open(loc+'_words.pkl', 'rb')),
      4         pickle.load(open(loc+'_idx.pkl','rb')))

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 0: ordinal not in range(128)

Tiramisu with non-square images?

Just curious; is there a relatively easy way to adjust tiramisu-keras for non-square images?

ValueError: ('The specified size contains a dimension with value <= 0', (4096, 0))

ubuntu 16.04 LTS, no GPU w/ python 3.5.2
all packages were installed recently (as of 10/7/2017)
Made sure theano backend is setup

in Lesson1.ipynb, i am getting this error.

Any deas?

vgg = Vgg16()
# Grab a few images at a time for training and validation.
# NB: They must be in subdirectories named based on their category
batches = vgg.get_batches(path+'train', batch_size=batch_size)
val_batches = vgg.get_batches(path+'valid', batch_size=batch_size*2)
vgg.finetune(batches)
vgg.fit(batches, val_batches, batch_size, nb_epoch=1)

Found 0 images belonging to 0 classes.
Found 0 images belonging to 0 classes.

ValueError Traceback (most recent call last)
in ()
4 batches = vgg.get_batches(path+'train', batch_size=batch_size)
5 val_batches = vgg.get_batches(path+'valid', batch_size=batch_size*2)
----> 6 vgg.finetune(batches)
7 vgg.fit(batches, val_batches, batch_size, nb_epoch=1)

/mnt/hgfs/T/security/fast.ai/courses/deeplearning1/nbs/vgg16.py in finetune(self, batches)
95
96 def finetune(self, batches):
---> 97 self.ft(batches.num_class) # Keras2
98 classes = list(iter(batches.class_indices))
99 for c in batches.class_indices:

/mnt/hgfs/T/security/fast.ai/courses/deeplearning1/nbs/vgg16.py in ft(self, num)
91 model.pop()
92 for layer in model.layers: layer.trainable=False
---> 93 model.add(Dense(num, activation='softmax'))
94 self.compile()
95

/usr/local/lib/python3.5/dist-packages/keras/models.py in add(self, layer)
453 output_shapes=[self.outputs[0]._keras_shape])
454 else:
--> 455 output_tensor = layer(self.outputs[0])
456 if isinstance(output_tensor, list):
457 raise TypeError('All layers in a Sequential model '

/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py in call(self, inputs, **kwargs)
526 'layer.build(batch_input_shape)')
527 if len(input_shapes) == 1:
--> 528 self.build(input_shapes[0])
529 else:
530 self.build(input_shapes)

/usr/local/lib/python3.5/dist-packages/keras/layers/core.py in build(self, input_shape)
825 name='kernel',
826 regularizer=self.kernel_regularizer,
--> 827 constraint=self.kernel_constraint)
828 if self.use_bias:
829 self.bias = self.add_weight((self.units,),

/usr/local/lib/python3.5/dist-packages/keras/engine/topology.py in add_weight(self, shape, initializer, name, trainable, regularizer, constraint)
362 """
363 initializer = initializers.get(initializer)
--> 364 weight = K.variable(initializer(shape), dtype=K.floatx(), name=name)
365 if regularizer is not None:
366 self.add_loss(regularizer(weight))

/usr/local/lib/python3.5/dist-packages/keras/initializers.py in call(self, shape, dtype)
203 limit = np.sqrt(3. * scale)
204 return K.random_uniform(shape, -limit, limit,
--> 205 dtype=dtype, seed=self.seed)
206
207 def get_config(self):

/usr/local/lib/python3.5/dist-packages/keras/backend/theano_backend.py in random_uniform(shape, minval, maxval, dtype, seed)
1967 seed = np.random.randint(1, 10e6)
1968 rng = RandomStreams(seed=seed)
-> 1969 return rng.uniform(shape, low=minval, high=maxval, dtype=dtype)
1970
1971

/usr/local/lib/python3.5/dist-packages/theano/sandbox/rng_mrg.py in uniform(self, size, low, high, ndim, dtype, nstreams)
1342 raise ValueError(
1343 "The specified size contains a dimension with value <= 0",
-> 1344 size)
1345
1346 else:

ValueError: ('The specified size contains a dimension with value <= 0', (4096, 0))

Lesson 4 - loss blows up

Have you seen that sometimes the loss explodes when running the collaborative filtering?

Suggestion: Set Keras parameters programmatically instead of via JSON

Instead of having users copy over a JSON config file to run a notebook, why not add the following to every lesson:

from keras import backend as K
K.set_image_data_format('channels_last')
K.set_image_dim_ordering('th')

This works for me without any issue while allowing me to preserve my existing Keras config. I'm using a Tensorflow backend on an NVidia GPU.

InvalidArgumentError: Negative dimension size caused by subtracting 2 from 1 for 'max_pooling2d_10/MaxPool' (op: 'MaxPool') with input shapes: [?,1,112,128]

When I run the lesson1 notebook, I get the above mentioned error. Quick google search brought me here. How do I resolve this?

'DirectoryIterator' object has no attribute 'num_class'

Hi Robi,

Thanks for sharing this great work. I tried the copy for lesson01.

I have the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-9-e637d623c59d> in <module>()
----> 1 vgg.finetune(batches)
      2 vgg.fit(batches, val_batches, batch_size, nb_epoch=1)
      3 

C:\Users\Wen\Documents\Python file_Arif\fastai\vgg16.py in finetune(self, batches)
     95 
     96     def finetune(self, batches):
---> 97         self.ft(batches.num_class)  # Keras2
     98         classes = list(iter(batches.class_indices))
     99         for c in batches.class_indices:

AttributeError: 'DirectoryIterator' object has no attribute 'num_class'

Do you know what is the problem?

As a note:

• I've updated the .keras/keras.json file as suggested (I'm using theano btw)
• I'm using Keras 2.1.2

Vgg weights and jason file?

I was trying to run the file but it seems that the vgg.py file downloads vgg weights and imagenet class index in jason format from a web adress 'www.platform.ai' which is not working currently. Can you provide a link to the cached weights and the jason file? Particularly the jason file? I need them to run the file.
Thanks in advance.

Tensorflow support

Hi, Roebius,

I try to use your fastai notebook in the environment with tensorflow. And it always fail. I know that we need to change the image_dim_order. And I have changed the keras.json to use tensorflow.

What do I miss any other changes?

Thanks

Can't pip install pygpu on Python 3.5 / Ubuntu 16

When I try to run the following to ensure I have the necessary packages for Part 1 with Keras 2 and Python 3.5 on Ubuntu 16.04:
pip install -r requirements_part1_ubuntu.txt , I always get an error regarding the install of pygpu.

I can reproduce this error if I simply type either of the following as well:
pip install pygpu
pip install pygpu==0.6.4

Could not find a version that satisfies the requirement pygpu (from versions: )
No matching distribution found for pygpu

Now I did find a way around this by following the instruction here: http://deeplearning.net/software/libgpuarray/installation.html#step-by-step-install However, it would definitely way more convenient if one could install pygpu via pip instead of the much longer/verbose process at that link.

Any idea how one can get pip to install pygpu (in the context of Python 3.5 / Ubuntu 16)? Anything I may be missing?

Thanks

Stateful model with keras section returns loss: nan

I'm using the theano backend, and the LSTM example does not work. The statements:

model=Sequential([
        Embedding(vocab_size, n_fac, input_length=cs, batch_input_shape=(bs,8)),
        BatchNormalization(),
        LSTM(n_hidden, return_sequences=True, stateful=True),
        TimeDistributed(Dense(vocab_size, activation='softmax')),
    ])
model.compile(loss='sparse_categorical_crossentropy', optimizer=Adam())
mx = len(x_rnn)//bs*bs
model.fit(x_rnn[:mx], y_rnn[:mx], batch_size=bs, epochs=4, shuffle=False)

returns outputwith a nan-value for the loss.

Epoch 1/4
75072/75072 [==============================] - 20s - loss: nan     
Epoch 2/4
75072/75072 [==============================] - 20s - loss: nan     
Epoch 3/4
75072/75072 [==============================] - 20s - loss: nan     
Epoch 4/4
75072/75072 [==============================] - 20s - loss: nan

Calling Vgg16() in lesson1.ipynb gives error: The shape of the input to "Flatten" is not fully defined (got (0, 7, 512)

Following installation of all requirements for Part 1 on Ubuntu 16 / Python 3.5:

If I just go through lesson1.ipynb (part 1), which has the dogscats path, as soon as I try to execute
vgg=Vgg16()
I get the error below:

ValueError Traceback (most recent call last)
in ()
----> 1 vgg = Vgg16()

/home/bluelight/projects/fastai1/deeplearning_keras2/nbs/vgg16.py in init(self)
30 def init(self):
31 self.FILE_PATH = 'http://files.fast.ai/models/'
---> 32 self.create()
33 self.get_classes()
34

/home/bluelight/projects/fastai1/deeplearning_keras2/nbs/vgg16.py in create(self)
73 self.ConvBlock(3, 512)
74
---> 75 model.add(Flatten())
76 self.FCBlock()
77 self.FCBlock()

/home/bluelight/Envs/fastai1py3k2/lib/python3.5/site-packages/keras/models.py in add(self, layer)
461 output_shapes=[self.outputs[0]._keras_shape])
462 else:
--> 463 output_tensor = layer(self.outputs[0])
464 if isinstance(output_tensor, list):
465 raise TypeError('All layers in a Sequential model '

/home/bluelight/Envs/fastai1py3k2/lib/python3.5/site-packages/keras/engine/topology.py in call(self, inputs, **kwargs)
581 # Infering the output shape is only relevant for Theano.
582 if all([s is not None for s in _to_list(input_shape)]):
--> 583 output_shape = self.compute_output_shape(input_shape)
584 else:
585 if isinstance(input_shape, list):

/home/bluelight/Envs/fastai1py3k2/lib/python3.5/site-packages/keras/layers/core.py in compute_output_shape(self, input_shape)
486 raise ValueError('The shape of the input to "Flatten" '
487 'is not fully defined '
--> 488 '(got ' + str(input_shape[1:]) + '. '
489 'Make sure to pass a complete "input_shape" '
490 'or "batch_input_shape" argument to the first '

ValueError: The shape of the input to "Flatten" is not fully defined (got (0, 7, 512). Make sure to pass a complete "input_shape" or "batch_input_shape" argument to the first layer in your model.