arunarn2/hierarchicalattentionnetworks

Hierarchical Attention Networks for Document Classification in Keras

hierarchicalattentionnetworks's Issues

Masking error

Trying to run the network in Google Colab, throws an error due to the mask dimensions:

Running it on the computer returns
ValueError: Dimensions must be equal, but are 15 and 100 for 'hierarchical_attention_network_2/mul' (op: 'Mul') with input shapes: [?,15], [?,15,100].

Any help on how to fix this much appreciated,
Cheers!

Different implementation in keras and tensorflow?

Looking at the tensorflow code, it seems to me that tf version has only one bilstm encoder ( only for words).
Is that correct implementation?

Model not learning

I'm trying your HiarchicalAttentionNetworks in keras and no matter what I do, it predicts 1. I'm running on tf.keras on the amazon fine food reviews. (I replaced self.trainable_weights by self._trainable_weights to make it compile)

Here's a code sample

#Input
word_input = Input(shape=(max_senten_len,), dtype="float32")
#Embeddings
word_sequences = Embedding(input_dim=vocab_length, output_dim=EMBEDDING_DIM, 
                           weights=[embedding_matrix], input_length=max_senten_len, 
                           trainable=False)(word_input)
#Words GRU
word_gru=GRU(units = 100, return_sequences=True,  
             kernel_regularizer=l2_reg)(word_sequences)
#Words attention
word_att = HierarchicalAttention(100)(word_gru)
wordEncoder = Model(word_input, word_att)

#Sentences part
sent_input = Input(shape=(max_senten_num, max_senten_len), dtype='float32')
#Sentence Input
sent_encoder = TimeDistributed(wordEncoder)(sent_input)
#Sentence GRU
sent_gru=GRU(units = 100, return_sequences=True, 
             kernel_regularizer=l2_reg)(sent_encoder)

#Sentences attention
sent_att = Dropout(0.4)(HierarchicalAttention(100)(sent_gru))
#Prediction
preds = Dense(1, activation='sigmoid')(sent_att)
#Model
model_HAN = Model(sent_input, preds)
model_HAN.compile(loss='binary_crossentropy', optimizer="adam", metrics=['acc'])
model_HAN.summary()```

Visualization for keras version?

I tried to return the alphas (the attentions) in the class AttLayer(Layer), but Keras gets the error "you must feed value to sentence_input", Can you help me fix this problem?