benedekrozemberczki/attentionwalk

A PyTorch Implementation of "Watch Your Step: Learning Node Embeddings via Graph Attention" (NeurIPS 2018).

Benedek A. Rozemberczki/ Homepage / Twitter / GitHub / Google Scholar

Welcome stranger

• ⏰ Currently working on machine learning for drug discovery.
• 🤖 I would love to collaborate on the machine learning libraries ChemicalX and RexMex.

Great news

• 🧬 MOOMIN: Deep Molecular Omics Network for Anti-Cancer Drug Combination Therapy was accepted at CIKM 2022.
• 🪙 The Shapley Value in Machine Learning was accepted at IJCAI 2022.
• ⭐ A Unified View of Relational Deep Learning for Drug Pair Scoring was accepted at IJCAI 2022.
• ⚗️ ChemicalX: A Deep Learning Library for Drug Pair Scoring was accepted at KDD 2022.

attentionwalk's Issues

can I construct graph from binary matrix?

Sorry to bother you.
if I have a binary matrix like this, can I construct graph from this matrix?

Memory Error

I'm getting OOM errors even with small files. The attached file link_network.txt throws the following error:

Adjacency matrix powers: 100%|███████████████████████████████████████████████████████| 4/4 [00:00<00:00, 108.39it/s]
Traceback (most recent call last):
  File "src\main.py", line 79, in <module>
    main()
  File "src\main.py", line 74, in main
    model = AttentionWalkTrainer(args)
  File "E:\AttentionWalk\src\attentionwalk.py", line 70, in __init__
    self.initialize_model_and_features()
  File "E:\AttentionWalk\src\attentionwalk.py", line 76, in initialize_model_and_features
    self.target_tensor = feature_calculator(self.args, self.graph)
  File "E:\AttentionWalk\src\utils.py", line 53, in feature_calculator
    target_matrices = np.array(target_matrices)
MemoryError

I guess this is due to the large indices of the nodes. Any workarounds for this?

miscalculations of normalized adjacency matrix

Thanks for sharing this awesome repo.

The issue is I found that loss_on_target will become extreme big while training from the original code, and I think is due to the miscalculation of normalized_adjacency_matrix .

From your original code, normalized_adjacency_matrix is been calculated by:

normalized_adjacency_matrix = degs.dot(adjacency_matrix)

However while the matrix hasn't been normalize but simply multiple by degree of nodes.
I think the part of normalized_adjacency_matrix should be modified like its original definition:

  normalized_adjacency_matrix = degs.power(-1/2)\
                                    .dot(adjacency_matrix)\
                                    .dot(degs.power(-1/2))

It'll turn out to be more reasonable loss shown below:

Am I understand it correctly?

embedding result

Hello! My research interest is link prediction. In def save_embedding(self), can save the embedding result with a vector? How to solve? Thank you

Directed weighted graphs

Is it possible to use the code with directed and weighted graphs? The paper states the attention walk framework for unweighted graphs only, but i'd like to use it for such types of networks.
Thank you for your attention.

left_factors and right_factors' size don't match

Thank you for sharing the code.
I encounter this error.

Are directed graphs supported?

Excellent work. I am just wondering if the directed graghs are supported in this implimentation?

Nan parameters

Thanks for your pytorch code. I found that my parameters become Nan during training.
Nan parameters include model.left_factors, model.right_factors, model.attention. All the entries of them become Nan during training. And also the loss.
I'm trying to find the reason. I would appreciate it if you could give me some help or hints.

problem with being killed

Hi, I tried to train the model with new dataset which have about 60000 nodes,
but I have a problem of getting Killed suddenly. Do you have any idea why?
Thanks :)