twitter-research / sign Goto Github PK

SIGN: Scalable Inception Graph Network

License: Apache License 2.0

Python 100.00%

sign's Introduction

SIGN: Scalable Inception Graph Networks [arXiv]

This repository contains the code to run the SIGN model on the ogbn-papers100M dataset, the largest, publicly available node classification benchmark at the time of writing. If you want to know more about SIGN checkout its arXiv page and its ICML 2020 GRL+ Workshop version.

Requirements

Dependencies with python 3.8.5:

torch==1.5.0
torch_geometric==1.6.1
torch_scatter==2.0.5
torch_sparse==0.6.7
ogb==1.2.3

Preprocessing, Training & Evaluation

# Generate SIGN features and save them as sign_333_embeddings.pt
python preprocessing.py --file_name sign_333_embeddings --undirected --directed --directed_asymm_norm --undirected_set_diag --directed_remove_diag

# Train SIGN model based on the preprocessed features generated above and write results at sign_results.txt
python sign_training.py --dropout 0.3 --lr 0.00005 --hidden_channels 512 --num_layers 3 --embeddings_file_name sign_333_embeddings.pt --result_file_name sign_results.txt

# Train SIGN-xl model based on the preprocessed features generated above and write results at sign-xl_results.txt
python sign_training.py --dropout 0.5 --lr 0.00005 --hidden_channels 2048 --num_layers 3 --embeddings_file_name sign_333_embeddings.pt --result_file_name sign-xl_results.txt

Cite us

@inproceedings{sign_icml_grl2020,
    title={SIGN: Scalable Inception Graph Neural Networks},
    author={Fabrizio Frasca and Emanuele Rossi and Davide Eynard and Benjamin Chamberlain and Michael Bronstein and Federico Monti},
    booktitle={ICML 2020 Workshop on Graph Representation Learning and Beyond},
    year={2020}
}

sign's People

Contributors

Stargazers

Watchers

Forkers

jingmouren beesitech thanit456 royvelich astolman sharique1006 nadavru jinqingyu mro15 hwr0318 minhtuankha chamezopoulos sunilgitb levinx01

sign's Issues

Requesting access to code used for the operators in the paper

Dear Authors,

Thanks for sharing the codebase for SIGN. I really enjoyed reading the paper and found it to be really interesting. However, I was wondering if you could share the code for generating the operators used in the paper. Specifically, I am interested in the code for the "random-walk normalized PPR diffusion" and "row-normalized triangle-induced adjacency matrix" operators used.

As far as I know, only the powers of the normalized adjacency matrix operator is included in this repository. I did go through some closed issues and was made aware that the codebase might be updated in the future with the full code. Even though the SIGN model is now available in PyG and DGL, I believe it will be very beneficial to have access to the original code to reproduce the results in the paper. If you could point me to some other repository used to generate the other two operators, that would be greatly appreciated as well.

Warm Regards,
Paul Louis

How is the concatenation operation performed in the model?

Hi, thank you for sharing your work!

I am trying to understand the implementation from the paper and I came across a question about the concatenation operation. I assume that the MLP class from the code represents the architecture from Figure 1.

The forward pass in the code, however, does not explicitly contain a concatenate operation. Looking at this loop it seems that the layers are being applied in cascade instead of forming parallel branches.

    def forward(self, x):
        for i, lin in enumerate(self.lins[:-1]):
            x = lin(x)
            x = self.bns[i](x)
            x = F.relu(x)
            x = F.dropout(x, p=self.dropout, training=self.training)
        x = self.lins[-1](x)
        return torch.log_softmax(x, dim=-1)

In other words, what I would expect is to have something similar to this (I have not tested the code):

    def forward(self, x):
        branches = []
        for i, lin in enumerate(self.lins[:-1]):
            _x = lin(x)
            _x = self.bns[i](_x)
            _x = F.relu(_x)
            _x = F.dropout(_x, p=self.dropout, training=self.training)
           branches.append(_x)
        x = torch.cat(branches)
        x = self.lins[-1](x)
        return torch.log_softmax(x, dim=-1)

Can you indicate me whether or not this interpretation is correct? Am I missing something and the concatenation takes place in some other part of the code? Thank you in advance.

The code is different from the paper

In the paper, GCN-normalized, PPR-based, and triangle-based adjacency matrices are utilized, but I cannot find those adjacency matrices in the code. Would you update code as the implementation of the paper?

It seems like pytorch_sparse issue:

It seems like pytorch_sparse issue:
Traceback (most recent call last):
File "preprocessing.py", line 142, in
main()
File "preprocessing.py", line 100, in main
adj = get_adj(row, col, N, asymm_norm=args.undirected_asymm_norm, set_diag=args.undirected_set_diag, remove_diag=args.undirected_remove_diag)
File "preprocessing.py", line 19, in get_adj
adj = adj.set_diag()
File "/home/zhu044/.local/lib/python3.6/site-packages/torch_sparse/diag.py", line 96, in
self, values, k)
File "/home/zhu044/.local/lib/python3.6/site-packages/torch_sparse/diag.py", line 47, in set_diag
new_row[mask] = row
RuntimeError: shape mismatch: value tensor of shape [50238473] cannot be broadcast to indexing result of shape [50056641]

Originally posted by @allenhaozhu in #2