namkyeong/afgrl

The official source code for "Augmentation-Free Self-Supervised Learning on Graphs"

Augmentation-Free Self-Supervised Learning on Graphs

The official source code for Augmentation-Free Self-Supervised Learning on Graphs paper, accepted at AAAI 2022.

Overview

Inspired by the recent success of self-supervised methods applied on images, self-supervised learning on graph structured data has seen rapid growth especially centered on augmentation-based contrastive methods. However, we argue that without carefully designed augmentation techniques, augmentations on graphs may behave arbitrarily in that the underlying semantics of graphs can drastically change. As a consequence, the performance of existing augmentation-based methods is highly dependent on the choice of augmentation scheme, i.e., hyperparameters associated with augmentations. In this paper, we propose a novel augmentation-free self-supervised learning framework for graphs, named AFGRL. Specifically, we generate an alternative view of a graph by discovering nodes that share the local structural information and the global semantics with the graph. Extensive experiments towards various node-level tasks, i.e., node classification, clustering, and similarity search on various real-world datasets demonstrate the superiority of AFGRL.

Augmentations on images keep the underlying semantics, whereas augmentations on graphs may unexpectedly change the semantics.

Requirements

• Python version: 3.7.10
• Pytorch version: 1.8.1
• torch-geometric version: 1.7.0
• faiss: 1.7.0

Hyperparameters

Following Options can be passed to main.py

--dataset: Name of the dataset. Supported names are: wikics, cs, computers, photo, and physics. Default is wikics.
usage example :--dataset wikics

--task: Name of the task. Supported names are: node, clustering, similarity. Default is node.
usage example :--task node

--layers: The number of units of each layer of the GNN. Default is [256]
usage example :--layers 256

--pred_hid: The number of hidden units of predictor. Default is [512]
usage example :--pred_hid 512

--topk: The number of neighbors for nearest neighborhood search. Default is 4.
usage example :--topk 4

--num_centroids: The number of centroids for K-means Clustering . Default is 100.
usage example :--num_centroids 100

--num_kmeans: The number of iterations for K-means Clustering . Default is 5.
usage example :--num_kmeans 5

How to Run

You can run the model with following options

• To run node classification (reproduce Table 2 in paper)

sh run_node_classification.sh

• To run node clustering (reproduce Table 3 in paper)

sh run_node_clustering.sh

• To run similarity search (reproduce Table 4 in paper)

sh run_similarity_search.sh

• or you can run the file with above mentioned hyperparameters

python main.py --embedder AFGRL --dataset wikics --task node --layers [1024] --pred_hid 2048 --lr 0.001 --topk 8

Cite (Bibtex)

• If you find AFGRL useful in your research, please cite the following paper:
  • Lee, Namkyeong, Junseok Lee, and Chanyoung Park. "Augmentation-Free Self-Supervised Learning on Graphs." AAAI 2022.
  • Bibtex

@article{lee2021augmentation,
  title={Augmentation-Free Self-Supervised Learning on Graphs},
  author={Lee, Namkyeong and Lee, Junseok and Park, Chanyoung},
  booktitle={AAAI},
  year={2022}
}

afgrl's Issues

[Errno 2] No such file or directory: 'data/pyg/WikiCS/processed/byg.data.pt'

It seems like a problem with the dataset "wikics", but i don't know where the problem is.

Here the stack is:

Exception has occurred: FileNotFoundError
[Errno 2] No such file or directory: 'data/pyg/WikiCS/processed/byg.data.pt'
File "/home/wanguancheng/github/AFGRL/data.py", line 64, in init
self.data, self.slices = torch.load(path)
File "/home/wanguancheng/github/AFGRL/models/AFGRL.py", line 45, in _init
self._dataset = Dataset(root=args.root, dataset=args.dataset)
File "/home/wanguancheng/github/AFGRL/models/AFGRL.py", line 33, in init
self._init()
File "/home/wanguancheng/github/AFGRL/main.py", line 22, in main
embedder = AFGRL_ModelTrainer(args)
File "/home/wanguancheng/github/AFGRL/main.py", line 28, in
main()

Question about logistic regression implementation

Thank you for providing your code!

I had a question about your logistic regression implementation. I noticed that the LogisticRegression class consists of a single nn.Linear layer without applying the logistic function.

Although the logistic function is order-preserving (and therefore will not change the end result after training since you take the argmax anyways), it seems not applying the logistic function may affect the stability of the convergence of your method. This would also seem to make your classifier a linear transform instead of logistic regression. Was this an intentional decision in your experimental design (e.g. were there better results without the logistic function?), or was it an oversight?

Thanks!

Inconsistent result of BGRL on Wiki-CS dataset

Thanks for your great work fiirst! But the experiment result of BGRL on Wiki-CS dataset of your paper is much inconsistent with the BGRL paper and your own experiment result in repository BGRL_Pytorch. Did I make any stupid mistakes?

For a self-supervised algorithm, is it right adding labels of data when training?