daydreamer2023/pytorch_geometric

Geometric Deep Learning Extension Library for PyTorch

---

Documentation | Paper

PyTorch Geometric (PyG) is a geometric deep learning extension library for PyTorch.

It consists of various methods for deep learning on graphs and other irregular structures, also known as geometric deep learning, from a variety of published papers. In addition, it consists of an easy-to-use mini-batch loader, multi gpu-support, a large number of common benchmark datasets (based on simple interfaces to create your own), and helpful transforms, both for learning on arbitrary graphs as well as on 3D meshes or point clouds.

---

PyTorch Geometric makes implementing Graph Neural Networks a breeze (see here for the accompanying tutorial). For example, this is all it takes to implement the edge convolutional layer:

import torch
from torch.nn import Sequential as Seq, Linear as Lin, ReLU
from torch_geometric.nn import MessagePassing

class EdgeConv(MessagePassing):
    def __init__(self, F_in, F_out):
        super(EdgeConv, self).__init__(aggr='max')  # "Max" aggregation.
        self.mlp = Seq(Lin(2 * F_in, F_out), ReLU(), Lin(F_out, F_out))

    def forward(self, x, edge_index):
        # x has shape [N, F_in]
        # edge_index has shape [2, E]
        return self.propagate(edge_index, x=x)  # shape [N, F_out]

    def message(self, x_i, x_j):
        # x_i has shape [E, F_in]
        # x_j has shape [E, F_in]
        edge_features = torch.cat([x_i, x_j - x_i], dim=1)  # shape [E, 2 * F_in]
        return self.mlp(edge_features)  # shape [E, F_out]

In addition, PyTorch Geometric is fast, even compared to other deep graph neural net libraries:

Dataset	Epochs	Model	DGL-DB	DGL-GS	PyG
Cora	200	GCN	4.19s	0.32s	0.25s
		GAT	6.31s	5.36s	0.80s
CiteSeer	200	GCN	3.78s	0.34s	0.30s
		GAT	5.61s	4.91s	0.88s
PubMed	200	GCN	12.91s	0.36s	0.32s
		GAT	18.69s	13.76s	2.42s
MUTAG	200	RGCN	18.81s	2.40s	2.14s
DGL-DB: DGL 0.2 with Degree Bucketing.
DGL-GS: DGL 0.2 with gather and scatter optimizations inspired by PyG.
Training runtimes obtained on a NVIDIA GTX 1080Ti.

---

In detail, the following methods are currently implemented:

• SplineConv from Fey et al.: SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels (CVPR 2018)
• GCNConv from Kipf and Welling: Semi-Supervised Classification with Graph Convolutional Networks (ICLR 2017)
• ChebConv from Defferrard et al.: Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering (NIPS 2016)
• NNConv adapted from Gilmer et al.: Neural Message Passing for Quantum Chemistry (ICML 2017)
• GATConv from Veličković et al.: Graph Attention Networks (ICLR 2018)
• SAGEConv from Hamilton et al.: Inductive Representation Learning on Large Graphs (NIPS 2017)
• GraphConv from, e.g., Morris et al.: Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks (AAAI 2019)
• GatedGraphConv from Li et al.: Gated Graph Sequence Neural Networks (ICLR 2016)
• GINConv from Xu et al.: How Powerful are Graph Neural Networks? (ICLR 2019)
• ARMAConv from Bianchi et al.: Graph Neural Networks with Convolutional ARMA Filters (CoRR 2019)
• SGConv from Wu et al.: Simplifying Graph Convolutional Networks (CoRR 2019)
• APPNP from Klicpera et al.: Predict then Propagate: Graph Neural Networks meet Personalized PageRank (ICLR 2019)
• AGNNConv from Thekumparampil et al.: Attention-based Graph Neural Network for Semi-Supervised Learning (CoRR 2017)
• RGCNConv from Schlichtkrull et al.: Modeling Relational Data with Graph Convolutional Networks (ESWC 2018)
• SignedConv from Derr et al.: Signed Graph Convolutional Network (ICDM 2018)
• DNAConv from Fey: Just Jump: Dynamic Neighborhood Aggregation in Graph Neural Networks (ICLR-W 2019)
• EdgeConv from Wang et al.: Dynamic Graph CNN for Learning on Point Clouds (CoRR, 2018)
• PointConv (including Iterative Farthest Point Sampling, dynamic graph generation based on nearest neighbor or maximum distance, and k-NN interpolation) from Qi et al.: PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation (CVPR 2017) and PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space (NIPS 2017)
• XConv from Li et al.: PointCNN: Convolution On X-Transformed Points (official implementation) (NeurIPS 2018)
• PPFConv from Deng et al.: PPFNet: Global Context Aware Local Features for Robust 3D Point Matching (CVPR 2018)
• GMMConv from Monti et al.: Geometric Deep Learning on Graphs and Manifolds using Mixture Model CNNs (CVPR 2017)
• A MetaLayer for building any kind of graph network similar to the TensorFlow Graph Nets library from Battaglia et al.: Relational Inductive Biases, Deep Learning, and Graph Networks (CoRR 2018)
• GlobalAttention from Li et al.: Gated Graph Sequence Neural Networks (ICLR 2016)
• Set2Set from Vinyals et al.: Order Matters: Sequence to Sequence for Sets (ICLR 2016)
• Sort Pool from Zhang et al.: An End-to-End Deep Learning Architecture for Graph Classification (AAAI 2018)
• Dense Differentiable Pooling from Ying et al.: Hierarchical Graph Representation Learning with Differentiable Pooling (NeurIPS 2018)
• Graclus Pooling from Dhillon et al.: Weighted Graph Cuts without Eigenvectors: A Multilevel Approach (PAMI 2007)
• Voxel Grid Pooling from, e.g., Simonovsky and Komodakis: Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs (CVPR 2017)
• Top-K Pooling from Gao and Ji: Graph U-Net (ICLR 2019 submission) and Cangea et al.: Towards Sparse Hierarchical Graph Classifiers (NeurIPS-W 2018)
• Local Degree Profile from Cai and Wang: A Simple yet Effective Baseline for Non-attribute Graph Classification (CoRR 2018)
• Jumping Knowledge from Xu et al.: Representation Learning on Graphs with Jumping Knowledge Networks (ICML 2018)
• Deep Graph Infomax from Veličković et al.: Deep Graph Infomax (ICLR 2019)
• All variants of Graph Auto-Encoders from Kipf and Welling: Variational Graph Auto-Encoders (NIPS-W 2016) and Pan et al.: Adversarially Regularized Graph Autoencoder for Graph Embedding (IJCAI 2018)
• RENet from Jin et al.: Recurrent Event Network for Reasoning over Temporal Knowledge Graphs (ICLR-W 2019)

---

Head over to our documentation to find out more about installation, data handling, creation of datasets and a full list of implemented methods, transforms, and datasets. For a quick start, check out our examples in the examples/ directory.

If you notice anything unexpected, please open an issue and let us know. If you are missing a specific method, feel free to open a feature request. We are constantly encouraged to make PyTorch Geometric even better.

Installation

Ensure that at least PyTorch 1.0.0 is installed and verify that cuda/bin, cuda/include and cuda/lib64 are in your $PATH, $CPATH and $LD_LIBRARY_PATH respectively, e.g.:

$ python -c "import torch; print(torch.__version__)"
>>> 1.0.0

$ echo $PATH
>>> /usr/local/cuda/bin:...

$ echo $CPATH
>>> /usr/local/cuda/include:...

and

$ echo $LD_LIBRARY_PATH
>>> /usr/local/cuda/lib64

on Linux or

$ echo $DYLD_LIBRARY_PATH
>>> /usr/local/cuda/lib

on macOS. Then run:

$ pip install --verbose --no-cache-dir torch-scatter
$ pip install --verbose --no-cache-dir torch-sparse
$ pip install --verbose --no-cache-dir torch-cluster
$ pip install --verbose --no-cache-dir torch-spline-conv (optional)
$ pip install torch-geometric

See the Frequently Asked Questions and verify that your CUDA is set up correctly if you encounter any installation errors.

Running examples

cd examples
python gcn.py

Cite

Please cite our paper (and the respective papers of the methods used) if you use this code in your own work:

@inproceedings{Fey/Lenssen/2019,
  title={Fast Graph Representation Learning with {PyTorch Geometric}},
  author={Fey, Matthias and Lenssen, Jan E.},
  booktitle={ICLR Workshop on Representation Learning on Graphs and Manifolds},
  year={2019},
}

Running tests

python setup.py test