mengliu1998/deepergnn

Official PyTorch implementation of "Towards Deeper Graph Neural Networks" [KDD2020]

Towards Deeper Graph Neural Networks

This repository is an official PyTorch implementation of DAGNN in "Towards Deeper Graph Neural Networks" (KDD2020). Our implementation is mainly based on PyTorch Geometric, a geometric deep learning extension library for PyTorch.

For more insights, (empirical and theoretical) analysis, and discussions about deeper graph neural networks, please refer to our paper.

Meng Liu, Hongyang Gao, and Shuiwang Ji. Towards Deeper Graph Neural Networks.

Other unofficial implementations:

• An implementation in DGL [PyTorch]
• An implementation in GraphGallery [PyTorch]

Reference

@inproceedings{liu2020towards,
  title={Towards Deeper Graph Neural Networks},
  author={Liu, Meng and Gao, Hongyang and Ji, Shuiwang},
  booktitle={Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining},
  year={2020},
  organization={ACM}
}

Requirements

• PyTorch
• PyTorch Geometric >= 1.3.1
• NetworkX
• tqdm

Note that the versions of PyTorch and PyTorch Geometric should be compatible and PyTorch Geometric is related to other packages, which need to be installed in advance. It would be easy by following the installation instruction.

PyTorch Geometric 1.3.1 was used in this code. If you have a newer version installed already, you may encounter an error about "GCNConv.norm" when running this code. Refer to this issue for a possible solution. (2020.8.12 update: This issue has been solved in the current code. Now, our code works for PyTorch Geometric >= 1.3.1.)

Run

• To evaluate DAGNN on OGB dataset ogbn-arxiv, run

CUDA_VISIBLE_DEVICES=0 python main_ogbnarxiv.py

Dataset	#Para	Validation	Test
ogbn-arxiv	43857	72.90±0.11	72.09±0.25

• To reproduce our results in Table 2 and 3, run

bash run.sh

deepergnn's Issues

datasets: urllib.error.URLError: <urlopen error [Errno 11004] getaddrinfo failed>

**Hi, Thx for your nice work!

I got this error (Actually, all datasets got the same error):**

$ bash run.sh
=====Cora=====
---Fiexd Splits---
Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.x
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1318, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1026, in _send_output
self.send(msg)
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 964, in send
self.connect()
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1392, in connect
super().connect()
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 936, in connect
(self.host,self.port), self.timeout, self.source_address)
File "C:\ProgramData\Anaconda3\lib\socket.py", line 704, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
File "C:\ProgramData\Anaconda3\lib\socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11004] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "dagnn.py", line 84, in
dataset = get_planetoid_dataset(args.dataset, args.normalize_features)
File "F:\PapersCode\DeeperGNN\DeeperGNN\datasets.py", line 9, in get_planetoid_dataset
dataset = Planetoid(path, name)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\datasets\planetoid.py", line 55, in init
super(Planetoid, self).init(root, transform, pre_transform)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\data\in_memory_dataset.py", line 54, in init
pre_filter)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\data\dataset.py", line 89, in init
self._download()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\data\dataset.py", line 141, in _download
self.download()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\datasets\planetoid.py", line 105, in download
download_url('{}/{}'.format(self.url, name), self.raw_dir)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\data\download.py", line 31, in download_url
data = urllib.request.urlopen(url)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 532, in open
response = meth(req, response)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 564, in error
result = self._call_chain(*args)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 756, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 526, in open
response = self._open(req, data)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 544, in _open
'_open', req)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1361, in https_open
context=self._context, check_hostname=self._check_hostname)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1320, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 11004] getaddrinfo failed>

Then I changed the DNS. However, I got another error:

$ bash run.sh
=====Cora=====
---Fiexd Splits---
Downloading https://github.com/kimiyoung/planetoid/raw/master/data/ind.cora.x
Traceback (most recent call last):
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1318, in do_open
encode_chunked=req.has_header('Transfer-encoding'))
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1239, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1285, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1234, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1026, in _send_output
self.send(msg)
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 964, in send
self.connect()
File "C:\ProgramData\Anaconda3\lib\http\client.py", line 1400, in connect
server_hostname=server_hostname)
File "C:\ProgramData\Anaconda3\lib\ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "C:\ProgramData\Anaconda3\lib\ssl.py", line 814, in init
self.do_handshake()
File "C:\ProgramData\Anaconda3\lib\ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "C:\ProgramData\Anaconda3\lib\ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ConnectionResetError: [WinError 10054] Զ▒▒▒▒▒▒ǿ▒ȹر▒▒▒һ▒▒▒▒▒е▒▒▒▒ӡ▒

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "dagnn.py", line 84, in
dataset = get_planetoid_dataset(args.dataset, args.normalize_features)
File "F:\PapersCode\DeeperGNN\DeeperGNN\datasets.py", line 9, in get_planetoid_dataset
dataset = Planetoid(path, name)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\datasets\planetoid.py", line 31, in init
super(Planetoid, self).init(root, transform, pre_transform)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\data\in_memory_dataset.py", line 53, in init
pre_filter)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\data\dataset.py", line 82, in init
self._download()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\data\dataset.py", line 118, in _download
self.download()
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\datasets\planetoid.py", line 45, in download
download_url('{}/{}'.format(self.url, name), self.raw_dir)
File "C:\ProgramData\Anaconda3\lib\site-packages\torch_geometric\data\download.py", line 31, in download_url
data = urllib.request.urlopen(url)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 223, in urlopen
return opener.open(url, data, timeout)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 532, in open
response = meth(req, response)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 642, in http_response
'http', request, response, code, msg, hdrs)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 564, in error
result = self._call_chain(*args)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 756, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 526, in open
response = self._open(req, data)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 544, in _open
'_open', req)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 504, in _call_chain
result = func(*args)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1361, in https_open
context=self._context, check_hostname=self._check_hostname)
File "C:\ProgramData\Anaconda3\lib\urllib\request.py", line 1320, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [WinError 10054] Զ▒▒▒▒▒▒ǿ▒ȹر▒▒▒һ▒▒▒▒▒е▒▒▒▒ӡ▒>

I have found different solutions on the Internet, but nothing worked for me.
I would appreciate it if u could help me.
Thank you!

Can we get the paper here in advance?

Thanks for sharing your awesome work, but I can't find the paper here.
would you like to release pdf version of the paper? or you can share it with me. My email is [email protected].

Thanks anyway

Performance on Open Graph Benchmark

Hi Meng,

Open Graph Benchmark(OGB) is gaining its popularity as benchmarks for GNNs, compared to citation networks we normally use, they are more scalable and systematic. I'd like to see DAGNN's results on OGB, and I did some experiments myself on ogbn-arxiv by slightly modifying your code (making it work on SparseTensor). But I failed to reproduce a better accuracy compared to vanilla GCN. (GCN: 71.74%, while DAGNN only ~67%).

The Colab link is here. I tested K=5, 10, 20. Note that for consistency, I don't apply weight decay here since the GCN implementation doesn't either, and I don't think that would count for the 4% difference. Would you mind helping figure out what's wrong with my code or why this would happen? Thank you very much.

How to draw the Figure 3 of DeeperGNN paper ?

How to draw the Figure 3 of DeeperGNN paper ?

Performance on adaptive adjustment mechanism

Hi,
I want to test the performance on adaptive adjustment mechanism. If the code of adaptive adjustment mechanism is

pps = torch.stack(preds, dim=1)   # nxkxc
retain_score = self.proj(pps)     # nxkx1
retain_score = retain_score.squeeze()  # nxk
retain_score = torch.sigmoid(retain_score)
retain_score = retain_score.unsqueeze(1) # nx1xk
out = torch.matmul(retain_score, pps).squeeze() # nx1xc -> nxc

After i comment out these lines and use preds[-1] as out, the performance of test accuracy declines to 0.397. It is even lower than MLP. So, Are these the code of adaptive adjustment mechanism?

There is a problem when we run the dataset Cora

0%| | 0/100 [00:00<?, ?run/s]Traceback (most recent call last):
File "E:/machine learning/spectral and algebraic theory/20200814Myself/DeeperGNN-master/DeeperGNN/dagnn.py", line 113, in
run(dataset, Net(dataset), args.runs, args.epochs, args.lr, args.weight_decay, args.early_stopping, permute_masks, lcc=False)
File "E:\machine learning\spectral and algebraic theory\20200814Myself\DeeperGNN-master\DeeperGNN\train_eval.py", line 121, in run
eval_info = evaluate(model, data)
File "E:\machine learning\spectral and algebraic theory\20200814Myself\DeeperGNN-master\DeeperGNN\train_eval.py", line 175, in evaluate
pred = logits[mask].max(1)[1]
RuntimeError: cannot perform reduction function max on tensor with no elements because the operation does not have an identity

PyG version

HI, could you please provide the version of PyG? I met an error "AttributeError: type object 'GCNConv' has no attribute 'norm'" when I ran the code.