To reproduce the results on Cora, PubMed, and Citeseer

[bwdeng20]

To reproduce the results on Cora, PubMed, and Citeseer

[LirongWu]

Thank you for being interested in our work. Please note that (1) We use DGL to implement all the GNN models on three citation datasets (Cora, Citeseer, and Pubmed). In order to evaluate the model with different splitting strategy (fewer and harder label rates), you need to replace the file (dgl/data/citation_graph.py) with the citation_graph.py provided. (2) The results of four data splitting settings for the Cora, Citeseer and Pubmed datasets are reported in the paper, depending on the parameter "percent", and only the hyperparameters for one of the data splitting settings are provided in the param folder; the hyperparameters for the remaining settings can be searched in a similar way using the hyperparameter search tool NNI. (3) Since different hardware platforms vary greatly, fixed hyperparameters may not give exactly the same results in different hardware, so we suggest you to use NNI to search hyperparameters according to the search space described in the paper. If you still find it difficult after trying, please feel free to contact us and we can provide you with scripts for hyperparameter search and hyperparameters for other data splitting settings, but this may take a little time as we need to organize this work that is almost two years old.

[bwdeng20]

Thanks for the swift reply.

For fast verification, I have only tested three citation datasets with one split setting mentioned in Table 6. Could you please tell me which splits the default hyperparameter files under the param directory are for?

[LirongWu]

You can refer to the parameter "percent" in the param file, which tells us the percentage of labeled data used for training. For example, when percent=0.005 (default hyperparameter for Cora), it means that there are 0.5% of labeled nodes used for training, which corresponds to the result of TrainPCT=0.5% in Table 6.