Missing Related Paper

[wutaiqiang]

Missing Related Paper

[LirongWu]

Thank you very much for sharing the similarities and differences between the two papers, and we are happy to cite and discuss PGKDin the updated arxiv version.

[LirongWu]

We thank you for your interest in our work.

FF-G2M was submitted to AAAI2023 in September 2022 and was accepted in December 2023. We will consider citing your work in the next arxiv release. BTW, it would be very kind if you would like to cite and compare FF-G2M in your work.

I notice that we have all deeply explored the problem of GNN-to-MLP KD, and I would welcome a discussion if possible.

[wutaiqiang]

Thanks for the reply. FF-G2M tries to distill the frequency information from GNNs to MLPs, while PGKD (the paper mentioned before) tries to distill the graph impacts of graph edges on node representations from GNNs to MLPs.

Both two papers try to convey the graph information, but the FF-G2M focuses on frequency information and PGKD focuses on the spatial information. Indeed, the low-frequency information is similar(or say, related) to the intra-class edges and high-frequency information is similar(or say, related) to the inter-class edges.

As the papers are related and close in time. It would be very valuable to discuss each other in both papers.

[wutaiqiang]

More details for these two methods：

FF-G2M distills low-frequency information and high-frequency information, while PGKD distills intra-class edge information and inter-class edge information.

Low-frequency information means the information remained after the message-passing in GNNs, which is related to the intra-class edges which capture the homophily property for nodes from the same class. The difference is that FF-G2M considers all the nodes while PGKD introduces the prototype to achieve such local smoothness. The prototypes can help improve the robustness when there is noise in the representations from GNN teachers.

High-frequency information represents the differences between one node feature and its neighborhood features in the spatial domain. Similarly to that, PKGD concludes that inter-class edges determine the pattern of class distances. FF-G2M compares the distance patterns among connected nodes while PGKD considers the class prototypes and is more lightweight.

Anyway, FF-G2M is a nice paper, and wishing both two papers can benefit the community.

[wutaiqiang]

We have cited the FF-G2M in the PGKD paper and will update in arxiv soon.