A Fair Evaluation of Graph Foundation Models for Node Property Prediction

Research Question and Approach

This study addresses the lack of a unified evaluation framework for Graph Foundation Models (GFMs) in node property prediction. The authors conduct a rigorous, large-scale reevaluation of nine recent GFMs, comparing them against strong, well-tuned Graph Neural Network (GNN) baselines. The study utilizes ten diverse datasets from the GraphLand benchmark to ensure practical relevance and broad domain coverage, moving beyond the narrow or buggy datasets often found in existing literature.

The researchers categorize the evaluated GFMs into two distinct groups: PFN-based models and non-PFN-based models. The non-PFN-based GFMs consistently underperform compared to properly tuned classic GNNs. In contrast, PFN-based models—which learn to process datasets in a single forward pass—demonstrate superior predictive performance. Specifically, GraphPFN, when fine-tuned, achieves the best results across all ten datasets. However, this performance gain comes at the cost of significantly higher computational overhead during inference compared to the lightweight, efficient inference typical of traditional GNNs.

As the field of Graph Machine Learning shifts toward foundation models, this study provides a necessary reality check. It highlights that many proposed GFMs fail to provide a performance advantage over well-tuned, traditional GNNs. By demonstrating that only PFN-based architectures currently offer a competitive edge, the paper provides a clearer roadmap for future research and emphasizes the importance of rigorous, standardized evaluation protocols in the development of graph-based foundation models.