TOPOFORMER: TOPOLOGY MEETS ATTENTION FOR GRAPH LEARNING

How can the structural insights provided by Topological Data Analysis (TDA) be integrated into modern deep learning architectures without the computational bottlenecks and vectorization challenges associated with traditional Persistent Homology (PH)?

The authors introduce Topoformer, a framework that bridges TDA and Transformer-based learning. The core innovation is the Topo-Scan module, which decomposes a graph into a sequence of topological slices. Unlike standard PH, which computes global persistence diagrams that often suffer from early saturation, Topo-Scan uses a sliding window approach over node or edge filtrations. This generates a sequence of Betti numbers (representing connected components and cycles) and size statistics (node and edge counts) for each slice. These ordered sequences are then fed into a Transformer encoder, allowing the model to attend to multi-scale structural patterns while remaining computationally efficient and parallelizable.

Topoformer demonstrates state-of-the-art performance across various graph classification and molecular property prediction benchmarks. By bypassing the need for complex, non-differentiable persistence diagram computations, the model achieves a more stable and predictable training process. The authors provide theoretical guarantees on the stability of their topological encodings, showing that small perturbations in the input filtration function result in bounded changes in the output sequences. This makes the architecture both robust to noise and highly scalable for large-scale graph learning tasks.

This work provides a practical path for incorporating topological inductive biases into graph foundation models. By transforming topological information into a format that standard Transformers can consume, Topoformer allows researchers to leverage the structural power of TDA without the typical overhead of specialized topological software. This unification of topological theory and attention-based deep learning offers a promising direction for improving generalization in non-Euclidean data domains.