Comparative Analysis of GAT and BERT for Human-Like Playtesting

Automated playtesting is essential for balancing puzzle games, yet traditional methods like Convolutional Neural Networks (CNNs) often struggle to generalize when game mechanics change or when board layouts involve non-adjacent connections. This paper investigates whether more flexible architectures—specifically BERT (a transformer-based model) and GAT (a Graph Attention Network)—can better model player behavior in Candy Crush Saga (CCS). The authors compare these models against a CNN baseline, evaluating their ability to predict human-like moves and estimate level difficulty (measured as Attempts Per Success, or APS).

The researchers represent the game board in two ways: as a grid (for the CNN and board-based BERT) and as a graph (for the GAT). The BERT model is tested in both a text-based format, which treats the board as a sequence, and a board-based format. The GAT model treats each tile as a node and defines edges based on adjacency and special game mechanics like portals. By using attention mechanisms, the GAT can weigh the importance of different board elements dynamically, allowing it to account for long-range dependencies that CNNs typically miss. All models were trained on historical player data to predict the next move and simulate full level playthroughs.

The study demonstrates that both BERT and GAT architectures provide superior performance over CNNs, particularly on challenging board configurations. The GAT, in particular, excels by explicitly modeling the relational topology of the board, which is critical for games where distant tiles influence strategic decisions. By moving away from rigid grid-based representations, these models reduce the need for extensive feature engineering, making them more adaptable to the introduction of new game mechanics or varying board sizes. This research highlights a shift toward more generalizable, data-driven playtesting tools that can better mimic the diversity of human strategy.