Bringing Back Rule Induction to Fluid Intelligence Research? An Initial Validation of the ARC-AGI Benchmark in Humans

This study investigates whether the Abstraction and Reasoning Corpus for Artificial General Intelligence (ARC-AGI)—a benchmark originally designed to test AI reasoning—can serve as a valid measure of human fluid intelligence (gf). While fluid intelligence is often defined by the ability to induce novel relations, most traditional psychometric tests rely on a limited set of recurring rules. The authors evaluate whether ARC-AGI, which emphasizes novel rule induction, effectively captures the construct of fluid intelligence in humans.

The researchers conducted a laboratory study with 100 participants, who completed 20 selected ARC-AGI items alongside established tests for figural fluid intelligence and figural originality. The goal was to establish a measurement model for ARC-AGI and determine its place within the nomological network of human cognitive abilities.

The study provides initial evidence that ARC-AGI is a robust measure of human fluid intelligence. The results show that ARC-AGI performance correlates strongly (rho = .63) with established figural reasoning tests, supporting its convergent validity. Conversely, the association between ARC-AGI and figural originality was weak, suggesting that the benchmark primarily taps into reasoning capacity rather than creative divergent thinking. These findings align with the perspective that fluid intelligence tasks are heavily constrained by cognitive processing capacity rather than creative rule generation.

This research bridges the gap between AI evaluation and human psychometrics. By validating an AI benchmark in human subjects, the authors demonstrate that innovative AI task designs can enrich the measurement of human cognitive abilities. Furthermore, this study suggests that for AI benchmarks to be meaningful, they should be systematically embedded into the established nomological networks of human intelligence. This approach provides a rigorous framework for evaluating whether AI systems are truly demonstrating general intelligence or merely memorizing patterns.