DIARC: Distinguishing Positive and Negative Samples Helps Improving ARC-like Reasoning Ability of Large Language Models

Large language models (LLMs) often struggle with the Abstraction and Reasoning Corpus (ARC), a benchmark requiring the induction of latent transformation rules from limited grid-based examples. While existing methods rely on supervised fine-tuning (SFT) with correct input-output pairs, this paper investigates whether LLMs can improve their reasoning by learning to distinguish between correct solutions and plausible, rule-incorrect alternatives.

The authors propose DIARC, a framework that applies preference alignment to ARC tasks. Instead of only training on correct target grids, the model is trained on preference pairs consisting of a correct output and a rejected output. The researchers developed three strategies to construct these negative samples:

• Output-level visual transformations: Applying operations like grid-block pooling, rigid shifts, and morphological changes to create visually plausible but rule-incorrect grids.
• DSL-level rule inversion: Identifying recurring patterns in task-specific programs and applying semantic reversals to generate rule-incorrect outputs.
• Task-specific rule editing: Using a stronger LLM to generate alternative rules that are semantically close to the original transformation but lead to different outcomes.

These preference pairs are used to fine-tune models using Direct Preference Optimization (DPO), which encourages the model to assign higher likelihood to the correct output compared to the generated near-miss alternatives.

Experimental results across six ARC-like benchmarks (including ARC-AGI-1, ARC-AGI-2, and ConceptARC) demonstrate that DIARC consistently improves performance over standard SFT baselines. The method proves effective across different model families, such as Llama-3.2, Minitron, and Qwen3. Notably, the Qwen3-4B model equipped with DIARC achieved over 96% accuracy on several benchmarks, outperforming various specialized models and larger closed-source systems. The study highlights that learning from informative errors is a powerful, underutilized strategy for enhancing abstract reasoning in symbolic tasks.