Mathematical supervision for language models is typically flattened into isolated problem-solution pairs, discarding the curricular organization and evidential structure of the original sources. We introduce TOME-Math (Textbook-Oriented Mathematical Evidence), a large-scale corpus that converts 2,118 textbooks spanning the core areas of university-level mathematics subjects into approximately 380K structured proof-supervision records while preserving chapter order, proposition-proof links, dependency references, and source provenance. Leveraging these structural signals, we explore curriculum learning across multiple mathematical subjects and characterize scaling patterns with supervision volume and source breadth. At full scale, fine-tuning on TOME-Math raises the unweighted mean score across our held-out evaluations and established public benchmarks by 16.7 percentage points over the base model, while exceeding all three public instruction-data baselines evaluated in aggregate. Our findings highlight structured textbooks as both a scalable source of mathematical supervision and an empirical framework for studying the impact of data organization on mathematical reasoning. We will release the corpus to facilitate further research on structured mathematical supervision.
Sam: A mathematics textbook’s order can be useful training data, not just packaging. This study finds that preserving that order, and exposing omitted proof steps, helps a language model learn mathematical reasoning.
Alex: That sounds useful for anyone choosing training material. Is the claim about better textbooks, or better use of what textbooks already contain?
Sam: Better use of what they contain. Youqing Fang, Zhouqi Hua, and colleagues present a dataset and experimental method in an arXiv preprint, “Scaling Mathematical Proof Supervision from Structured Textbooks.” Their full training recipe raises the tested model’s average benchmark score by about seventeen percentage points over its starting checkpoint.
Alex: So the practical question is whether people building mathematical models should preserve more than the answers.
Sam: Yes. We’ll look at what they preserve, which comparisons actually isolate its value, and why the quality checks leave reasons for caution.
Alex: Start with what gets lost. A question and a worked solution already seem like a useful mathematical teaching unit.
Sam: They are useful, but the authors argue they discard relationships. In a textbook, a definition supports a lemma, and that lemma supports a later theorem. Chapters also establish a progression. Flatten everything into unrelated question–answer pairs, and those connections stop being explicit.
Alex: But textbook data and linked proofs existed before this. What distinguishes this resource from those earlier efforts?
Sam: The paper acknowledges that history. NaturalProofs already preserves statements, proofs, and a reference graph linking results. Other resources collect textbook text or exercises. The contribution here is combining broad textbook coverage with chapter progression, proof dependencies, and explicit tasks for reasoning the textbook leaves unstated.
Alex: How broad is the collection, and what does an actual training example look like?
Sam: The collection draws on about twenty-one hundred textbooks. It produces roughly three hundred eighty thousand question–answer records. The authors call it TOME-Math. One stream pairs mathematical propositions with proofs; another turns omitted intermediate reasoning into standalone proof questions.
Alex: That second stream seems central. Can you make “omitted reasoning” concrete without taking us through a whole theorem?
Sam: One example comes from Diestel’s Graph Theory. The source proves that a graph has an even number of odd-degree vertices. Degree means the number of edges meeting a vertex. Adding all the degrees counts every edge twice, so the total is even.
Alex: But the reader still has to connect an even total with an even count of odd contributions.
Sam: The constructed answer spells that out. Even-degree vertices contribute nothing to the parity of the sum. Each odd-degree vertex flips it. For the total to be even, the number of those flips must be even. That illustrates the kind of intermediate reasoning the pipeline makes explicit.
Alex: How do they keep that process tied to the textbook, rather than asking a model to invent plausible mathematics?
Sam: They recover the table of contents and chapter boundaries from scanned textbook files. Then they separate exposition from native exercises. From the exposition, models extract propositions and proofs, resolve references to earlier results, and inspect proofs for missing steps. Candidate obligations are rewritten to declare the symbols and definitions they need.
Alex: That involves several opportunities for mistakes. What happens before a candidate reaches the dataset?
Sam: Automated checks assess completeness, accuracy, mathematical formatting, self-containment, and whether an item forms a coherent unit. Unsupported proof steps count as failures, not stylistic defects. But the later quality audit raises concerns about how well that source-grounding standard is achieved. We should keep that distinction in mind.
Alex: Let’s separate the headline result from the explanation for it. What did the main comparison establish?
Sam: All training runs start from the same Qwen base checkpoint. The authors compare TOME-Math with three public mathematical instruction datasets. Each gets a full training pass under shared optimization settings. TOME-Math has the highest unweighted average across six evaluations, leading on everything except the AIME competition set.
Alex: Those datasets differ in content and length, though. Winning that comparison would not establish that chapter order caused the improvement.
Sam: The authors explicitly say so. These runs have different total training-token counts, despite sharing the training setup. The comparison establishes the usefulness of the whole package. To isolate organization, they run more controlled experiments.
Alex: Before those, how do they test whether the model can handle unfamiliar textbook problems?
Sam: Their long-form evaluation contains three hundred problems drawn from books excluded from training. It covers several mathematical subjects and is proof-heavy, rather than exclusively proofs. Native exercises are kept separate from the proof-extraction stream, and all training records from the reserved books are removed.
Alex: Now suppose I already have a model and cannot retrain it. Does textbook structure still help?
Sam: In their retrieval experiment, yes. Retrieval means supplying selected source material alongside the question. The frozen model’s single-answer success estimate rises from about thirteen percent without context to about seventeen percent with structured propositions and explicit proof-gap tasks. All context-supplying conditions get the same token allowance.
Alex: So the useful comparison is not just context versus no context. It is carefully selected evidence versus random or nearby material from the same book.
Sam: And the structured policy does better than those alternatives. They remove the target exercise and material overlapping its reference answer from the retrieval pool. Random same-book context and chapter-local context sometimes fail to help particular subjects. Access alone is not the same as useful evidence selection.
Alex: What happens when they change the learning order instead of the evidence supplied at test time?
Sam: That is curriculum learning: deliberately ordering training examples. Within each subject and training budget, the schedules use identical examples. They compare shuffled order, backward textbook progression, and forward progression. They also test whether presenting local gap tasks before the remaining proof examples helps.
Alex: Which part of that recipe carries the result?
Sam: Forward progression with gap tasks first wins in every subject-and-budget combination they test. In these subject-specific experiments, chapter direction contributes most, with gap-first ordering adding a smaller gain. In the full-corpus ablation, gap-first is the stronger ordering signal. Removing gap tasks also hurts every benchmark.
Alex: That makes the organizing principle more credible, but not identical at every scale. Does the number of source books matter independently of training volume?
Sam: They test that too. Spreading a fixed token budget across more books improves performance in every tested subject. That supports source breadth as another useful dimension, beyond simply adding examples. But all these model experiments use the same backbone, so generalization across model families remains untested here.
Alex: We also need to trust the judgments of a successful proof. Are these machine-checked proofs?
Sam: No. Even the MiniF2F and ProofNet evaluations assess generated natural-language proofs using a language-model judge. For the textbook evaluation, repeated reviews must all find no substantive error. Human calibration still finds false acceptance, and estimates that roughly one in five rejected responses should have been accepted.
Alex: And you flagged a separate concern about the training records themselves.
Sam: The corpus audit reports that source groundedness and evidence alignment each pass for fewer than half the records. That does not mean every remaining answer is mathematically wrong. It does mean the pipeline’s source-faithfulness goals should not be mistaken for guaranteed source-faithfulness in the released records.
Alex: Given those reservations, who should read the full document, and where should they begin?
Sam: Researchers building mathematical training corpora, proof-generating models, or evidence-retrieval systems should read “Leveraging Textbook Structure.” Start with the matched-example curriculum comparisons. Then read the corpus quality audit and human calibration of the verifier. For everyone else, remember this: mathematical training data includes the connections between answers, not just the answers themselves.
Alex: Keep the connections, and check the evidence.