CREDENCE: Claim Reduction for Decomposition & Enhanced Credibility Semantic Metrics and Convergence Analysis

Research Question and Approach

Automated fact-checking often fails when processing compound sentences containing multiple assertions. While decomposing these into atomic claims improves verification, existing evaluation metrics like Jaccard-F1 rely on token overlap, which unfairly penalizes models that paraphrase or restructure claims. Furthermore, current repair loops—used to fix decomposition errors—often lack formal guarantees, potentially leading to infinite loops or degraded output quality. The authors present Credence, a framework that replaces token-based metrics with a semantic fidelity metric (Semantic-F1) and establishes formal convergence properties for the repair pipeline.

Key Contributions

The authors introduce Semantic-F1, which uses BGE-large cosine similarity to evaluate claim decomposition. This metric better aligns with human judgment by rewarding semantic equivalence rather than exact word matching. Additionally, the authors provide formal convergence theorems: they prove that rule-based repair is monotone and finitely terminating under an oracle parser, whereas LLM-based self-repair is non-monotone and requires an early-exit guard to prevent quality degradation. The framework also includes three new benchmarks (SocialClaimSplit, WikiSplitBench, and ClaimDecompBench) to measure cross-domain generalization.

By shifting from token-overlap to semantic-based evaluation, Credence provides a more accurate and robust way to measure the performance of claim decomposition models. The formal convergence analysis offers a necessary safeguard for developers, ensuring that automated repair loops do not introduce new errors or fail to terminate. This is particularly important for privacy-sensitive or high-stakes fact-checking applications where reliability and interpretability are paramount.