On-Device Deep Research at 4B: Exposure Bounds

This study investigates how to optimize small-scale (4B parameter) research agents for personal laptops. The author separates citation quality into two distinct metrics: cited claim faithfulness (whether the cited source supports the claim) and trustworthy coverage (whether the agent cites the correct sources). By fixing the model and varying two key levers—exposure (the number of characters provided per source) and source quality (gold vs. retrieved papers)—the study identifies which factors drive performance and at what cost.

The research reveals a clear separation of concerns. Faithfulness is primarily a function of exposure. Increasing the character limit per source from 400 to 1500 significantly boosts the model's ability to support its claims, with faithfulness rising from 0.45 to 0.58 on retrieved sources. Notably, at high exposure, the performance gap between using gold sources and retrieved sources disappears, suggesting that the model's grounding capability is limited by the amount of text it reads rather than the inherent quality of the retrieved documents.

Conversely, trustworthy coverage is bound by the retrieval stage. Because the agent's citation recall remains stagnant at approximately 0.40 regardless of how much source text is provided, increasing exposure cannot compensate for poor retrieval. The agent effectively uses the sources it is given, but it cannot fix the fact that it failed to retrieve the correct evidence in the first place.

For developers building on-device research tools, this paper provides a practical, cost-effective recipe: prioritize increasing the per-source character budget before attempting complex architectural changes. Since the extra exposure costs only about 235 output tokens, it is a computationally cheap way to maximize grounding. Once exposure is optimized, the remaining performance ceiling is determined solely by the retrieval system's recall. This finding challenges the assumption that small models are inherently incapable of using context effectively, showing instead that they simply require sufficient exposure to perform reliably.

AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.