SPEAR: Selection-aware Personalized End-to-end Adaptive Rewriting and Retrieval for Community Search

In e-commerce search, query reformulation is essential for bridging the gap between user intent and item retrieval. However, traditional two-stage systems—where rewrite generation and retrieval are optimized separately—suffer from structural misalignment. While end-to-end path-based architectures (like PDN) have been proposed to unify these stages, they often struggle with a phenomenon the authors call the generic-word dominance effect. In this scenario, models learn to favor frequent, broad rewrites (e.g., "phone case") that score well on engagement metrics but fail to capture the specific intent of the user's query.

The authors propose SPEAR (Selection-aware Personalized End-to-end Adaptive Rewriting and Retrieval). The framework introduces three key structural innovations to address the limitations of existing path-based models:

1. Dual-Embedding Backbone with Gradient Isolation: The model uses a shared encoder for queries and rewrites but branches into separate Recall and Rank heads. By applying stop-gradient control from the CTR-driven ranking branch to the recall branch, the system protects semantic representations from being distorted by short-term engagement signals.
2. Multiplicative Gating Aggregator: Unlike traditional additive path scoring, which allows high-confidence generic rewrites to inflate scores regardless of relevance, SPEAR uses a multiplicative gating mechanism. A rewrite only contributes to the final score if both its selection confidence and its item relevance are high.
3. Dynamic Rewrite Selector: This component generates request-specific weights and calibration terms (scale and bias) conditioned on the user context and the original query, allowing the system to adapt its rewrite strategy to the specific search context.

SPEAR was evaluated on 100,000 industrial search sessions from the Dewu community search platform. Offline results showed an 18.2% improvement in rewrite semantic similarity and a 99.5% increase in click recall compared to the production baseline. Online A/B testing confirmed these gains, demonstrating a 0.259% increase in query-view CTR and a 0.733% increase in average reading depth. The system has been fully deployed in production since 2025.