LAST: The Last Query Token Guides Visual Token Pruning for Edge–Cloud Collaborative MLLM Inference

As multimodal large language models (MLLMs) move toward edge-cloud collaborative architectures, the transmission of dense visual-token sequences creates significant latency and bandwidth bottlenecks. The authors investigate how to perform query-aware visual token pruning at the edge—before the data reaches the cloud—without requiring access to the cloud model's internal states, costly autoregressive generation, or complex attention aggregation.

LAST (Last Query Token Guidance) leverages the causal attention mechanism inherent in transformer-based models. In a standard multimodal prefill pass, the final query token attends to the entire preceding visual sequence and query context. The authors propose using this specific attention signal as a lightweight, query-conditioned importance score.

Instead of simply selecting the top-scoring tokens, which might lead to redundancy, LAST employs an importance-weighted greedy k-center selection algorithm. This ensures that the retained subset of tokens is both highly relevant to the user's query and visually diverse. Because the edge and cloud models share the same vision encoder, the selected tokens can be transmitted directly to the cloud model without additional alignment or retraining.

Evaluated across 11 multimodal benchmarks, LAST demonstrates a superior accuracy-efficiency trade-off compared to existing vision-driven and query-dependent pruning methods. Notably, it preserves 95.4% of the full-token accuracy while discarding 87.5% of the visual tokens. The method introduces minimal edge-side overhead, as it requires only a single, decoding-free prefill pass, making it highly suitable for resource-constrained edge devices.

This work provides a practical, training-free solution for scaling MLLMs in edge-cloud environments. By shifting the burden of token selection to the edge using a lightweight proxy, LAST significantly reduces cloud-side computation and communication costs while maintaining high performance across diverse tasks, including document understanding and general visual question answering.