Zhenyi Shen, Junru Lu, Wei Liu, Jiangnan Ye, Jiazheng Li, Di Yin, Xing Sun, Tameem Adel, Yali Du, Lin Gui, Yulan He
8 min
Abstract
Prefill-to-Autoregressive Cross-model Transfer (PACT), a form of latent communication between language models, combines a larger model's prompt representations with a smaller model's efficient generation, offering a promising way to balance capability and inference cost. However, existing systems provide limited support for independently evaluating large-small model collaboration, and improved task performance alone does not establish whether the receiver benefits from the sender's information. We introduce PLatour, an open-source PACT framework that connects frozen models through lightweight, zero-initialized linear bridges trained directly with task rewards and supports hybrid-attention architectures. PLatour significantly outperforms open-source baselines on generation tasks and surpasses Mostik's reported results on shared mathematics and coding benchmarks using a substantially smaller sender. However, controlled experiments show that the gains on some tasks can be reproduced by uninformative sender signals, self-communication, or receiver-only training. We find that trainable bridges can act as parameter-efficient adaptation mechanisms without exploiting useful sender information. We therefore propose PACT-Check to quantify how much of a trained PACT system's gain genuinely comes from communication. Finally, we discuss the opportunities and challenges of PACT systems. Code is available at https://github.com/zhenyi4/platour.
Alex: They initialize the bridge weights to zero. Before training, the connection changes nothing, preserving the receiver's original behaviour. They connect layers at roughly corresponding depths, and changes introduced lower down flow through the receiver's later layers.
Sam: And what teaches the bridge which additions are helpful?
Alex: Reinforcement learning: training with rewards for generated answers. Both original models stay frozen; only the bridge changes. Unlike alignment or distillation approaches, the system isn't first taught to match internal representations or imitate the sender's answers.
Sam: So matching the models isn't the target. Getting the smaller model to solve the task is.
Alex: That's the distinction. The authors treat the sender's representations as tools the receiver might use. Their bridge uses simple linear projections, which transform those representations without a more elaborate network.
Sam: But a trainable connection is also extra machinery attached to the receiver. Couldn't that improve performance even if the sender contributes nothing useful?
Alex: That's the central problem. Freezing the original models doesn't eliminate adaptation. A trained bridge can change the receiver's computation, so an improvement doesn't automatically identify information transfer as the reason.
Sam: How do they separate those explanations rather than just acknowledge the ambiguity?
Alex: They introduce PACT-Check, with complementary controls. The first is dummy transfer. It keeps fixed sender features from the prompt template while removing question-dependent communication. The trained bridge stays unchanged, and the sender isn't called during this test.
Sam: That asks whether a reusable conditioning signal can substitute for reading the actual question. What does the second intervention test?
Alex: Irrelevant transfer supplies sender representations from an unrelated question, while the receiver still gets the original problem. A drop shows sensitivity to mismatched information. But sensitivity alone doesn't establish useful transfer: the wrong input could simply disrupt the system.
Sam: Those interventions test an already trained connection. What if another training route achieves the same improvement without an external sender?
Alex: That's why they also train separate controls. Self-communication uses the receiver as both sender and receiver. In a particular matching setup, this becomes a residual adapter: a learned adjustment to its own hidden states. The trained-receiver control adapts the receiver directly, without sender assistance.
Sam: So the tests ask both whether this system needs relevant information and whether its gains can be reproduced without it.
Alex: And PACT-Score makes the comparison explicit. It's the system's accuracy minus the strongest applicable control's accuracy. A positive margin supports a benefit beyond those controls, not a complete explanation of what information travelled.
Sam: Give us a case where the larger sender seems to contribute something.
Alex: On Chemistry questions requiring generated reasoning, the standalone receiver scored about forty-one percent. PLatour reached about sixty-nine percent. This was the Qwen model setting with a very large sender and a much smaller receiver.
Sam: That's a substantial improvement over the original receiver. How much survives comparison with adaptation?
Alex: The strongest control, direct receiver training, reached about sixty-six percent. PLatour therefore retained a modest advantage over every tested control. But Chemistry had only two hundred and ten test questions, so I'd keep that positive result tightly scoped.
Sam: And where does the headline improvement become less convincing evidence of communication?
Alex: On GSM8K, a mathematical word-problem benchmark, PLatour gained about eight and a half percentage points over the original receiver. Yet its margin over the strongest control was only about a quarter of a point. Unrelated sender information reproduced almost all its performance.
Sam: The system got better, but that result doesn't tell us the sender supplied useful problem-specific knowledge.
Alex: The harder mathematics and coding experiments reinforce that concern. Neither tested PACT system beat its compared trained-receiver baseline on domain-average accuracy. Fixed or irrelevant sender signals also reproduced much of the improvement.
Sam: Yet the authors also outperform Mostik. How should a listener weigh that comparison?
Alex: PLatour exceeded Mostik's reported results on shared mathematics and coding benchmarks using a sender with thirty-five billion parameters. Mostik used a sender with seven hundred and fifty-three billion. But its training data are undisclosed, and PLatour trains separate bridges for mathematics and coding.
Sam: So it's a useful reference result, not a controlled demonstration that this architecture is superior. Does the efficiency argument hold?
Alex: Their timing experiments show additional upfront cost from processing the prompt through both models. For longer outputs, that cost is spread over more generation, and total inference time approaches the receiver alone. That's an opportunity, not evidence that every task needs a sender.
Sam: What most limits the broader conclusion about communication?
Alex: The evidence remains specific to the tested controls, tasks, and models. The main experiments connect models within the same family, so transfer across unrelated model families remains open. They also observe training instability with larger senders; greater sender size didn't reliably improve results.
Sam: Who should read the full document, and where should they start?
Alex: Researchers building latent communication systems or efficient inference pipelines should start with PACT-Check. Then read the results asking whether gains depend on useful sender information, especially Chemistry versus mathematics. Read the bridge architecture afterward if you want to implement the open-source framework.
Sam: And what should everyone else carry into their next discussion of collaborating models?
Alex: Better answers show that the system improved. To show communication helped, test what happens when the information is gone.
Sam: Keep that distinction in mind.