Yuchen Xian, Yunqiu Xu, Yang He, Yi Yang
5 min
Abstract
Multimodal image fusion aims to integrate complementary information from different modalities into a fused image that preserves rich local details while maintaining globally consistent appearance. Existing approaches build shared representations on 2D feature grids, which excel at modeling local structures but offer limited leverage over image-level global appearance factors. To balance these objectives, we introduce a compact 1D token interface based on a frozen pretrained image tokenizer for modeling non-local appearance/base factors. Rather than using the tokenizer as a reconstruction backbone, our design uses the 1D token space as a global carrier while retaining the 2D spatial pathway for local structure restoration. Specifically, we introduce Selective Token Editing (STE), which sparsely updates/replaces a small set of critical tokens, providing a lightweight mechanism to steer global appearance coherence while keeping the fusion backbone unchanged and avoiding extra losses. Experiments on four commonly used benchmarks show that our method achieves the best overall performance, with consistent, multi-metric improvements in both global coherence and local fidelity. Project page: https://zju-xyc.github.io/1D-Fusion-Project-Page/
Alex: That's the next piece of the puzzle. The researchers use a technique they call Selective Token Editing. The idea is to identify the small handful of entries in that list that have the most influence over the final appearance—and only adjust those. Think of it like a sound engineer at a mixing desk. Instead of moving every single slider, they've learned from experience that just three or four sliders control most of what the audience hears.
Sam: And the system learns which sliders those are automatically?
Alex: Yes. They use a mathematical tool that helps the system figure out, during training, which entries are the most influential. It works by turning what would normally be a fuzzy, gradual selection into a crisp, decisive one—so the system commits to specific choices rather than spreading its adjustments thinly across everything.
Sam: So instead of retraining the whole system every time you want to adjust the lighting, you just tweak a few specific entries in that list. That's a much more targeted approach.
Alex: And a more stable one. One of the persistent problems in image fusion is what the authors call the many-to-one issue—where changing one aspect of the image inadvertently affects something else. By isolating the global controls in their own dedicated space and only touching the most critical ones, the system avoids that cascade of unintended changes.
Sam: Okay, so you've got your global settings handled cleanly in that compact list. But the final image still has to be a 2D picture. How do those two worlds come back together?
Alex: The researchers address that with what they call a Token-to-Map interface. The compact list gets expanded back into a rough 2D layout, and then the system fills in the fine spatial detail—essentially recovering the high-resolution texture that the 2D pathway was preserving all along. The global guidance and the local detail are merged at the end, rather than being forced to coexist from the start.
Sam: So the whole architecture is really about keeping things separate for as long as possible, and only combining them once each part has been handled properly.
Alex: That's the core insight. By giving global properties their own dedicated, compact representation—and by being selective about which parts of that representation to adjust—the system can maintain sharp local detail while still producing consistent, well-balanced lighting across the whole scene. It's a structural solution to what had previously been treated as a tuning problem.
Sam: It makes you think about how often the way we represent a problem shapes what solutions seem possible. The researchers didn't find a better way to manage the 2D grid—they questioned whether the 2D grid was the right tool for part of the job at all.
Alex: And that reframing is what the paper is really about. Sometimes the most useful step is asking whether the structure you've inherited is actually suited to the task. Thanks for listening to ResearchPod.