Multimodal image fusion aims to integrate complementary information from different modalities into a fused image that preserves rich local details while maintaining globally consistent appearance. Existing approaches build shared representations on 2D feature grids, which excel at modeling local structures but offer limited leverage over image-level global appearance factors. To balance these objectives, we introduce a compact 1D token interface based on a frozen pretrained image tokenizer for modeling non-local appearance/base factors. Rather than using the tokenizer as a reconstruction backbone, our design uses the 1D token space as a global carrier while retaining the 2D spatial pathway for local structure restoration. Specifically, we introduce Selective Token Editing (STE), which sparsely updates/replaces a small set of critical tokens, providing a lightweight mechanism to steer global appearance coherence while keeping the fusion backbone unchanged and avoiding extra losses. Experiments on four commonly used benchmarks show that our method achieves the best overall performance, with consistent, multi-metric improvements in both global coherence and local fidelity. Project page: https://zju-xyc.github.io/1D-Fusion-Project-Page/
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a new approach to combining images from different types of sensors—something researchers call multimodal image fusion.
Sam: So before we get into the solution, what's the actual problem? Why do these combined images often look muddy or washed out?
Alex: Good starting point. Imagine you have two cameras pointed at the same scene. One is a regular visible-light camera, like the one on your phone. The other is an infrared camera, which detects heat instead of light—so it can "see" a person hiding in the dark just from their body warmth. Combining those two images should give you the best of both worlds: the sharp detail of the regular camera and the heat-sensing ability of the infrared one.
Sam: But something goes wrong in that combination?
Alex: Right. The problem is how computers currently handle that combination. They treat the image as a giant grid—every pixel has a position, and the system tries to manage everything through that same grid. The authors compare it to trying to run a whole city's electricity through a single neighbourhood's wiring. The system gets overloaded.
Sam: So the grid is trying to do two very different jobs at once—handle the fine details and manage the big-picture lighting—and it can't do both well?
Alex: Exactly. And here's why that matters: overall brightness and contrast aren't really about any single pixel. They're properties of the whole scene. When you force those global properties into the same grid as the fine local details—the edge of a building, the texture of a jacket—they get tangled together. You try to brighten the scene, and suddenly the sharp edges blur. You try to preserve the detail, and the lighting looks off.
Sam: So the fix has to be about separating those two jobs. Give each one its own dedicated space.
Alex: That's precisely what the researchers do. They introduce what they call a 1D token interface. Instead of cramming global information into the 2D pixel grid, they store it in a short, compact list of numbers—like a small control panel sitting beside the main image. Each entry in that list represents a global property of the scene: overall brightness, contrast, that kind of thing.
Sam: So it's less like a map and more like a list of dials?
Alex: That's a good way to put it. The 2D grid stays focused on spatial detail—textures, edges, fine structure. The 1D list handles the scene-wide settings. Because they're separate, adjusting one doesn't disturb the other.
Sam: But how does the system know which entries in that list actually matter? If you have a long list of dials, some of them must be more important than others.
Alex: That's the next piece of the puzzle. The researchers use a technique they call Selective Token Editing. The idea is to identify the small handful of entries in that list that have the most influence over the final appearance—and only adjust those. Think of it like a sound engineer at a mixing desk. Instead of moving every single slider, they've learned from experience that just three or four sliders control most of what the audience hears.
Sam: And the system learns which sliders those are automatically?
Alex: Yes. They use a mathematical tool that helps the system figure out, during training, which entries are the most influential. It works by turning what would normally be a fuzzy, gradual selection into a crisp, decisive one—so the system commits to specific choices rather than spreading its adjustments thinly across everything.
Sam: So instead of retraining the whole system every time you want to adjust the lighting, you just tweak a few specific entries in that list. That's a much more targeted approach.
Alex: And a more stable one. One of the persistent problems in image fusion is what the authors call the many-to-one issue—where changing one aspect of the image inadvertently affects something else. By isolating the global controls in their own dedicated space and only touching the most critical ones, the system avoids that cascade of unintended changes.
Sam: Okay, so you've got your global settings handled cleanly in that compact list. But the final image still has to be a 2D picture. How do those two worlds come back together?
Alex: The researchers address that with what they call a Token-to-Map interface. The compact list gets expanded back into a rough 2D layout, and then the system fills in the fine spatial detail—essentially recovering the high-resolution texture that the 2D pathway was preserving all along. The global guidance and the local detail are merged at the end, rather than being forced to coexist from the start.
Sam: So the whole architecture is really about keeping things separate for as long as possible, and only combining them once each part has been handled properly.
Alex: That's the core insight. By giving global properties their own dedicated, compact representation—and by being selective about which parts of that representation to adjust—the system can maintain sharp local detail while still producing consistent, well-balanced lighting across the whole scene. It's a structural solution to what had previously been treated as a tuning problem.
Sam: It makes you think about how often the way we represent a problem shapes what solutions seem possible. The researchers didn't find a better way to manage the 2D grid—they questioned whether the 2D grid was the right tool for part of the job at all.
Alex: And that reframing is what the paper is really about. Sometimes the most useful step is asking whether the structure you've inherited is actually suited to the task. Thanks for listening to ResearchPod.