Creative capabilities of MLLMs matter in design, communication, education, and human--AI collaboration, yet remain difficult to evaluate because explicit targets and reward signals are scarce compared with accuracy-oriented tasks. Cross-concept understanding is a core cognitive capacity underlying receptive creativity. It enables a perceiver to recover intended meaning from non-obvious but meaningful conceptual relations. We operationalize item construction as cross-concept encoding and model inference as cross-concept decoding. We introduce C4, a cognition-inspired evaluation framework for Chengyu (Chinese idiom)-based Cross-Concept Creativity. Its encoding component maps target slots to imageable substitute concepts along bridge paths in a manually annotated and third-party-reviewed cross-concept network, enabling batch generation with explicit structure, difficulty indexed by bridge count and depth, and exact answers. Using this framework, we instantiate the C4 Evaluation Set (C4-Eval), comprising 184 synthetic items and 37 human-created cross-concept chengyu figures collected from online sources. We manually construct and review cross-concept relations, bridge paths, and reasoning processes for the collected figures. Each C4-Eval item is instantiated in five task settings, yielding 884 primary answer-recovery cases. Across ten evaluated MLLMs, the strongest closed models reach 50.7% and 48.0% primary accuracy, while open-source models remain substantially lower. Candidate constraints improve accuracy sharply, but bridge hints and explanation requests provide only modest gains. These results expose a substantial gap in how current MLLMs decode creatively encoded meaning through cross-concept relations. The code is in the supplementary material.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a study that tries to measure something that usually feels impossible to pin down: creativity.
Sam: That's right. The paper explores what the researchers call "receptive creativity." Think of it this way: when you read a poem and understand what the poet really meant—even though they never said it directly—that's receptive creativity. You're reverse-engineering a creative leap. The question this paper asks is whether AI models can do the same thing, or whether they're just making educated guesses based on surface patterns.
Alex: So it's asking whether an AI truly understands the logic behind a creative leap, or whether it's just pattern-matching its way to a plausible-sounding answer?
Sam: Exactly. And the researchers argue that if an AI can't decode a metaphor back to its original meaning, it doesn't really "understand" the concept at all. To test this, they built a new framework called C4. It uses traditional Chinese four-character idioms—called chengyu (/tʃʌŋjuː/)—as the target for creative decoding.
Alex: So they're using these idioms as a kind of creative cipher. The AI has to crack the code.
Sam: That's a good way to put it. A chengyu is a fixed, culturally grounded expression with a very specific meaning—which gives the researchers an exact answer to check against. Here's how the cipher works: they take one of these idioms and replace parts of it with related concepts. Maybe a word that sounds similar, or a cultural symbol that points toward the original meaning. That creates what they call a "bridge path"—a chain of associations the AI has to follow to get back to the original idiom.
Alex: Like a trail of clues leading back to the answer.
Sam: Exactly. And the difficulty scales with how long and how abstract that trail is. They measure this using two things: "bridge count," which is how many steps the path has, and "bridge depth," which is how indirect or culturally specific each step is. A simple puzzle might replace one word with something that sounds nearly identical. A harder one might require the AI to understand layers of cultural symbolism before it can even begin to reconstruct the original meaning.
Alex: So it takes something that feels completely subjective—understanding a creative metaphor—and turns it into a structured, measurable problem. How do the models actually hold up?
Sam: Not well, by the paper's own measure. Even the strongest models they tested only reached around fifty percent accuracy on these tasks. And that's worth sitting with for a moment. These are systems trained on enormous amounts of text, capable of writing essays and solving complex problems—and they're getting roughly half of these creative decoding puzzles right.
Alex: Which suggests that generating creative-sounding text and actually understanding the logic behind creativity are two very different things.
Sam: That's the core finding. The models aren't just missing the answer—they're missing the logic of the connection. And the researchers found a telling detail that helps explain why. When models were given a list of possible answers to choose from, their performance improved notably. That tells you something important: the knowledge is in there somewhere. The models have the right information stored away. What they can't do is actively navigate the chain of associations to reach it on their own.
Alex: So it's like having all the puzzle pieces in front of you, but not being able to see how they fit together without someone showing you the picture on the box.
Sam: That's a fair comparison. And it gets more interesting when you look at what happens when a model does guess the right answer. The researchers found that even then, models often struggled to reconstruct the correct bridge path—the actual chain of reasoning that leads from the image or clue to the idiom. They'd produce an explanation that sounds plausible, but it doesn't match the specific cultural or phonetic steps the puzzle was built on.
Alex: So it's like a student who writes down the right answer on a maths test but shows completely wrong working.
Sam: That's a very accurate comparison. The model is essentially generating a path that looks logical, without actually tracing the intended one. The researchers describe this as a kind of hallucination—not of facts, but of reasoning. The model invents a coherent-sounding explanation rather than recovering the real one.
Alex: Is this a problem that more training data could fix? Or is it something deeper in how these systems are built?
Sam: The paper suggests it's the latter. It's not that the models lack information—the candidate-list experiment showed they often have the right answer stored somewhere. The gap is in the reasoning process itself. Current systems are better at recognising patterns than at following a structured chain of logic from point A to point B to point C. That kind of step-by-step causal reasoning—tracing a bridge rather than guessing at a destination—is where they fall short.
Alex: And that's a meaningful distinction. Because if it were just a data problem, you could theoretically fix it by feeding the model more examples. But if it's a reasoning problem, the solution has to be more fundamental.
Sam: Right. And that's part of what makes this framework useful. By forcing models to show their work—to not just identify the idiom but reconstruct the path that leads to it—C4 gives developers a way to see exactly where the reasoning breaks down. It's not just a test of knowledge; it's a test of cognitive process.
Alex: Though I imagine this framework has its own limits. It's built on Chinese idioms, which are culturally very specific. Would it translate to other languages?
Sam: That's the main limitation the authors acknowledge. The strength of C4 comes from its manually annotated network of associations—people carefully mapped out which sounds, symbols, and cultural references connect to which idioms. That precision is what makes it a rigorous test. But it also means that applying this approach to English, or Arabic, or any other language would require building that entire annotation structure from scratch. It's a specialised instrument, and the authors are candid about that.
Alex: So the next challenge for the field is figuring out how to scale this kind of structured, culturally grounded benchmark across different languages and contexts.
Sam: That's where the research points. The framework itself is the contribution here—not a solution, but a more rigorous way of asking the question. Before this, measuring creative understanding in AI was largely subjective. C4 gives the field a structured, verifiable target. Whether a model truly grasps the logic of a metaphor, or is simply producing a convincing imitation of understanding—that distinction now has a way to be tested.
Alex: And that seems like a meaningful step. Not because it solves the problem, but because it makes the problem visible in a way it wasn't before.
Sam: Exactly. And the fifty percent ceiling these models hit is a useful reminder that fluency isn't the same as understanding. These systems can write, translate, and summarise with considerable skill. But tracing the inner logic of a creative leap—following the bridge from image to meaning—remains genuinely difficult for them. That gap is worth paying attention to. Thanks for listening to ResearchPod.