Embodied agents, such as robots, will need to interact in situated environments where successful communication often depends on reasoning over social norms: shared expectations that constrain what actions are appropriate in context. A key capability in such settings is norm-based reference resolution (NBRR), where interpreting referential expressions requires inferring implicit normative expectations grounded in physical and social context. Yet it remains unclear whether Large Language Models (LLMs) can support this kind of reasoning. In this work, we introduce SNIC (Situated Norms in Context), a human-validated diagnostic testbed designed to probe how well state-of-the-art LLMs can extract and utilize normative principles relevant to NBRR. SNIC emphasizes physically grounded norms that arise in everyday tasks such as cleaning, tidying, and serving. Across a range of controlled evaluations, we find that even the strongest LLMs struggle to consistently identify and apply social norms, particularly when norms are implicit, underspecified, or in conflict. These findings reveal a blind spot in current LLMs and highlight a key challenge for deploying language-based systems in socially situated, embodied settings.
Alex: Welcome to another episode of ResearchPod. Sam, what paper are we diving into today?
Sam: We're looking at a study titled "Where Norms and References Collide: Evaluating LLMs on Normative Reasoning," by researchers from Tufts University and the Army Research Lab. The central puzzle it uncovers is this: large language models—the powerful AI systems behind chatbots like GPT—often fail at picking the right object in everyday scenes unless you spell out the unwritten social rules for them.
Alex: So, basically, the paper is testing whether these AIs can figure out what people really mean when there's more than one choice, like handing over the right mug in a kitchen?
Sam: Yes, exactly. Imagine you're in a shared kitchen and someone says, "Pass me a mug." There are three nearby: one clean, two dirty with old coffee. Without extra words, most people pick the clean one because of a basic shared rule—you serve drinks with clean items, not used or grimy ones. The study shows AIs grab a dirty one about half the time, missing that hidden expectation.
Alex: Huh. Right, so the core problem here is these AIs aren't reading between the lines on everyday social habits?
Sam: That's the heart of it. Social norms are like unspoken agreements that guide what feels right in a situation—things like keeping food areas clean or tidying by priority. The researchers built a test set called SNIC to check this: they created scenes from real tasks like cleaning or serving, with tricky choices that only make sense if you apply those norms.
Alex: Okay, so it's not just understanding words, but layering on what people expect without saying it. What makes these norms so hard for the AIs to pick up?
Sam: Norms aren't always spelled out in books or data; they're picked up from context, like avoiding dirty dishes when serving guests, but grabbing them for cleanup. They can clash too—one rule might say prioritize safety, another hygiene—and AIs waver inconsistently across similar setups.
Alex: Okay, so these clashes make it tricky. But how did the researchers make sure their test scenes really captured what humans expect—did they just guess, or check with real people first?
Sam: They started small to get it right. First, they created 120 basic scenes—short stories set in places like kitchens or libraries, describing a task like cooking or cleaning, with a vague request like "hand me the mug" and several similar items around. Each scene hid a social rule that pointed to one right choice, like picking the clean mug for cooking. Then, to confirm humans agreed, they ran an online test with 210 everyday people who read these stories, picked what they thought was meant, and explained why. For the best scenes, most people chose the rule-guided item—like over half in serving-clean-items cases—proving the setups worked as planned.
Alex: Huh, so they filtered based on what crowds actually picked. That sounds solid for avoiding researcher bias. What happened next to make a big test set?
Sam: They kept only 51 top scenes where the rule-choice won or tied for most votes—ensuring real human backing. From there, they expanded it to 9,000 by tweaking details: swap mug for spoon, kitchen for restaurant, but keep the same hidden rule driving the pick. It's like remixing a recipe while the key ingredient stays the same, so every new version still tests the same social smarts.
Alex: Okay, so SNIC gives a solid test set rooted in human choices. How exactly did they put these top AIs through their paces—what did the tests look like?
Sam: They set up three clear conditions to probe what the models knew. First, they gave each AI the full scene description plus a precise logical breakdown of everything in it—like tagging each object with a number and listing its traits exactly, so no vague words could trip it up. This uses First-Order Logic, or FOL: a strict way to spell out facts step by step, such as "object one is a clean mug" or "setting is kitchen." They tested models like GPT-4o-mini and others on 9,000 scenes this way.
Alex: So that FOL part is basically a cheat sheet making the scene crystal clear, no poetry allowed. But even with that, did the AIs pick the norm-guided item reliably?
Sam: Not really. Accuracy hovered around 44 percent with the full scene and FOL, reaching about 60 percent for the best model. Dropping the FOL to just the scene description barely changed things—still around 44 percent average. The logic here suggests these AIs don't grasp the unwritten rules on their own; extra clarity on objects doesn't help if the social expectation is missing.
Alex: Huh. So formalizing the scene doesn't unlock the norms. What was the third test—did they try feeding in the rules directly?
Sam: Yes—they added an explicit list of the relevant social norms right in the prompt, like "serve clean items" or "tidy hazards first," alongside the scene and FOL. Accuracy jumped to 70 percent across models, showing these systems can follow norms when told what they are. This highlights a key gap: LLMs have the smarts to apply rules but lack built-in knowledge of everyday ones, at least without direct guidance.
Alex: Right—that makes the blind spot sharp. It's not reasoning power; it's missing the cultural playbook. Does the paper tie this back to real-world robot tasks, like avoiding the dirty mug faux pas?
Sam: Precisely. In home settings, resolving "pass the mug" without norm awareness leads to odd choices, like handing a grimy one during cooking. The nine norm categories in SNIC, from cleanliness to conflicts, let them spot patterns. The main takeaway is clear: for agents in kitchens or living rooms, explicit norm training could bridge this.
Alex: So overall, these tests paint a picture of models that can apply rules when handed to them, but not spot them independently. Did the results show any patterns in how models handled different types of rules?
Sam: They did. Some rule types showed links because of similar structures—for instance, serving clean items aligned with cooking clean ones. More notably, negative correlations appeared between tidying rules and conflicts, like models favoring safety over neatness even when prompts suggested switching priorities. This suggests the models follow a fixed hierarchy from training, not flexible reasoning about clashes.
Alex: Huh, so they stick to a baked-in order rather than adapting. And how did models do on those original 51 human-checked scenes compared to the full set?
Sam: Performance edged up slightly on the smaller set—confirming the expansion added tougher variations and distractors. Scaling stressed consistency across more realistic tweaks, exposing wobbles that simpler tests miss.
Alex: Right—that reinforces the dataset's value. It seems the root issue is these norms just aren't common enough in the text AIs train on to pick up naturally.
Sam: Precisely. Unlike everyday facts that get repeated often, these rules—like avoiding a stranger's toothbrush or prioritizing hazards—are rarely spelled out directly. They're culturally shared but implicit, so training data lacks clear examples of norm-guided picks in scenes.
Alex: Makes sense for why norms trip them up more than plain logic. Looking ahead, what does this mean practically for things like home robots?
Sam: It points to needs in assistive systems—for instance, a robot cleaning a table should prioritize spills over dishes, or serve clean mugs without prompting. Embedding norm awareness could make agents safer and more intuitive in homes. The study suggests targeted training on datasets like SNIC would help.
Alex: Fair point. But the paper flags some limits too—anything worth noting there?
Sam: Yes, a few key ones. It's text-only, so no testing with vision or real-world cues that robots use. The augmented scenes weren't rechecked by humans, and it skips training tweaks like feedback loops. These keep the focus on current models' baselines, but future work needs multimodal and training tests.
Alex: So while it highlights a clear capability gap—and a path via explicit norms—real deployment will need those extensions. Overall, SNIC offers a solid tool to track progress on these unwritten rules. Thanks, Sam—that's a thorough look at how norms shape everyday AI choices. Listeners, that's our dive into this Tufts and Army Research Lab study. Thanks for listening to ResearchPod.