While household robots are often evaluated based on task completion, everyday domestic environments involve value-conflicting situations in which robots are expected to choose actions that prioritize other values than task success, such as human autonomy, efficiency, or social appropriateness. Yet, there are no benchmarks for evaluating robots' value preferences in such scenarios. We introduce RobotValues, a benchmark to evaluate household robot planners in 10K value-conflict scenarios. Each instance consists of a realistic household image with multiple plausible robot actions that prioritize different human values. We construct RobotValues through LLM-assisted scenario generation, stakeholder-grounded value extraction, image generation and automatic quality control. Using RobotValues we evaluate VLMs used in robotics and find that models exhibit default value preferences, including safety and accommodation, while underselecting privacy-prioritizing actions. When the models are instructed to prioritize specific values that conflict with their own preferences, they often fail to override their default actions, choosing incorrect actions for 80% of the time. These findings suggest that household robot evaluation should measure not only task completion or safety compliance, but also whether robots can choose among plausible actions when human values conflict.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at how household robots make decisions when there isn't one clear "right" answer.
Sam: We're discussing a new study about household robots. The core puzzle is this: robots are usually tested on whether they can complete a task — did it clean the floor, did it pick up the cup? But in real homes, robots constantly face situations where doing the task well might conflict with something else people care about, like privacy or personal space.
Alex: So this paper is basically asking whether we can test these robots on their "moral" judgment, rather than just their ability to clean a floor?
Sam: Exactly. The researchers created a benchmark called ROBOTVALUES — a collection of ten thousand scenarios where a robot must choose between competing priorities. Safety versus respect for someone's personal space, for example. It's a way to measure how a machine navigates the messy trade-offs of daily life.
Alex: Why is this so hard to measure in the first place?
Sam: The challenge is that values are what the researchers call "situated" — meaning the right answer depends entirely on context. Imagine a robot deciding whether to keep vacuuming while someone is asleep in the next room. If it continues, it's being efficient. If it stops, it's respecting the person's rest. Neither choice is objectively wrong. You can't just grade that on a pass-fail scale.
Alex: Right. It's not just about whether the robot has good sensors. It's about the internal logic it uses to decide what matters most in that moment.
Sam: Exactly. And to capture that, the researchers used what they call a "stakeholder-grounded" approach. Think of it like putting each decision on trial. For every scenario, the system generates reactions from the people affected by the robot's choice — the person sleeping, a visiting guest, whoever is involved. Those reactions are then used to figure out which values the robot's action actually served or ignored.
Alex: That sounds like an enormous undertaking. How did they actually build ten thousand of these scenarios without just making things up?
Sam: They used a multi-stage pipeline. It started with what they call "persona seeds" — detailed profiles of different kinds of households, different family structures, different living situations — to make sure the scenarios covered a wide range of real-world contexts. Then they used large language models, which are the same kind of AI systems behind modern chatbots, to generate the scenarios and the possible actions a robot might take. Filters then sorted out anything unrealistic or low-quality.
Alex: Wait — so they used AI to judge the AI's work? How do we know those judges were actually reliable?
Sam: That's a fair concern, and they addressed it directly. They checked the AI filters against human annotators — real people who rated whether each scenario felt realistic. The AI judges turned out to be quite consistent with human judgment. It's a practical solution: you can't ask a human to personally review ten thousand videos, so you build a reliable automated system and then verify it works.
Alex: Okay. So once they had this benchmark, what did they actually find when they ran it on current robots?
Sam: They found that these robots have rigid default preferences baked in from their training. Most models consistently favored safety or efficiency — and largely ignored privacy. To measure this, the researchers used something called a Bradley-Terry score. Think of it like a sports tournament bracket, but for values. Each value competes against the others across thousands of scenarios, and the score tells you which ones the robot consistently picks as the winner.
Alex: So the robots effectively have a built-in "personality" — a fixed hierarchy of what they think matters. What happens when you explicitly tell one of these robots to prioritize privacy instead?
Sam: That's the central finding. Even with direct instructions, the robots struggled to override their defaults. It's as if the training has already "locked in" a set of priorities, and a simple instruction at the surface level isn't enough to change what's underneath.
Alex: So telling a robot "please respect my privacy" might not actually work, because the robot's deeper training is pulling it in a different direction?
Sam: That's what the data suggests. The researchers also tested the models on real camera images from actual homes, not just text descriptions, and found the same pattern held. The robots couldn't reliably shift away from their trained biases even in more realistic conditions. The implication is that if you want a robot to genuinely respect privacy, you can't just ask it to — you have to build that value into the training from the start.
Alex: Which means the problem isn't really at the instruction level. It's much deeper than that.
Sam: Right. And that's actually why this benchmark matters. Before you can fix a problem, you need a way to measure it clearly. ROBOTVALUES gives researchers a structured way to see exactly which values a robot is ignoring and by how much — which is a necessary first step toward building systems that can genuinely adapt to what different people in different homes actually care about.
Alex: That's a meaningful contribution, then. Not a solution, but a clear diagnostic tool.
Sam: That's a fair way to put it. The paper is careful not to overstate what's been solved. What it offers is a rigorous way to ask the question — and the early answers suggest current systems have some significant gaps when it comes to values that aren't efficiency or safety.
Alex: It makes you think differently about what "a good robot" actually means. It's not just one that completes the task. It's one that completes the task in a way that fits the people it's living with.
Sam: And that turns out to be a much harder problem than it first appears.
Alex: Thanks for listening to ResearchPod.