Jongwook Han, Hyeongjin Kim, Yohan Jo
5 min
Abstract
While household robots are often evaluated based on task completion, everyday domestic environments involve value-conflicting situations in which robots are expected to choose actions that prioritize other values than task success, such as human autonomy, efficiency, or social appropriateness. Yet, there are no benchmarks for evaluating robots' value preferences in such scenarios. We introduce RobotValues, a benchmark to evaluate household robot planners in 10K value-conflict scenarios. Each instance consists of a realistic household image with multiple plausible robot actions that prioritize different human values. We construct RobotValues through LLM-assisted scenario generation, stakeholder-grounded value extraction, image generation and automatic quality control. Using RobotValues we evaluate VLMs used in robotics and find that models exhibit default value preferences, including safety and accommodation, while underselecting privacy-prioritizing actions. When the models are instructed to prioritize specific values that conflict with their own preferences, they often fail to override their default actions, choosing incorrect actions for 80% of the time. These findings suggest that household robot evaluation should measure not only task completion or safety compliance, but also whether robots can choose among plausible actions when human values conflict.
Sam: That's a fair concern, and they addressed it directly. They checked the AI filters against human annotators — real people who rated whether each scenario felt realistic. The AI judges turned out to be quite consistent with human judgment. It's a practical solution: you can't ask a human to personally review ten thousand videos, so you build a reliable automated system and then verify it works.
Alex: Okay. So once they had this benchmark, what did they actually find when they ran it on current robots?
Sam: They found that these robots have rigid default preferences baked in from their training. Most models consistently favored safety or efficiency — and largely ignored privacy. To measure this, the researchers used something called a Bradley-Terry score. Think of it like a sports tournament bracket, but for values. Each value competes against the others across thousands of scenarios, and the score tells you which ones the robot consistently picks as the winner.
Alex: So the robots effectively have a built-in "personality" — a fixed hierarchy of what they think matters. What happens when you explicitly tell one of these robots to prioritize privacy instead?
Sam: That's the central finding. Even with direct instructions, the robots struggled to override their defaults. It's as if the training has already "locked in" a set of priorities, and a simple instruction at the surface level isn't enough to change what's underneath.
Alex: So telling a robot "please respect my privacy" might not actually work, because the robot's deeper training is pulling it in a different direction?
Sam: That's what the data suggests. The researchers also tested the models on real camera images from actual homes, not just text descriptions, and found the same pattern held. The robots couldn't reliably shift away from their trained biases even in more realistic conditions. The implication is that if you want a robot to genuinely respect privacy, you can't just ask it to — you have to build that value into the training from the start.
Alex: Which means the problem isn't really at the instruction level. It's much deeper than that.
Sam: Right. And that's actually why this benchmark matters. Before you can fix a problem, you need a way to measure it clearly. ROBOTVALUES gives researchers a structured way to see exactly which values a robot is ignoring and by how much — which is a necessary first step toward building systems that can genuinely adapt to what different people in different homes actually care about.
Alex: That's a meaningful contribution, then. Not a solution, but a clear diagnostic tool.
Sam: That's a fair way to put it. The paper is careful not to overstate what's been solved. What it offers is a rigorous way to ask the question — and the early answers suggest current systems have some significant gaps when it comes to values that aren't efficiency or safety.
Alex: It makes you think differently about what "a good robot" actually means. It's not just one that completes the task. It's one that completes the task in a way that fits the people it's living with.
Sam: And that turns out to be a much harder problem than it first appears.
Alex: Thanks for listening to ResearchPod.