Aimilios Hadjiliasi, Louis Nisiotis
8 min
The development of embodied Intelligent Virtual Agents (IVAs) that have cognitive capabilities in real-time interactive virtual environments remains a challenge, even with today's advancements in technology. Existing architectures are often focused on either the implementation of low-level reactive control systems that are constrained by commercial game engines, or high-level representations of reasoning models that can be difficult to implement in virtual worlds. This paper builds on that notion and proposes a modular cognitive architecture for deploying embodied IVAs. This architecture builds on existing, pre-established frameworks such as the Sense-Think-Act paradigm and the Belief-Desire-Intention cognitive model, among others, and aims to provide a reusable implementation-oriented framework as a template for deploying IVA "brains" in interactive 3D computing systems. The proposed architecture contributes by providing a modular, implementation-oriented framework for the deployment of embodied, cognitive-capable IVAs and bridges the gap between high-level agent reasoning models with real-time embodied execution, for scalable, adaptive, and explainable agents in complex interactive virtual environments.
Developing Intelligent Virtual Agents (IVAs) that exhibit human-like cognition while operating within the performance constraints of modern game engines is a persistent challenge. Existing solutions often force a trade-off: they either prioritize low-level reactive behaviors (like state machines) that lack long-term reasoning, or they utilize high-level cognitive models (like BDI or SOAR) that are difficult to integrate into real-time 3D environments. The Cognitive Embodied Agent Architecture (CEAA) is proposed as a modular, implementation-oriented template to unify these levels.
CEAA organizes agent functionality into three distinct layers: the User and Environment Layer, the Knowledge Layer, and the Agent Layer.
By separating the agent's cognitive reasoning from its physical execution, the architecture allows developers to implement complex, goal-oriented behavior that can be mapped to specific animations and actions within engines like Unity or Unreal.
This architecture addresses the "implementation gap" in current IVA development. By providing a structured, modular approach, CEAA enables the creation of agents that are not only reactive to immediate user input but also capable of long-term planning, adaptation, and explainable decision-making. This is particularly relevant for applications in education, training, and virtual museums, where agents must act as autonomous, intelligent partners rather than simple scripted characters.
Alex: How do they organize that?
Sam: They use a model that's been around in artificial intelligence research for a while. The idea is that a reasoning agent should keep track of three things: what it believes to be true about the world, what it wants to achieve, and what it has committed to doing. The paper calls this the Belief-Desire-Intention model. The plain version is simply that the agent shouldn't just react to every new thing that happens — it should hold onto a goal until there's a genuinely good reason to change it.
Alex: Like a person who doesn't abandon their plan just because something minor goes wrong.
Sam: Exactly. And that stability matters because virtual worlds are dynamic. Users move, situations shift, and the agent has to adapt without losing track of what it was trying to do in the first place.
Alex: So how does the architecture actually keep all of that organized?
Sam: The authors split the system into distinct layers, each with a clear job. One layer is the world itself — where users, agents, and objects exist and interact. A second layer keeps a structured record of what has happened in that world, so events don't just vanish the moment they occur. Think of it like a shared noticeboard that every part of the system can read from and add to. The third layer is where the agent does its own reasoning and turns decisions into visible actions.
Alex: Why the shared noticeboard? Why not just have the agent watch the world directly?
Sam: Because a busy virtual world generates a huge amount of information, and the agent can't — and shouldn't — pay attention to all of it. The noticeboard approach means the world reports changes once, and the agent only looks for the pieces that are relevant to its own goals. That selective attention keeps the agent from wasting effort on things that don't matter to it.
Alex: So the agent is filtering, not just absorbing everything.
Sam: Right. And once it finds something relevant, it checks its stored experience to see whether something similar has happened before. That's where memory comes in — not just a diary of past events, but a searchable record the agent can use to recognize patterns and avoid treating every situation as completely new.
Alex: What does that stored experience actually contain?
Sam: The paper describes it as including past events, factual knowledge, general concepts, and even models of how specific users tend to behave. The agent can search this store by matching the current situation to similar past ones, which helps it give responses that feel context-aware rather than generic.
Alex: And after it checks its memory, what happens?
Sam: The central coordinator — the part that manages the whole decision process — takes the relevant information, checks it against the agent's current beliefs and goals, and works out what to do next. It compares competing possible goals, picks the one that matters most right now, and turns that into a committed intention. If the situation changes significantly, it can revise that intention. The paper says this process might use rule-based reasoning or other decision methods depending on the application.
Alex: So the flow is: notice something, filter it, check memory, update the goal, then plan the action?
Sam: That's the core loop. Once a decision is made, a planner turns it into an ordered sequence of steps — still at an abstract level, like "greet the user" or "explain this exhibit." Then a separate layer translates that abstract plan into something the character's body can actually perform: speech, movement, a gesture, a change in gaze direction.
Alex: Why does the body side need its own layer? Why not just go straight from decision to action?
Sam: Because an agent can only do what it's physically set up to do. The paper treats the agent's capabilities — its movement range, facial expressions, voice, gaze — as a kind of database. Before the agent acts, it checks that database to make sure the action it's chosen is actually something this particular character can perform. That step is what keeps the visible behavior believable rather than jarring.
Alex: So the visible character isn't just decoration. It actually shapes what the agent can express.
Sam: That's the paper's point about embodiment. A believable virtual agent depends on matching inner decisions with outward behavior — especially in real time, where timing matters. As a concrete example, the authors describe using this framework in a Unity-based virtual museum about ENIAC, one of the earliest electronic computers. Embodied teaching agents in that museum could guide users, answer questions, and give feedback, and the design allowed an agent to notice when a presentation had already been delivered and adjust what it did next rather than simply repeating itself.
Alex: That's a useful illustration. It shows the memory layer actually doing something practical.
Sam: It does. And that's an important distinction — the memory isn't passive storage. It actively changes what the agent decides to do. The paper also reports that earlier studies using this approach in both desktop and VR versions of the museum showed users responded positively and that learning outcomes were supported. But the authors are careful to say those results are encouraging, not conclusive.
Alex: What's still missing, then?
Sam: The authors are straightforward about this. The architecture is still at a conceptual and prototype stage. It hasn't been rigorously benchmarked against other approaches on things like processing delay, how easily developers can build with it, or how well it scales when the virtual world gets more complex. More technical evaluation is needed before those questions have clear answers.
Alex: So the paper is offering a structured way to think about the problem, not a finished solution.
Sam: That's a fair summary. The contribution is organizational: keep the environment, the knowledge store, the reasoning process, and the action system distinct but connected. That separation makes each part easier to build, easier to explain, and easier to adapt when the application changes. Whether it outperforms other approaches in practice is a question the authors leave open for future work.
Alex: It sounds like a useful framework for anyone trying to build virtual agents that do more than just react.
Sam: That's the intent. The paper's broader argument is that virtual agents in real-time environments need to be goal-directed and context-aware, not just fast. And achieving that requires a design where the thinking, the memory, and the visible behavior are all part of the same coherent system rather than bolted together as an afterthought. Thanks for listening to ResearchPod.