We present $N_0$-TWAM, a tactile-native world-action model for contact-rich manipulation that predicts both future vision and future contact. To our knowledge, it is the first tactile world-action model trained at large scale, and it shows strong capability on contact-rich tasks. We pre-train $N_0$-TWAM at large scale with visuo-tactile joint training over tactile-rich demonstrations spanning six embodiments and 450 tasks. We use NeoForce, a unified force-based tactile representation, to form a physically grounded contact signal that conditions action generation. To improve long-horizon and multi-stage manipulation, we introduce tactile contact events for task staging and advance through them during execution. For real-time efficiency, we adopt an asymmetric Mixture-of-Transformers architecture that pairs a full-width expert for video prediction with slim experts for downstream action and tactile prediction. Evaluations on both real and simulated benchmarks justify the capabilities of $N_0$-TWAM across a range of contact-rich tasks, and demonstrate the benefit of data scaling for precise tactile and action prediction. In summary, $N_0$-TWAM endows a world-action model with predictive capabilities to foresee vision, touch and action, building a solid foundation for fine-grained manipulation on open contact-rich tasks. The codebase and model checkpoints will be made publicly available to foster further research and development in tactile-enabled robotic manipulation.
Alex: Welcome to another episode of ResearchPod.
Sam: Today we're looking at a new robotic system called N0-TWAM, and the problem it's trying to solve is one that trips up almost every robot in use today. Most robots rely almost entirely on cameras — they see the world, but they can't feel it. That works fine for simple tasks, but it falls apart the moment a robot needs to do something delicate, like plugging in a cable or assembling a small component. Those tasks depend on touch, not just sight.
Alex: Like plugging in a USB cable in the dark. You can't see the port, but your fingers find it anyway.
Sam: Exactly that. And N0-TWAM is built around that same principle — it treats touch as a primary sense, equal to vision. But it goes one step further. Rather than just reacting to what it feels, the robot tries to predict how a physical interaction will feel before it even happens.
Alex: So most robots are purely reactive — they wait to feel something go wrong. N0-TWAM is trying to anticipate?
Sam: That's the core shift. Think of a chef who doesn't just watch the pan, but also feels the heat and anticipates how the food will change texture before it burns. N0-TWAM builds a mental prediction of both what it will see and what it will feel, and then uses that prediction to guide its movements in real time.
Alex: How does it actually build that prediction? You mentioned something called a "Mixture-of-Transformers."
Sam: The name sounds technical, but the concept is straightforward. Imagine a small team of specialists working together. There's a Vision Expert that handles what the robot sees, a Tactile Expert that handles what the robot feels through its fingertips, and an Action Expert that decides what the robot should do next. They all share the same mental workspace — so they can communicate instantly — but each one also keeps its own private notes for things only it needs to track. That way, the robot processes vision and touch simultaneously, without one drowning out the other.
Alex: Why keep them separate at all, though? Why not just feed everything into one system?
Sam: Because the two types of data are very different in scale. Visual information is dense — every frame is packed with detail. Tactile signals, by contrast, are subtle. If you mix them together carelessly, the visual data tends to overwhelm the touch data, and the robot loses exactly the fine-grained feedback it needs. Keeping separate experts preserves the clarity of each signal. There's also a practical efficiency gain: once the model has made its predictions, it stores those results so the Action Expert can act quickly without recalculating everything from scratch.
Alex: So it's a predict-then-act loop. The robot imagines what the future looks and feels like, then moves accordingly.
Sam: That's the right way to think about it. And there's a clever detail in how it handles the gap between prediction and reality. The system runs two parallel streams at once. One stream carries the robot's prediction — what it expects to feel. The other carries the live sensor data — what it's actually feeling right now. By comparing those two streams in real time, the robot can confirm it's on track. If it's inserting a cable and feels the expected resistance followed by a click at the right moment, it knows the task is going well. If something feels off, it can adjust.
Alex: Does the research show this actually performs better than systems that rely on vision alone?
Sam: The results are notable. N0-TWAM outperformed vision-only systems by a meaningful margin on the tasks tested. The authors also ran what are called ablation studies — essentially, they deliberately removed parts of the system one at a time to see what broke. When they removed the predictive foresight, performance dropped. When they removed the real-time tactile observation, it dropped again. That tells you both components are doing real work — neither is just decorative. For fine-grained physical manipulation, touch isn't a bonus feature. It's load-bearing.
Alex: Are there limits to how far this scales? What happens with longer, more complex tasks?
Sam: That's where the current design shows its constraints. The model is built for speed and precision on short, well-defined movements. But it uses a strict sequential structure — each step builds directly on the last. If the robot misses a tactile cue early in a sequence, that error can compound, making recovery difficult over longer tasks. The authors identify extending the prediction window as a key next step. If the robot could anticipate further ahead in its own movement sequence, it could handle more complex chains of action.
Alex: What about the sensors themselves — is the system locked to one specific type?
Sam: Currently, yes. It's trained on vision-based tactile sensors, which work a bit like a soft gel surface that deforms under pressure and captures that deformation as an image. But the researchers point toward a broader goal: incorporating other types of feedback, like thermal or acoustic sensors. A system trained on a wider variety of inputs would be more flexible — better able to transfer to different robot hands or different materials without being retrained from scratch.
Alex: It's a meaningful shift in how we think about robots. Not just machines that see the world, but ones that can physically interact with it — and feel their way through uncertainty the way we do. Thanks for walking us through this, Sam. And thank you all for listening to ResearchPod.