Real-time 3D human mesh recovery from a single image on mobile devices is a challenging problem with many applications, including robotics, biomechanics, AR/VR, and exergames. Current state-of-the-art models are either too slow to run efficiently on mobile devices or sacrifice accuracy for speed. We present InstantHMR, a family of open-source single-image regressors suited for mobile inference, based on the Momentum Human Rig (MHR) parametric model and trained exclusively on publicly available annotations from the SAM 3D Body dataset. Our approach achieves remarkable performance on a Samsung Galaxy S23 NPU, recovering a 6,890-vertex body mesh in 5.5 ms per person, and 0.98 ms on an NVIDIA RTX 4070 GPU. This speed surpasses all existing methods we evaluated. On mobile, it is 12x faster than the single mobile-compatible method that slightly outperforms ours in accuracy, while achieving competitive accuracy with 40.0 mm PA-MPJPE on 3DPW, 48.8 mm on EMDB and 41.7 mm on RICH. To ensure fair comparisons, we re-evaluate 11 state-of-the-art methods under a unified protocol aligned with the closest MHR-based competitors, including Meta's official MHR-to-SMPL conversion, and report their reproduction results. Additionally, we measure latency across eight different targets, ranging from desktop TensorRT to two phone NPUs. Code and models are available at https://github.com/Naturalpad-studio/InstantHMR.
Sam: A phone can reconstruct a person's three-dimensional body shape fast enough for live feedback. This study makes that practical on tested Samsung phone accelerators, without matching the most accurate models.
Alex: For someone building a rehabilitation tool or a robot interface, the distinction matters. Is the contribution better body estimates, or less waiting for them?
Sam: Less waiting, while retaining competitive accuracy. InstantHMR is a method paper and arXiv preprint by Mohamed Adjel, Kahina Chalabi, and Vincent Bonnet, from NaturalPad and LAAS-CNRS. Their medium model produces a body surface in about five and a half milliseconds on a Galaxy S23's neural processing unit, a chip specialized for running neural networks.
Alex: That sounds useful wherever feedback needs to follow movement immediately. But speed alone doesn't tell us whether the reconstructed body is good enough.
Sam: That's the central trade-off for robotics, rehabilitation, and augmented reality researchers. We'll unpack why this model suits a phone, how the authors tested that trade-off, and what remains unproven for live use.
Alex: Start with the bottleneck. Other systems already reconstruct bodies from pictures, so what kept those systems off phones?
Sam: The task is called human mesh recovery: estimating a three-dimensional body surface from an image. Recent accurate systems were built for graphics processors. Several couldn't run on the tested phone accelerators at all, because of memory limits or failures after compilation. Faster phone-compatible alternatives generally lost accuracy.
Alex: And a video system can look fast while still making you wait. Does this paper separate processing speed from that kind of delay?
Sam: It does. Some video methods use later frames to improve an earlier estimate, so the answer must wait for those frames. InstantHMR uses only the current image. That makes it a possible building block for an online system, not a complete demonstration of one.
Alex: What is the design choice that makes it phone-friendly? Simply shrinking an existing network wouldn't necessarily solve the hardware problem.
Sam: The model predicts the controls of a body rig rather than every surface point independently. It uses the Momentum Human Rig, a parameterized body model whose controls set pose and shape. Once those controls are predicted, the rig generates the body surface.
Alex: Predicting body-model controls is already an established approach in this field. Where does the new contribution sit?
Sam: In a particular phone-oriented combination, not the invention of parameterized bodies. A convolutional image encoder extracts features using operations phone accelerators handle well. A small transformer decoder, which queries those image features, predicts the body controls and camera position. The entire exported computation, including generating the surface, runs on the accelerator without falling back to the phone's main processor.
Alex: So hardware fit is part of the method. What example shows why that matters beyond just having fewer parameters?
Sam: Consider NLF, an accurate competing method. It estimates surface points through separate spatial queries, then combines scores to locate each point. On the phone accelerator, much of its time goes into moving, rearranging, and reducing large tensors rather than convolutions. InstantHMR instead predicts the rig controls once, then generates the surface through fixed body-model calculations.
Alex: The same workload can therefore look quite different on a desktop and a phone. Does their direct comparison support that interpretation?
Sam: Yes. InstantHMR and the smaller NLF are fairly close on the phone's main processor. But on its neural accelerator, NLF is about twelve times slower. Both run entirely on that accelerator, so the authors can rule out processor fallback as the explanation for this gap.
Alex: Where did the training supervision come from? A compact model could inherit the limitations of whatever larger model labelled its examples.
Sam: They used released human-annotated body fits from the SAM 3D Body dataset, rather than a teacher model's predictions. Training covered about four million person crops drawn from several image sources. None of the benchmark images used for their evaluation were in InstantHMR's training data.
Alex: Was performance mainly about architecture, or did the training recipe change the result too?
Sam: The training recipe mattered substantially. Most training images used the standard person crop, but others were zoomed, shifted, or partly occluded. Mixing those framings produced the largest single improvement the authors measured in their ablations. It teaches the model under different views of the crop, rather than assuming the person is always framed the same way.
Alex: Now the accuracy comparison. These systems predict different body representations, so a leaderboard could partly measure how outputs were converted.
Sam: That's a major concern here. Person boxes, camera focal lengths, joint definitions, and body-model conversions can all move the reported error. The authors re-evaluated eleven competing methods under a mostly matched protocol. For systems using their body rig, they applied Meta's official conversion to the common comparison model.
Alex: You said mostly matched. What exception should we keep in mind?
Sam: NLF used its own person detector, following its published setup, rather than the benchmark person boxes. The authors checked that changing those boxes had only a small effect for NLF. They also report whether their reproduced scores agree with each competitor's paper, rather than treating every published number as directly interchangeable.
Alex: Give us one accuracy result we can actually interpret, rather than a string of benchmark scores.
Sam: On EMDB, one held-out benchmark, InstantHMR's medium model has an aligned average joint-position error of about forty-nine millimetres. The smaller NLF scores about forty-five millimetres under this evaluation. Lower is better, but this metric first aligns the reconstructed body to the reference, so it isn't an absolute positioning guarantee.
Alex: A modest accuracy loss there buys a much shorter wait on the tested accelerator. How broad is the evidence behind that comparison?
Sam: They evaluate across three body-reconstruction benchmarks and measure latency across desktop and phone targets. On the tested phone accelerators, InstantHMR is the most accurate method among those meeting their real-time threshold of thirty frames per second. My reading is that the model-level trade-off is well supported, especially because they rerun competitors rather than just borrowing headline scores.
Alex: But model-level timing is not camera-to-feedback timing. What does the stopwatch leave out?
Sam: Detection and cropping are excluded for every method. The timing starts with the person crop and includes producing the posed surface. It doesn't include the official body-model conversion used for benchmark scoring. So this supports a fast reconstruction component, not a measured end-to-end application.
Alex: For movement feedback, I'd also care whether the body flickers between frames. Did they test that?
Sam: No. The authors report no temporal metric for jitter or acceleration error. Each frame is estimated independently, and evaluating stability, including with their demo's smoothing filter, is future work. That's the most important gap if your application depends on smooth motion rather than isolated poses.
Alex: What else should stop a researcher from treating this as ready for any phone or product?
Sam: Accelerator measurements cover Qualcomm-equipped Android phones, not Apple's Neural Engine. Hands receive no dedicated close-up processing. And the main weights are research-only because some training images have restrictive licences. A commercially usable variant exists, but the authors report substantially worse accuracy with its smaller training corpus.
Alex: With those boundaries, who should read the full document, and where should they begin?
Sam: Researchers building mobile body estimators should start with the evaluation protocol, then the explanation of why InstantHMR is fast on the phone accelerator. Read the limitations before planning a live application. For everyone else, remember this: a model isn't fast on a phone just because it's fast on a desktop. The operations have to fit the chip.
Alex: Keep that distinction in mind when choosing your next baseline.