Arvind Pillai, Dimitris Spathis, Fahim Kawsar, Mohammad Malekzadeh
6 min
Photoplethysmography (PPG) is a ubiquitous non-invasive sensing technique used in everything from clinical pulse oximeters to consumer smartwatches. Despite its widespread use, machine learning models for PPG are typically task-specific, data-hungry, and struggle to generalize across different devices or populations. The authors sought to address these limitations by developing PaPaGei, the first open-source foundation model specifically designed for PPG signals, aiming to provide a robust, general-purpose encoder for diverse health monitoring tasks.
The researchers pre-trained PaPaGei on over 57,000 hours of unlabeled PPG data sourced from three large public datasets (VitalDB, MIMIC-III, and MESA). They introduced a novel self-supervised learning (SSL) framework that incorporates domain-specific knowledge of PPG morphology. Unlike standard contrastive learning, which might treat all segments from a patient as identical, the PaPaGei-S variant uses a morphology-aware objective. This objective optimizes the model to recognize physiological markers like the systolic peak and dicrotic notch by predicting metrics such as the stress-induced Vascular Response Index (sVRI), Inflection Point Area (IPA), and Signal Quality Index (SQI).
PaPaGei demonstrates superior performance compared to state-of-the-art time-series foundation models and self-supervised benchmarks. Across 20 downstream tasks—including cardiovascular health, sleep disorder detection, pregnancy monitoring, and wellbeing assessment—the model improved classification metrics by 6.3% and regression metrics by 2.9% in at least 14 tasks. Notably, the model achieves these gains while being significantly more efficient, outperforming models that are up to 70 times larger. The authors also established a benchmark for evaluating model bias across different skin tones, providing a foundation for more equitable health monitoring.
By releasing PaPaGei as an open-source model, the authors provide a critical tool for researchers who lack the resources to train large-scale models from scratch. This work shifts the paradigm in physiological signal analysis from building bespoke models for every new task toward using a unified, pre-trained backbone that can be fine-tuned for a wide range of clinical and consumer health applications.
Photoplethysmography (PPG) is the leading non-invasive technique for monitoring biosignals and cardiovascular health, with widespread adoption in both clinical settings and consumer wearable devices. While machine learning models trained on PPG signals have shown promise, they tend to be task-specific and struggle with generalization. Current research is limited by the use of single-device datasets, insufficient exploration of out-of-domain generalization, and a lack of publicly available models, which hampers reproducibility. To address these limitations, we present PaPaGei, the first open foundation model for PPG signals. The model is pre-trained on over 57,000 hours of data, comprising 20 million unlabeled PPG segments from publicly available datasets. We introduce a novel representation learning approach that leverages domain knowledge of PPG signal morphology across individuals, enabling the capture of richer representations compared to traditional contrastive learning methods. We evaluate PaPaGei against state-of-the-art time-series foundation models and self-supervised learning benchmarks across 20 tasks from 10 diverse datasets, spanning cardiovascular health, sleep disorders, pregnancy monitoring, and wellbeing assessment. Our model demonstrates superior performance, improving classification and regression metrics by 6.3% and 2.9% respectively in at least 14 tasks. Notably, PaPaGei achieves these results while being more data- and parameter-efficient, outperforming models that are 70x larger. Beyond accuracy, we examine model robustness across different skin tones, establishing a benchmark for bias evaluation in future models. PaPaGei can serve as both a feature extractor and an encoder for multimodal models, opening up new opportunities for multimodal health monitoring.
Sam: And this is done without anyone manually labeling the data?
Alex: Right. This is what's called self-supervised learning. The model finds its own patterns without a human having to go through millions of recordings and tag each one. That matters enormously at this scale, because manually labeling wearable data would be practically impossible.
Sam: The paper also mentions something called a "Mixture of Experts" architecture. Can you unpack that?
Alex: Think of it like a hospital department rather than a single doctor. Instead of one model trying to handle every health metric at once, PAPAGEI has multiple internal specialists — each one trained to focus on a specific physical property of the pulse signal. When the model needs to assess something like how your blood vessels are responding to stress, it activates the relevant specialist rather than running everything through one overloaded system.
Sam: So it's more targeted, and presumably more efficient?
Alex: Precisely. It only activates what it needs for a given task, which keeps the computational load manageable. That efficiency matters a lot if you eventually want to run something like this on a wearable device rather than a distant server.
Sam: Which brings up an interesting point about privacy. If the model is small and efficient enough to run locally, you're not sending sensitive health data to the cloud?
Alex: That's one of the practical motivations, yes. A model that can run on the device itself avoids the privacy risks and battery costs of constantly transmitting your health data elsewhere. The paper frames parameter efficiency — meaning getting strong results with fewer computational resources — as a deliberate design goal, not just a side benefit.
Sam: And how does PAPAGEI actually perform compared to other models?
Alex: The paper reports that it achieves competitive or better results than models that are far larger — in some comparisons, up to seventy times larger in terms of parameters, which is roughly the measure of a model's size and complexity. The argument is that by baking in physiological knowledge from the start, you need less raw computational power to reach the same level of understanding.
Sam: That's a meaningful gap. Are there limitations the researchers are upfront about?
Alex: Yes, and they're worth taking seriously. The paper notes that the model's performance across different skin tones is still an open question. Pulse sensors work by shining light through the skin and measuring how much is absorbed by the blood. Skin pigmentation affects how that light behaves, which means a model trained predominantly on data from lighter-skinned individuals may not perform as reliably for everyone. The researchers flag this as a priority for future work.
Sam: That feels like a fundamental issue for any technology that's meant to be universal.
Alex: It is. And the paper is transparent about it. They point toward two approaches: gathering more data from diverse populations, and using a technique called domain adaptation, which is essentially fine-tuning the model to work better for specific groups. But they're clear that the current version isn't bias-free, and that clinical trust depends on being honest about that.
Sam: So where does this fit in the bigger picture? Is this just about heart rate, or is there a broader vision here?
Alex: The broader vision is to use PAPAGEI as a foundation — a reliable starting point that other systems can build on. The pulse signal is one of the most accessible windows into the body's physiology. If you have a model that genuinely understands it, you can start combining that with other data streams — sleep patterns, movement, temperature — to build a much more complete picture of someone's health over time.
Sam: A kind of integrated health portrait, rather than isolated readings.
Alex: That's the direction the field is moving. And what makes PAPAGEI notable in that context is the combination of things it gets right at once: it's open, meaning other researchers can build on it; it's efficient enough to be practical on real devices; and it's grounded in the actual physics of the signal rather than just pattern-matching on large datasets.
Sam: It's a measured but meaningful step. The skin tone limitation is a real constraint, but the underlying approach — learning the physics of the signal rather than memorizing patient identities — seems like a more principled foundation to build from.
Alex: That's a fair summary. The core shift is from asking "whose signal is this?" to asking "what is this signal telling us about the body?" That reframing is what gives the model its generality. Thanks for listening to ResearchPod.