Shreyas Rajesh, Kartik Sharma, Tonmoy Monsoor, Mehmet Yigit Turali, Richard Idro, Juliana Kayaga, Robert Sebunya, Tracy Tushabe Namata, Jessica Nichole Pasqua, Vwani Roychowdhury, Rajarshi Mazumder
5 min
Abstract
Specialist epilepsy expertise is scarce in resource-constrained settings, making LLM-based decision support attractive for frontline clinicians managing longitudinal treatment. Such systems must adapt to local prescribing practice and know when to defer. We study this problem in Ugandan pediatric epilepsy care, predicting anti-seizure medication regimens from longitudinal unstructured clinic notes. Standard prompting achieves non-trivial agreement with physician prescriptions, but neurologist review shows that many errors reflect distribution-miscalibrated prescribing defaults rather than failures to parse the local record. We introduce MANANA, a non-parametric prompt-learning framework that learns local prescribing guidance from a small patient-level training set. MANANA converts observed prescription errors into auditable prompt memories, instantiated in single-agent and multi-agent variants, and improves over classical ML models, direct LLM prompting, and prompt-optimization baselines across two independently collected Ugandan cohorts. We further propose Bayesian prompt averaging, which converts the learned prompt trajectory into prescription likelihoods and an uncertainty-based deferral signal. On the independently collected held-out cohort, this improves visit-level top-3 prescription accuracy by 4-8 percentage points over prompt-optimization baselines and enables selective prediction: the system can auto-handle the most confident half of cases at 95% precision, or the most confident quarter at 99% precision, while deferring lower-confidence cases for specialist review.
Sam: That's a valid concern, and the authors address it directly. The key is that the Architect separates two things that might seem the same but aren't: recognising that an error happened, and deciding to permanently commit a new rule based on it. A rule only gets written into the notebook if the same clinical signal recurs across different patients in different situations. That repetition requirement is what stops the model from over-learning on a single unusual case and becoming brittle.
Alex: So it's a filter for noise. It's waiting for a pattern to prove itself before trusting it.
Sam: Precisely. And that patience is what makes the confidence scores meaningful. When it does give an answer, it can also tell the doctor how sure it is—which is the foundation of safe, human-in-the-loop care. The doctor always knows whether they're looking at a high-confidence suggestion or one that warrants a second look.
Alex: That's a more honest kind of AI than one that just gives an answer and acts certain. It knows its own limits.
Sam: That's the goal. But it's worth being clear about what those limits actually are. The system learns from physician-prescribed regimens, which it treats as its ground truth. The problem is that those prescriptions are often shaped by local constraints—drug shortages, limited testing equipment—rather than purely ideal medical outcomes. So the model is effectively learning to mimic local practice. If the local standard of care is constrained by resources, the AI will naturally reflect those same constraints.
Alex: So it's not optimising medicine in some global, textbook sense. It's becoming an expert at navigating the specific reality of that particular clinic?
Sam: Exactly. The researchers acknowledge this as a fundamental trade-off. By grounding the system in real-world clinic data, they ensure the advice is practical and immediately actionable for the doctors on the ground—even if it doesn't match what you'd see in a well-resourced hospital. The paper suggests this framework could extend to other medical domains in similar settings, creating a network of self-adapting assistants that improve without requiring significant computing infrastructure.
Alex: It's a grounded approach. It's not waiting for a perfect system—it's building something useful for the clinic as it actually exists today. Thanks for walking me through it, Sam. And thanks to our listeners for joining us for this look at medical AI in resource-constrained settings. Thanks for listening to ResearchPod.