Specialist epilepsy expertise is scarce in resource-constrained settings, making LLM-based decision support attractive for frontline clinicians managing longitudinal treatment. Such systems must adapt to local prescribing practice and know when to defer. We study this problem in Ugandan pediatric epilepsy care, predicting anti-seizure medication regimens from longitudinal unstructured clinic notes. Standard prompting achieves non-trivial agreement with physician prescriptions, but neurologist review shows that many errors reflect distribution-miscalibrated prescribing defaults rather than failures to parse the local record. We introduce MANANA, a non-parametric prompt-learning framework that learns local prescribing guidance from a small patient-level training set. MANANA converts observed prescription errors into auditable prompt memories, instantiated in single-agent and multi-agent variants, and improves over classical ML models, direct LLM prompting, and prompt-optimization baselines across two independently collected Ugandan cohorts. We further propose Bayesian prompt averaging, which converts the learned prompt trajectory into prescription likelihoods and an uncertainty-based deferral signal. On the independently collected held-out cohort, this improves visit-level top-3 prescription accuracy by 4-8 percentage points over prompt-optimization baselines and enables selective prediction: the system can auto-handle the most confident half of cases at 95% precision, or the most confident quarter at 99% precision, while deferring lower-confidence cases for specialist review.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a new way to help doctors in places where specialist help is hard to find.
Sam: We're discussing a paper on medical AI. The challenge is deploying reliable tools in clinics—like those in rural Uganda—where specialists are scarce and local practices differ significantly from those in high-income countries.
Alex: So this paper is asking how we can make AI tools that actually work for local doctors, without needing to constantly retrain them from scratch?
Sam: Exactly. Retraining an AI model is expensive and technically demanding—most rural clinics simply don't have the computing power or the engineering staff to do it. So this research introduces a different approach, a system called MANANA. Rather than rewriting the AI's core programming every time it encounters something new, MANANA treats the AI's knowledge like an evolving notebook. The AI reads that notebook before making any decision, and new rules only get added to it once the same pattern has been seen repeating across multiple different patients.
Alex: So instead of changing the "brain" of the AI, it's just adding notes to a shared notebook that the AI consults before it makes a decision?
Sam: That's a perfect way to put it. The system has three working parts. First, a Predictor, which suggests medications. Second, an Inspector, which checks those suggestions against what a real doctor actually prescribed. And third, an Architect, which decides which lessons from that comparison are genuinely worth keeping for the future.
Alex: And how does it know when to trust itself? If a case is really tricky, does it just guess, or does it know when it's out of its depth?
Sam: That's the second key part of their approach, which they call Bayesian Prompt Averaging. Think of it like consulting a panel of experts who've each been watching the AI learn at different points in time. The system looks back at its own notebook at several different stages of its development. If all those versions agree on a treatment, the system is confident. If they give different answers, it recognises that it's uncertain and flags the case for a human doctor.
Alex: Oh—so it handles the routine, clear-cut cases on its own, and passes the confusing ones up to a human specialist?
Sam: Exactly. By doing this, the system can manage a meaningful portion of cases with high precision, effectively acting as a filter. It reserves the specialist's limited time for the situations where the AI is genuinely unsure—which is exactly where human judgement matters most.
Alex: But wait—doesn't that selective approach risk the system only ever learning from a narrow slice of cases? If it keeps deferring the hard ones, does it ever get better at them?
Sam: That's a valid concern, and the authors address it directly. The key is that the Architect separates two things that might seem the same but aren't: recognising that an error happened, and deciding to permanently commit a new rule based on it. A rule only gets written into the notebook if the same clinical signal recurs across different patients in different situations. That repetition requirement is what stops the model from over-learning on a single unusual case and becoming brittle.
Alex: So it's a filter for noise. It's waiting for a pattern to prove itself before trusting it.
Sam: Precisely. And that patience is what makes the confidence scores meaningful. When it does give an answer, it can also tell the doctor how sure it is—which is the foundation of safe, human-in-the-loop care. The doctor always knows whether they're looking at a high-confidence suggestion or one that warrants a second look.
Alex: That's a more honest kind of AI than one that just gives an answer and acts certain. It knows its own limits.
Sam: That's the goal. But it's worth being clear about what those limits actually are. The system learns from physician-prescribed regimens, which it treats as its ground truth. The problem is that those prescriptions are often shaped by local constraints—drug shortages, limited testing equipment—rather than purely ideal medical outcomes. So the model is effectively learning to mimic local practice. If the local standard of care is constrained by resources, the AI will naturally reflect those same constraints.
Alex: So it's not optimising medicine in some global, textbook sense. It's becoming an expert at navigating the specific reality of that particular clinic?
Sam: Exactly. The researchers acknowledge this as a fundamental trade-off. By grounding the system in real-world clinic data, they ensure the advice is practical and immediately actionable for the doctors on the ground—even if it doesn't match what you'd see in a well-resourced hospital. The paper suggests this framework could extend to other medical domains in similar settings, creating a network of self-adapting assistants that improve without requiring significant computing infrastructure.
Alex: It's a grounded approach. It's not waiting for a perfect system—it's building something useful for the clinic as it actually exists today. Thanks for walking me through it, Sam. And thanks to our listeners for joining us for this look at medical AI in resource-constrained settings. Thanks for listening to ResearchPod.