Simran Saggu, Hirad Daneshvar, Reza Samavi, Paulo Pires, Roberto B Sassi, Thomas E Doyle, Judy Zhao, Ahmad Mauluddin, Laura Duncan
6 min
This study investigated whether advanced deep learning techniques could improve the prediction of 30-day emergency department (ED) revisits for children and youth receiving mental health outpatient services. The researchers compared a Graph Neural Network (GNN) against a Recurrent Neural Network (RNN) and several conventional baseline models, including XGBoost, decision trees, and logistic regression. The study utilized administrative electronic health record (EHR) data from McMaster Children's Hospital, focusing on patients aged 4–17 who had at least one ED visit between 2002 and 2021.
While traditional machine learning models often treat patient data as flat, independent rows, and RNNs focus primarily on the sequential order of events, GNNs offer a more sophisticated approach. By representing patient history as a graph—where nodes represent variables like diagnoses or services and edges represent the relationships between them—GNNs can capture complex, multidirectional dependencies. This allows the model to account for how various clinical factors, symptoms, and service interactions influence one another over time, potentially providing a more accurate picture of a patient's risk profile.
The GNN model demonstrated the best overall performance, outperforming the RNN by an F1-score increase of 0.0511 and the top-performing conventional model (XGBoost) by 0.0470. The improvements were particularly notable in recall and negative predictive value, suggesting the GNN is more effective at identifying patients who are at risk of returning to the ED. Despite these gains, the authors note that the models are not yet ready for clinical deployment, as the performance metrics have not reached the high thresholds required for reliable, real-world decision support.
Predicting ED revisits is a critical step toward early intervention and the efficient allocation of mental health resources. By identifying high-risk youth before they return to the ED, healthcare systems can implement targeted support strategies that prevent crises and improve patient outcomes. This study provides evidence that graph-based deep learning is a promising, albeit still developing, tool for navigating the complex dynamics of pediatric mental health service utilization.
BACKGROUND: The proportion of Canadian youth seeking mental health support from an emergency department (ED) has risen in recent years. As EDs typically address urgent mental health crises, revisiting an ED may represent unmet mental health needs. Accurate ED revisit prediction could aid early intervention and ensure efficient healthcare resource allocation. We examine the potential increased accuracy and performance of graph neural network (GNN) machine learning models compared to recurrent neural network (RNN), and baseline conventional machine learning and regression models for predicting ED revisit in electronic health record (EHR) data. METHODS: This study used EHR data for children and youth aged 4-17 seeking services at McMaster Children's Hospital's Child and Youth Mental Health Program outpatient service to develop and evaluate GNN and RNN models to predict whether a child/youth with an ED visit had an ED revisit within 30 days. GNN and RNN models were developed and compared against conventional baseline models. Model performance for GNN, RNN, XGBoost, decision tree and logistic regression models was evaluated using F1 scores. RESULTS: The GNN model outperformed the RNN model by an F1-score increase of 0.0511 and the best performing conventional machine learning model by an F1-score increase of 0.0470. Precision, recall, receiver operating characteristic (ROC) curves, and positive and negative predictive values showed that the GNN model performed the best, and the RNN model performed similarly to the XGBoost model. Performance increases were most noticeable for recall and negative predictive value than for precision and positive predictive value. CONCLUSIONS: This study demonstrates the improved accuracy and potential utility of GNN models in predicting ED revisits among children and youth, although model performance may not be sufficient for clinical implementation. Given the improvements in recall and negative predictive value, GNN models should be further explored to develop algorithms that can inform clinical decision-making in ways that facilitate targeted interventions, optimize resource allocation, and improve outcomes for children and youth.
Sam: And did the graph-based model actually score better?
Alex: It did. The GNN showed a meaningful improvement over the traditional sequential models. That said, the authors are careful to note that the current performance level may not yet be high enough for full clinical use. It is a promising result, but not a finished product.
Sam: There's another challenge here, isn't there? If only a small number of patients actually return to the emergency department, the computer might just learn to guess "no return" every time and still look accurate on paper.
Alex: You've identified a classic problem in this kind of research. If the "revisit" group is much smaller than the "no revisit" group, the model can cheat — it gets a decent score simply by defaulting to the majority answer. It never actually learns what a revisit looks like.
Sam: So how do they prevent that?
Alex: They use a technique called "undersampling." They take the small group of revisit cases and pair them with an equal-sized sample from the larger group. By balancing the two sides, they force the model to take both outcomes seriously and actually learn the warning signs, rather than taking the easy way out.
Sam: Like balancing the scales before you start weighing anything.
Alex: Exactly. It is a reminder that the output of any model depends entirely on how carefully you prepare the input.
Sam: There's another issue that comes up with these complex models, though. Even if it works, can a doctor actually understand why the computer flagged a particular patient?
Alex: That is one of the study's most significant limitations. GNNs are good at capturing complex, non-linear patterns — connections that no human would easily spot in a spreadsheet. But that complexity comes at a cost. You cannot easily extract a simple list of reasons for a prediction. The logic is buried deep inside the model. Researchers sometimes call this the "black box" problem.
Sam: So it's a genuine trade-off. Better accuracy, but less ability to explain the reasoning to the medical team.
Alex: That is the core tension. The authors acknowledge that making these models more transparent is a vital area for future work. A doctor needs to be able to trust a tool before they act on it.
Sam: What about the data itself — were there other gaps?
Alex: Yes. The dataset lacked certain social and demographic details that might influence why a young person returns to the emergency department — things like family circumstances or access to community support. The researchers suggest that future work should try to combine medical records with broader data sources.
Sam: Like school attendance records, or whether a family has access to mental health services in their area?
Alex: Precisely. Integrating that kind of non-medical information could give a much more complete picture of a child's situation — and potentially allow for support to be offered before a crisis even develops.
Sam: So the long-term goal isn't just to predict a return visit. It's to prevent the crisis from building in the first place.
Alex: That is the broader aim. By identifying those at risk early, hospitals could shift from reacting to emergencies toward proactive, community-based support. The research points in that direction, even if the tools are not yet ready for routine clinical use.
Sam: It sounds like a meaningful step — not a finished answer, but a more sophisticated way of thinking about patient history than older sequential methods allowed.
Alex: That is a fair assessment. The work highlights the potential of graph-based models to improve how we understand and allocate care for young patients. The path forward involves refining these models, improving their transparency, and eventually drawing on broader data to support better outcomes. Thanks for listening to ResearchPod.