Purpose: Automated medical image-based prediction of clinical outcomes, such as overall survival (OS), has great potential in improving patient prognostics and personalized treatment planning. We developed a deep regression framework using tissue-wise FDG-PET/CT projections as input, along with a temporal input representing a scalar time horizon (in days) to predict OS in patients with Non-Small Cell Lung Cancer (NSCLC). Methods: The proposed framework employed a ResNet-50 backbone to process input images and generate corresponding image embeddings. The embeddings were then combined with temporal data to produce OS probabilities as a function of time, effectively parameterizing the predictions based on time. The overall framework was developed using the U-CAN cohort (n = 556) and evaluated by comparing with a baseline method on the test set (n = 292). The baseline utilized the ResNet-50 architecture, processing only the images as input and providing OS predictions at pre-specified intervals, such as 2- or 5-year. Results: The incorporation of temporal data with image embeddings demonstrated an advantage in predicting OS, outperforming the baseline method with an improvement in AUC of 4.3%. The proposed model using clinical + IDP features achieved strong performance, and an ensemble of imaging and clinical + IDP models achieved the best overall performance (0.788), highlighting the complementary value of multimodal inputs. The proposed method also enabled risk stratification of patients into distinct categories (high vs low risk). Heat maps from the saliency analysis highlighted tumor regions as key structures for the prediction. Conclusion: Our method provided an automated framework for predicting OS as a function of time and demonstrates the potential of combining imaging and tabular data for improved survival prediction.
Alex: Welcome to another episode of ResearchPod.
Sam: Today we're looking at a study titled "Time-driven Survival Analysis from FDG-PET/CT in Non-Small Cell Lung Cancer." The central puzzle is predicting survival for lung cancer patients at any time after diagnosis, rather than fixed points like two or five years.
Alex: So older methods give just snapshots—like checking if someone reaches exactly two years—and that misses details?
Sam: Yes. Fixed predictions waste data from patients who leave studies early or die from other causes, called censoring. They also treat all deaths the same, ignoring how risk changes over time. This paper models survival as a smooth curve you can check at any time, like sliding a marker along a timeline.
Alex: That sounds more useful for planning treatments. How do they turn scans into predictions that handle any time?
Sam: The scans are FDG-PET/CT images. PET lights up active cancer spots by tracking a sugar-like tracer, while CT gives body maps. They slice these 3D scans into 2D views by tissue type—like bone or tumor—using density to simplify for computers. A network pulls key patterns from those views and combines them with a time number in days. It trains to predict if the patient is alive at that exact moment, learning the full survival shape without fixed checkpoints.
Alex: So one model adapts to any time you pick, instead of separate ones for each year?
Sam: Right. Tested on 556 patients, it beat a baseline of fixed-time image models on a key accuracy measure. It also sorts patients into risk groups more reliably. Heat maps showed it focuses on tumor spots, which fits clinical expectations.
Alex: Tumor focus makes sense. Inside the model, how does it blend scan patterns with a specific time?
Sam: They feed simplified images into a pre-trained network—like eyes scanning a crowded photo for shapes layer by layer. It condenses the image into a summary of features, such as tumor shapes or activity levels. A simple network turns the time—say, 550 days—into a matching summary. They multiply these summaries element by element, like a dimmer switch tweaking room lights. A final step outputs survival odds at that time.
Alex: To learn across all times, how do they train without biasing toward certain periods?
Sam: For living patients, they pick six random times from scan to last check-in, labeling them alive. For those who died, six before death as alive, twelve after as deceased. This creates balanced training points with variety each round. Random sampling beat uniform picks for real-world performance, building a full survival curve.
Alex: Like sampling clips from a patient's timeline to learn the whole story. How does it compare to other methods doctors use, like DeepHit or DeepSurv?
Sam: DeepHit breaks time into steps like calendar months, handling competing risks. DeepSurv updates the Cox method, where risk factors multiply a baseline death rate, using networks for flexibility. This method beat both on average accuracy across time points, using images or clinical data.
Alex: It beats them without clinical extras. What about training speed?
Sam: One network trains much faster than ten separate ones. It resists overfitting with varied samples. Adding tumor-specific image views boosted results most.
Alex: For risk groups, they used that area score over time?
Sam: Yes, AUSPC totals predicted survival odds over five years—higher means lower risk, like more area under a steady curve. K-means clustering split patients into high and low risk, with clear survival differences on Kaplan-Meier plots, especially by tumor stage. Ensembling images with clinical data hit the highest scores.
Alex: Limits like small subgroup sizes?
Sam: Yes, the paper cautions on that, preserving rigor. It also notes a slight underestimation of long-term survival on plots, and reliance on one hospital's data.
Alex: Why 2D views instead of full 3D?
Sam: Full 3D risks overfitting—like cramming quirks for one test instead of general rules. 2D tissue views cut complexity while keeping essentials for better generalization.
Alex: Images with clinical data gave the edge because scans catch tumor details notes might miss.
Sam: Exactly. Combined, they yielded top scores. With cautions on compute needs and single-site data, it suggests a solid tool for flexible forecasts from routine scans.
Alex: Pulling it together, this single model offers adjustable survival odds from scans, outperforming fixed-time setups and others, with tumor-focused insights and clear risk splits.
Sam: Yes. It avoids wasting censored data and captures risk changes—a meaningful step for personalized planning. The code is public for further work.
Alex: This makes survival estimates more practical for doctors. Thanks for listening to ResearchPod.