We introduce xplainfi, an R package built on top of the mlr3 ecosystem for global, loss-based feature importance methods for machine learning models. Various feature importance methods exist in R, but significant gaps remain, particularly regarding conditional importance methods and associated statistical inference procedures. The package implements permutation feature importance, conditional feature importance, relative feature importance, leave-one-covariate-out, and generalizations thereof, and both marginal and conditional Shapley additive global importance methods. It provides a modular conditional sampling architecture based on Gaussian distributions, adversarial random forests, conditional inference trees, and knockoff-based samplers, which enable conditional importance analysis for continuous and mixed data. Statistical inference is available through multiple approaches, including variance-corrected confidence intervals and the conditional predictive impact framework. We demonstrate that xplainfi produces importance scores consistent with existing implementations across multiple simulation settings and learner types, while offering competitive runtime performance. The package is available on CRAN and provides researchers and practitioners with a comprehensive toolkit for feature importance analysis and model interpretation in R.
Alex: Welcome to another episode of ResearchPod.
Sam: This episode covers a research paper on xplainfi, an R package for calculating feature importance in machine learning models. The main challenge is measuring which data inputs truly drive predictions when those inputs are linked together in real data.
Alex: So usual ways of checking importance get confused by connected features?
Sam: Yes. Models predict outcomes—like disease risk—using features such as age, blood pressure, or cholesterol. When features correlate and change together, simple shuffling breaks those links and spreads credit misleadingly. xplainfi keeps links intact.
Alex: Like two cholesterol measures that rise together—basic tests might blame both equally, but only one carries the real signal.
Sam: Exactly. It works within mlr3, R's shared tools for models, offering marginal methods like shuffling and conditional ones that respect dependencies. This builds trust in black-box models by pinpointing true drivers.
Alex: Without that, models are risky for real decisions like medicine.
Sam: The paper shows xplainfi matches other tools on tests, adding confidence intervals for certainty. It uses mlr3's learners, tasks, and resampling so users focus on interpretation.
Alex: How does it handle linked features without breaking connections?
Sam: It tests a feature by making small, realistic changes while holding others steady. Picture an orchestra: mute one violin but keep the rest in harmony to hear its true role. Random scrambling ruins the sound. xplainfi's conditional perturbations draw fake values that fit realistically with other features—this is conditional feature importance, or CFI.
Alex: So for cholesterol, it samples new values matching the patient's other stats.
Sam: Yes. Basic permutation feature importance, or PFI, mixes values randomly, creating impossible pairs like high cholesterol with low blood pressure. xplainfi's samplers fix this: one assumes bell-shaped patterns like heights and weights; another maps dependencies with trees for mixed data; a third uses knockoffs that pair fakes for testing.
Alex: It picks the right sampler to avoid shuffle pitfalls.
Sam: It also refits models without one feature—leave-one-covariate-out, or LOCO—to measure performance drops. Leave-one-group-out handles teams. These take more time but give clean measures. Shapley-style methods average a feature's value across combinations, with conditional versions.
Alex: That orchestra analogy shows why harmony matters for diagnosis.
Sam: Users get reliable scores with inference like corrected tests on repeats. It unifies methods in mlr3, unlike tools limited to basics.
Alex: How does it fit daily R workflows compared to others?
Sam: mlr3 breaks tasks into blocks like Legos: data tasks, learners like random forests, measures like error rates, resampling for fair splits. Set once, then compute importance across repeats—consistent and simple.
Alex: No reinventing the wheel? How does it stand out?
Sam: It's most complete for PFI, CFI, LOCO, and Shapley—broader than cpi's knockoffs or permimp's trees. vimp adds intervals but ties to slower ensembles. In linked features where one truly predicts, xplainfi scored the key one four times higher than the pretender.
Alex: That separates real signals.
Sam: It handles ensembles and tuning seamlessly. Python tools lack R pipelines or inference.
Alex: How does it manage uncertainty across runs?
Sam: Model importance is for one trained model on fixed data. Learner importance pairs untrained models with resampling like repeated cross-validation, averaging fresh scores for stability.
Alex: Resampling smooths quirks from single splits.
Sam: Access per-split details: baseline, post-change loss, differences. For permutations, Nadeau-Bengio correction widens intervals for overlapping data—like honest error bars on a team's games with shared players.
Alex: Adjusting because splits aren't independent.
Sam: Yes. Unadjusted intervals hid variability; corrected ones matched reality. Knockoffs use t-tests with Benjamini-Hochberg for multiple testing.
Alex: P-values flag reliable features.
Sam: It works with any mlr3 learner or pipeline. Simulations show solid coverage.
Alex: A concrete example for groups?
Sam: Conditional predictive impact, or CPI, pairs knockoff fakes and compares losses for p-values, adjusted across features. LOCO retrains without one feature; LOGO for groups. In simulated data with linked x1 and x2, LOCO scored x1 high and x2 near zero.
Alex: Grouping avoids splitting credit.
Sam: WVIM flips to leave-in for subsets. Samplers scale to groups or single features—like realistic penguin masses matching bill and island.
Alex: From singles to teams, no hassle.
Sam: Benchmarks on correlated data and bike rentals matched iml, vip, fippy—consistent rankings and competitive speed.
Alex: Reliable without slowdowns.
Sam: xplainfi unifies methods in mlr3 for any model, focusing on interpretation amid correlations. Tied to mlr3—not tidymodels—but trades breadth for depth. Some samplers need more compute.
Alex: Choices fit data and budget.
Sam: It aids AutoML with checks for regulated fields or hidden drivers—a step for trustworthy explanations.
Alex: Fills a practical gap. Thanks for breaking it down, Sam. Thanks for listening to ResearchPod.