From Dyad to Triad: Eliciting XAI Requirements in Stroke Rehabilitation

As brain-computer interfaces (BCIs) and robot-assisted therapies become more common in stroke rehabilitation, they shift the traditional clinician-patient dyad into a technology-mediated triad. This transition creates a critical need for explainable AI (XAI) that helps patients understand system feedback, calibrate trust, and maintain motivation. However, standard design methods often exclude stroke survivors—particularly those with aphasia or cognitive impairments—because they assume participants can discuss abstract algorithmic concepts.

To address this, the authors developed a video-based scaffolding protocol. In a formative study, three stroke survivors and their caregivers watched seven video scenarios depicting common rehabilitation dilemmas (e.g., inconsistent system success, data privacy, or conflicting feedback). Facilitators used four specific techniques to help participants articulate their needs: analogical bridging (using familiar metaphors like mobile signals), projective personas (depersonalizing sensitive topics), binary forcing (reducing cognitive load), and extended response time.

The protocol successfully surfaced nuanced and often conflicting XAI requirements that would likely remain hidden under standard elicitation methods. For instance, participants showed clear, divergent preferences regarding information granularity—some preferred minimal summaries while others demanded detailed, task-specific breakdowns.

Furthermore, the study revealed that trust in AI is not monolithic; some participants prioritized clinical authority, while others favored the perceived objectivity of machine-generated scores. Crucially, the researchers performed a reflexive analysis of their own facilitation, identifying three systematic risks: normative bias (where facilitators favor complex AI features over simple ones), hypothesis confirmation bias, and the presence effect (where the facilitator's presence inadvertently shapes participant responses).

This work argues that eliciting patient-facing XAI requirements is a necessary prerequisite for designing trustworthy human-machine systems, rather than an optional preliminary step. By treating facilitator-participant interactions as data, the study provides a reusable methodological framework that helps researchers balance the need for scaffolding with the risk of imposing their own assumptions on vulnerable populations. This approach is particularly vital for resource-constrained settings where unsupervised, technology-mediated rehabilitation is becoming a necessity.