Hamayoon Behmanush, Freshta Akhtari, Ingmar Weber, Vikram Kamath Cannanure
10 min
This paper tackles a profound challenge: how can generative AI (GenAI) serve as a safe, accountable learning companion for women in Afghanistan who have been banned from formal education since 2021? Through a remote participatory design (PD) study with 20 Afghan women—recruited via a survey of 140 and partnered with organizations like Code to Inspire—the authors uncover needs for GenAI that acts not just as a tutor, but as a peer, mentor, and career guide. These women rely on unstable mobile connections, shared devices, and self-directed online learning amid household duties, restricted mobility, and surveillance risks. The study uses storyboards to envision GenAI futures, revealing design directions that prioritize safety, accountability, and empowerment. Remarkably, the PD process itself boosted participants' aspirations (p=.01), agency (p=.01), and perceived opportunities (p=.03), showing AI envisioning can drive real change.
Formal education's absence strips away peers, mentors, and structure. Women turn to GenAI for companionship under constraints: it's 'always available' for programming help, soft skills, and employability guidance. Unlike passive info sources, they want GenAI as a virtual learning space—simulating discussions, career paths, and motivation. But realism matters: support must align with local opportunities (e.g., remote freelancing), avoiding mismatched promises like 'become a CEO' in a gender-restrictive context. Storyboards tested deployment modes (e.g., voice vs. text, app vs. messaging), favoring low-visibility, anonymous interactions to evade household surveillance.
Safety-first design is non-negotiable. Participants fear data leaks revealing learning (taboo for women), device monitoring by family/authorities, and exposure from voice chats. Key concerns: direct-answer reliance creates 'illusion of progress' without deep learning; culturally unsafe advice (e.g., ignoring household roles); and privacy in shared spaces. Solutions emphasize user control: opt-in sharing, ephemeral chats, and 'safety-first interactions' like indirect hints over answers. Household constraints demand bite-sized, flexible support fitting around chores, with Persian localization for accessibility.
GenAI risks undermining learning with quick fixes, bypassing reasoning. Participants want 'pedagogically aligned' help: scaffolding questions, reasoning traces, and progress checks to build genuine skills. Accountability means context-grounded responses—tailored to Afghanistan's realities, not generic Western advice. Co-design surfaced tensions: enthusiasm for AI companions vs. fears of dependency or deception.
Translating findings into actionable guidelines:
In gender-restrictive and surveilled contexts, where access to formal education may be restricted for women, pursuing education involves safety and privacy risks. When women are excluded from schools and universities, they often turn to online self-learning and generative AI (GenAI) to pursue their educational and career aspirations. However, we know little about what safe and accountable GenAI support is required in the context of surveillance, household responsibilities, and the absence of learning communities. We present a remote participatory design study with 20 women in Afghanistan, informed by a recruitment survey (n = 140), examining how participants envision GenAI for learning and employability. Participants describe using GenAI less as an information source and more as an always-available peer, mentor, and source of career guidance that helps compensate for the absence of learning communities. At the same time, they emphasize that this companionship is constrained by privacy and surveillance risks, contextually unrealistic and culturally unsafe support, and direct-answer interactions that can undermine learning by creating an illusion of progress. Beyond eliciting requirements, envisioning the future with GenAI through participatory design was positively associated with significant increases in participants' aspirations (p=.01), perceived agency (p=.01), and perceived avenues (p=.03). These outcomes show that accountable and safe GenAI is not only about harm reduction but can also actively enable women to imagine and pursue viable learning and employment futures. Building on this, we translate participants' proposals into accountability-focused design directions that center on safety-first interaction and user control, context-grounded support under constrained resources, and offer pedagogically aligned assistance that supports genuine learning rather than quick answers.
Alex: So these sketches acted like prototypes to test ideas safely, without building real software yet?
Sam: Precisely. Participants critiqued them element by element—retaining safe parts, rejecting unsafe ones, and proposing changes rooted in their realities, such as outputs that stay context-specific to avoid suspicious suggestions. This surfaced requirements for exposure control: features letting users hide traces quickly and predictably, prioritizing safety over full accuracy.
Alex: And did this hands-on critiquing change how they viewed their own futures?
Sam: Yes—the pre- and post-session surveys used an aspirations scale to measure shifts. After envisioning these safe AI companions, participants reported a statistically significant increase in perceived agency for education and more viable paths to jobs. The paper suggests co-designing accountable tools can lift aspirations by making tech feel like a reliable, hidden ally.
Alex: That connects the design process right back to real empowerment... without the risks. What about how the AI would actually teach, once built?
Sam: They emphasized step-by-step guidance over instant answers, like a tutor breaking down coding problems into small, reasoned chunks that build skills. For fragmented schedules and spotty connections, short resumable sessions—around 10 to 15 minutes—fit their lives, progressing in their preferred language from basics to advanced. This ensures genuine learning, not just quick fixes, in surveilled settings.
Alex: Okay, so the teaching fits their lives perfectly. But how exactly did they picture this AI companion stepping in for things like missing classmates or teachers?
Sam: Participants saw it filling gaps left by no in-person classes or peers—acting like a study buddy who's always there. They described it as sitting right beside them during sessions, available anytime on a phone or laptop, even updating itself when internet returns. This peer-like presence provides constant company without schedules or travel, making solo learning feel less isolating.
Alex: Like a friend who never leaves your side for homework... got it. And beyond just being there, what about guiding them through tough stuff?
Sam: Yes—they wanted mentor-like help too, patiently breaking down hard tasks like fixing code errors or planning algorithms into simple steps. It would explain ideas clearly, maybe even in voice if needed, and give ongoing feedback on strengths and weak spots over time. This builds real skills through summaries of progress, helping them adjust without a real teacher.
Alex: That sounds structured, almost like a coach tracking your game. How did they tie this to job skills, since self-study alone misses practice?
Sam: They linked it to employability, using the AI for soft skills like time management or email writing that classrooms usually teach. Role-plays could simulate freelancer talks on sites like Upwork, with tips on proposals tailored to real platforms. But all this hinged on safety—constraints like shared phones and family watching meant features for quick trace deletion and anonymous access came first.
Alex: Right... so the companionship only works if it's invisible to surveillance. That exposure control makes the whole vision reliable.
Sam: Precisely. The paper suggests this envisioning not only shaped accountable designs but pointed to GenAI's potential as a safe partial stand-in for lost communities, prioritizing control in restrictive settings.
Alex: So envisioning these companions highlights GenAI's role in replacing lost communities... but only if it handles the real-world limits like surveillance and mismatched advice?
Sam: That's right. Participants stressed privacy risks on shared phones, where even small traces—like chat logs—could alert family or authorities under strict household monitoring. They described everyday dangers, such as AI suggesting a dating app project or studying in a public cafe, which clashes with local norms and could spark family tension if seen.
Alex: Those examples make the risks concrete—like the AI giving advice that sounds helpful but actually endangers them. How else did current tools fall short in fitting their lives?
Sam: Beyond surveillance, there's contextual mismatch: AI outputs often ignore local job markets, suggesting skills for wealthier places, or code needing powerful computers they don't have. Everyday tips—like mixed-gender study spots—feel unrealistic for women staying home. Pedagogically, direct answers create an illusion of progress; users get solutions without building thinking skills.
Alex: Right, so it's not just safety, but the AI needing to match their exact circumstances to be useful. And the designs they proposed tackle these head-on?
Sam: Yes—one key idea is AI-facilitated virtual spaces, like anonymous chat rooms for same-level learners to share experiences and practice collaboration. This fills gaps in soft skills and peer support missing from in-person classes, with the AI moderating safely. For interactions, they prioritized safety-first features: quick deletion of all traces, no registration needed, and outputs that respect non-negotiable boundaries.
Alex: Virtual peers without exposure sounds like a smart workaround for isolation. Does the paper connect this back to building real trust?
Sam: It frames trust as warranted reliance—predictable behavior learners can anticipate, minimizing harm from privacy slips or mismatches. In these settings, accountability means exposure control first, letting women rely on the tool without fear. The study suggests this participatory approach uncovers these needs and fosters designs enabling safe, skill-building companionship.
Alex: So pulling this together, the study reframes what makes GenAI useful here—not just correct answers, but support you can rely on safely under real constraints.
Sam: Exactly. Participants saw GenAI less as a quick info source and more as a partial replacement for lost peers and mentors—providing ongoing study routines, feedback over time, and links to work opportunities. But this companionship only works if it manages visibility tightly, since more interaction means more potential traces on shared devices.
Alex: That tension between helpful presence and exposure risk seems central. How does the paper tie accountability back to all this?
Sam: In these high-risk settings, accountability goes beyond fairness or transparency—it's about exposure control. That means building in anonymous access, rapid deletion of traces, and outputs that stick to safe boundaries, so learners predict and manage what others might see or infer. The findings stress this as a core condition for safe use.
Alex: Makes sense—like designing the tool so it doesn't leave fingerprints. And localization fits right into that, matching everyday limits.
Sam: Yes, localization isn't just translation; it's aligning support to fragmented time, weak internet, low-end devices, and realistic jobs like remote freelancing. Pedagogically, it favors step-by-step reasoning and micro-sessions over instant fixes.
Alex: So evaluating GenAI shifts to whether it sustains routines without harm. What about the bigger picture from the participatory sessions themselves?
Sam: The sessions did more than gather ideas—they created space for reflection on constrained futures, leading to notable pre-post shifts in perceived agency and opportunity paths. With a small group of 20, these are short-term associations, not proof of lasting causal change. Still, it suggests participatory methods can make aspirations feel more concrete by centering users' realities in design.
Alex: Fair point on the sample size—keeps it grounded. Overall, this points to scalable tools empowering learning in restrictive places.
Sam: Precisely. Accountable GenAI could help millions build remote skills for global freelancing, prioritizing warranted reliance over raw output quality. The paper frames it as socio-technical support: safe, fitting, and future-oriented.
Alex: That's a clear, practical contribution. Thanks, Sam—this has been a thoughtful look at designing safe GenAI companions for women excluded from education. Thanks for listening to ResearchPod.