ResearchPod Summary
As generative AI (GenAI) becomes ubiquitous in higher education, the binary question of whether students use AI has become insufficient for understanding its impact on learning and academic integrity. This study sought to move beyond simple usage metrics by developing a theoretically grounded instrument—the GenAI Reliance Types Scale (GenAI-RTS)—to measure how students rely on AI during the extended, iterative process of academic writing.
The researchers employed a deductive, theory-driven approach to define four primary types of reliance: Strategic, Instrumental, Dependent, and Dialogic. They developed a 20-item scale and validated it using a multi-source framework, including a survey of 382 undergraduates at a U.S. Minority-Serving Institution and qualitative interviews with 14 students. The validation process included confirmatory factor analysis (CFA) to test competing models, Rasch analysis to refine the response format, and tests for scalar measurement invariance across gender, first-generation status, and academic discipline.
The study confirmed a five-factor structure for the GenAI-RTS. The initial concept of 'Strategic Reliance' was empirically split into two distinct facets: Deliberate Use and Critical Evaluation. These, along with Instrumental, Dependent, and Dialogic reliance, form a robust framework for profiling student behavior. The scale demonstrated acceptable to good reliability and, notably, achieved scalar measurement invariance across key demographic groups, suggesting that the instrument can be used to make fair comparisons between different student populations. Furthermore, the study found that Strategic reliance is positively correlated with AI literacy, and the different reliance profiles effectively differentiate students based on their writing processes and outcomes.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at a new way to understand how students use artificial intelligence in their writing.
Sam: That sounds timely. I've heard plenty of debates about whether students should be using AI, but it sounds like you're saying that question is already outdated.
Alex: Exactly. Because AI use is now nearly universal, asking "did you use it?" doesn't tell us anything useful about a student's learning or their integrity.
Sam: So this paper argues that the real issue isn't the fact of use, but the nature of that reliance?
Alex: Precisely. The authors developed the GenAI-RTS—the Generative AI Reliance Types Scale—to map exactly how students delegate their cognitive work to these tools.
Sam: And the core problem is that right now, we can't tell the difference between a student using AI as a brainstorming partner and one who is simply outsourcing their thinking entirely.
Alex: That's the practical challenge. The current approach—treating all AI use as either "okay" or "misconduct"—fails to capture how students actually work.
Sam: So how does this study solve that? Is it just a survey, or is there a deeper mechanism behind these "types" of reliance?
Alex: There's a real theoretical foundation. Instead of just observing student behaviour and guessing, the researchers built the scale on established ideas about how people learn and think. Specifically, they drew on what's called metacognition.
Sam: For listeners who haven't heard that term—metacognition is basically thinking about your own thinking, right? Like when you pause mid-essay to ask yourself whether your argument actually makes sense?
Alex: Exactly. It's the ability to plan, monitor, and evaluate your own work. And the researchers used that framework to define four distinct ways students rely on AI—distinguished by two things: how much control the student keeps over their own thinking, and how much of the intellectual work they hand off to the machine.
Sam: So it's a spectrum. If I'm using AI to check my grammar, I'm still in the driver's seat. If I'm asking it to write my entire essay, I've handed over the wheel.
This instrument provides educators and researchers with a standardized tool to move past the 'use vs. non-use' debate. By identifying specific reliance profiles, institutions can better design targeted AI literacy interventions, assess the impact of AI on cognitive development, and address potential equity gaps in how different student groups engage with these technologies.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.
Alex: Right. They call the thoughtful, controlled end of that spectrum "Strategic" and "Dialogic" reliance. The wholesale outsourcing they call "Dependent" reliance.
Sam: How did they prove their four-type model was actually the right one? Couldn't you carve up student behaviour in lots of different ways?
Alex: That's exactly the right question. They compared six different mathematical structures against their student data to see which one fit best. And the winner required splitting what they'd initially called "Strategic" use into two separate parts: planning and evaluating.
Sam: Why split them?
Alex: Because the data showed they're genuinely different skills. Students often evaluate AI output—checking whether what it produced is actually good—far more than they use AI to plan their work in the first place. Forcing those two behaviours into one category made the model less accurate.
Sam: That's a bit like assuming a chef who's great at tasting and adjusting a sauce must also be great at planning the menu. They're related skills, but not the same one.
Alex: That's a useful way to put it. They also ran something called Rasch Analysis on the individual survey questions—think of it as checking that each question in the survey is pulling its weight fairly, like making sure every hurdle in a race is set to the right height.
Sam: So they're not just testing students—they're testing the test itself.
Alex: Precisely. And they took one more step: they checked whether the survey measured the same thing consistently across different groups of students—different genders, different academic backgrounds, different majors. That process is called scalar invariance testing, and it's essentially a fairness check.
Sam: That matters a lot. A tool that works well for one group of students but gives skewed results for another isn't really useful for educators trying to make fair judgements.
Alex: Exactly. Without that check, you might end up with a scale that looks reliable on average but quietly misrepresents certain students. By confirming it holds up across groups, the researchers give educators something they can actually trust.
Sam: So the shift this paper is proposing is from a yes-or-no question—"did you use AI?"—to something much more informative: "how are you relying on it, and what does that tell us about your learning?"
Alex: That's the core of it. It gives educators a shared language—a way to distinguish between a student who is genuinely engaging with AI as a thinking tool and one who has stepped back from the thinking altogether. And that distinction, the paper argues, is where the real educational conversation needs to happen.
Sam: It also reframes the integrity question. Rather than treating AI use as inherently suspicious, it opens up the possibility of actually supporting students who are struggling—before a problem becomes a crisis.
Alex: That's well put. The goal isn't surveillance. It's understanding. And that's a meaningfully different starting point for both educators and students.
Sam: Thanks for walking us through this one. It's a good reminder that the tools change faster than our frameworks for understanding them—and that building better frameworks is genuinely useful work.
Alex: It is. Thanks for listening to ResearchPod.