MuTSE: A Human-in-the-Loop Multi-use Text Simplification Evaluator | Rares-Alexandru Roscan et al. | ResearchPod

Transcript: MuTSE: A Human-in-the-Loop Multi-use Text Simplification Evaluator

Alex: Welcome to another episode of ResearchPod.

Sam: This paper introduces MuTSE, a web tool for evaluating text simplification. Teachers often rewrite tough reading materials into simpler words and shorter sentences for students building language skills. The central challenge is comparing AI outputs from different instructions and models to see which ones keep the original meaning while staying easy to read.

Alex: So it's hard to track how well each simplified version matches the original text?

Sam: Yes. Picture testing five sets of instructions across three AI models—that's fifteen versions of the same text. Sorting which ones keep the original ideas while staying simple overwhelms anyone without good tools.

Alex: And the problem is tracking how each simplified sentence connects to the original?

Sam: Precisely. MuTSE makes evaluation visual and straightforward. It runs all prompt-model combinations side by side in real time.

Alex: So it's like a dashboard for comparing AI simplifications easily?

Sam: Yes—a human-in-the-loop setup where you guide and check outputs interactively. It runs tasks in parallel, so the batch finishes about as fast as the slowest one. Think of it like cooking multiple dishes in one oven.

Alex: How does it match sentences between the original and the simplified versions?

Sam: Its semantic alignment engine matches sentences by meaning first—like checking if two descriptions point to the same idea, even with different words. It creates a numerical fingerprint for each sentence's essence using a model called SBERT. Cosine similarity then measures how close those fingerprints are, like angles between arrows. It falls back to word overlaps or position if needed.

Alex: But texts don't always line up one-to-one. Simplifications might split or merge sentences.

Sam: Right. They add a linearity bias, lambda or λ, which penalizes matches far apart in position. You adjust it from zero for pure meaning to two for strict order; default is 0.5. This avoids false links.

Alex: Does it show this visually?

Sam: Yes—the interface displays the original next to columns for each version. Hover a sentence, and matches highlight across all, with stats like word count or reading ease.

Alex: Reading ease measures text difficulty?

Sam: Yes, like Flesch-Kincaid, which estimates the U.S. school grade needed based on sentence length and word complexity. Or Flesch Reading Ease, from 0 hard to 100 easy. These appear automatically. Teachers use scales like the CEFR, from A1 for total beginners to C2 for experts, to target the right level.

MuTSE: A Human-in-the-Loop Multi-use Text Simplification Evaluator

Rares-Alexandru Roscan, Gabriel Petre, Adrian-Marius Dumitran, Angela-Liliana Dumitran

Apr 10, 2026·3:18·arxiv·Original Paper

Text SimplificationLarge Language Models (LLMs)Human-in-the-Loop EvaluationSemantic AlignmentP×M Evaluation Matrix

Listen

3 min

Briefing

Overview

MuTSE is an interactive web application that revolutionizes how we evaluate Large Language Models (LLMs) for text simplification. Text simplification takes complex writing and makes it easier to read—think turning a dense academic paper into something a middle-school student can grasp. This is vital for Intelligent Tutoring Systems (ITS) and language learning apps, where content must match a learner's proficiency level, like CEFR A2 (beginner) or B1 (intermediate). Traditional evaluation is messy: researchers run static scripts comparing prompts and models one-by-one, while teachers poke at chat interfaces. MuTSE fixes this with a human-in-the-loop system that runs multiple prompt-model combos in parallel, visualizes results side-by-side, and helps humans judge quality efficiently.

P×M Evaluation Matrix

Imagine testing 5 different prompts across 4 LLMs—that's 20 versions of a simplified text to compare. MuTSE's core is the P×M matrix (P for prompts, M for models), which generates all permutations concurrently and displays them in a grid. You pick your source text, set proficiency targets, choose local/cloud LLMs, and boom—real-time side-by-side views. This scales evaluation from tedious manual checks to systematic analysis, perfect for finding the best prompt-model pair for educational use.

Semantic Alignment Engine

A big pain in simplification eval is tracking how one original sentence morphs across versions. MuTSE's tiered semantic alignment engine solves this visually. It maps simplified sentences back to the source using embeddings (like sentence vectors), TF-IDF for word overlap, and a 'linearity bias' heuristic (λ) that penalizes non-sequential rearrangements. Tiered means it starts coarse (full paragraph matching), then refines to sentence-level. The result? Color-coded alignments showing which parts changed, stayed the same, or got reordered—slashing cognitive load so you focus on quality, not hunting correspondences.

Human-in-the-Loop Workflow

Humans are still king for nuanced judgment (e.g., 'Is this culturally appropriate for A2 learners?'). MuTSE puts evaluators 'in the loop' with interactive tools: highlight misalignments, score readability via built-in metrics (Flesch-Kincaid), add custom annotations, and export structured data for datasets. It's reproducible—no more 'I ran this prompt once' ambiguity—and supports educators building ITS content or researchers tuning LLMs.

Why It Matters

LLMs excel at simplification but need prompt engineering. Without tools like MuTSE, optimization is fragmented. This bridges NLP research and education: generate, align, evaluate, iterate—all in one visual dashboard. Contributions include parallel workflows, novel alignment, and annotation frameworks, with code/demo available. For students: think of it as a 'prompt playground' with smart matching to debug why GPT-4 beats Llama on B1 French texts.

Abstract

As Large Language Models (LLMs) become increasingly prevalent in text simplification, systematically evaluating their outputs across diverse prompting strategies and architectures remains a critical methodological challenge in both NLP research and Intelligent Tutoring Systems (ITS). Developing robust prompts is often hindered by the absence of structured, visual frameworks for comparative text analysis. While researchers typically rely on static computational scripts, educators are constrained to standard conversational interfaces -- neither paradigm supports systematic multi-dimensional evaluation of prompt-model permutations. To address these limitations, we introduce \textbf{MuTSE}\footnote{The project code and the demo have been made available for peer review at the following anonymized URL. https://osf.io/njs43/overview?view_only=4b4655789f484110a942ebb7788cdf2a, an interactive human-in-the-loop web application designed to streamline the evaluation of LLM-generated text simplifications across arbitrary CEFR proficiency targets. The system supports concurrent execution of $P \times M$ prompt-model permutations, generating a comprehensive comparison matrix in real-time. By integrating a novel tiered semantic alignment engine augmented with a linearity bias heuristic ($λ$), MuTSE visually maps source sentences to their simplified counterparts, reducing the cognitive load associated with qualitative analysis and enabling reproducible, structured annotation for downstream NLP dataset construction.

MuTSE: A Human-in-the-Loop Multi-use Text Simplification Evaluator: Research Paper Podcast