voxmap-studio: an open-source speaker diarization annotation tool with built-in cost instrumentation

voxmap-studio is an open-source, React-based tool designed to streamline speaker diarization annotation. Unlike existing tools that focus solely on the final output, voxmap-studio integrates cost instrumentation, recording every edit operation and the time spent by the annotator. This allows researchers to quantitatively compare how different forms of automated assistance impact the efficiency and accuracy of the annotation process.

The tool utilizes a stride-accelerated diarization engine to initialize the annotation canvas, allowing users to correct existing hypotheses rather than drawing speaker turns from scratch. To ensure data quality, the tool enforces a confirmation-gated export process: annotators must verify each segment, and the system injects "phantom" attention checks—fake speech segments in silent gaps—to prevent unverified automatic output from being released as ground truth. The system records metrics such as edit operations (e.g., create, resize, split, delete, reassign) and active editing time, storing them in a JSON sidecar file.

A preliminary study using nine AMI meeting audio files compared three conditions: manual annotation (C1), engine initialization with uncertainty highlighting (C2), and engine initialization with gallery-based labeling and recommendations (C3). The results showed that unassisted manual annotation was the most expensive and least accurate. Automatic initialization (C2 and C3) shifted the annotator's workload from creating turns to correcting them. Interestingly, the simplest form of assistance (C2) proved to be the most efficient and accurate in this small sample, suggesting that more complex assistance features may not always yield proportional gains in productivity.

By treating annotation cost as a measurable, first-class output, voxmap-studio provides a framework for evidence-based improvements in data labeling workflows. This instrumentation enables developers to identify which specific assistance features actually save time, helping to lower the high costs associated with creating high-quality, human-verified training data for speaker diarization models.