Online Handwriting Trajectory Reconstruction from Kinematic Sensors using Temporal Convolutional Network

This paper addresses the challenge of reconstructing online handwriting (OH) trajectories—the digital path of a pen as it moves—using only the kinematic data captured by a digital pen's internal sensors. Unlike tablet-based systems that provide absolute coordinates, inertial measurement unit (IMU) sensors provide noisy, relative displacement data, making the reconstruction of precise handwriting strokes difficult.

The researchers developed a processing pipeline designed to map raw sensor signals (accelerometer, gyroscope, magnetometer, and force) to the corresponding handwriting trajectory. A key innovation is the use of Dynamic Time Warping (DTW) to align the asynchronous time-series data between the pen's sensor stream and the tablet's ground-truth recording. This alignment is critical because the pen and tablet operate at different sampling rates and experience variable transmission delays. The core of the reconstruction model is a Temporal Convolutional Network (TCN), which is trained to predict local displacement vectors (delta-x, delta-y) for each time frame. The authors also introduce the IRISA-KIHT dataset, a new benchmark for evaluating these reconstruction tasks.

The proposed TCN-based architecture demonstrates a significant improvement in reconstruction accuracy compared to existing state-of-the-art methods. By focusing on local displacement features and employing a robust alignment strategy, the model successfully handles the noise inherent in IMU sensors. The study also establishes a standardized evaluation protocol using the Fréchet distance, providing a more reliable metric for assessing the quality of reconstructed handwriting than previous methods that relied on indirect character recognition rates.

Digital pens that can write on any surface are valuable tools for education and collaborative work, as they bridge the gap between traditional paper-based learning and digital feedback. This research provides a scalable, high-performance method for turning low-cost kinematic sensors into precise digital input devices, facilitating better handwriting analysis and personalized feedback for learners without requiring specialized, expensive tablets.