FUTO Swipe: Layout-Agnostic Neural Swipe Decoding

Traditional neural swipe decoders are typically "hard-coded" to the specific keyboard layout they were trained on. This means that if a user changes their keyboard layout—or if a developer wants to support a new language with a different key arrangement—the model must be retrained from scratch using a new, often expensive, corpus of swipe data. This paper addresses the challenge of creating a layout-agnostic neural decoder that maintains high accuracy while being flexible enough to work on any contiguous mobile keyboard layout.

The researchers propose a two-part solution to achieve layout flexibility. First, they design an encoder that does not learn fixed parameters for each key. Instead, it emits spectral coefficients representing the user's intended character selection at specific spatial and temporal locations. At inference time, the system takes the current keyboard's key coordinates as input and maps these coefficients to specific keys using a cosine basis function.

Second, to ensure the model learns the underlying characteristics of a swipe gesture rather than memorizing a specific layout, the authors employ a "coordinated augmentation" strategy. During training, they apply geometric transformations—such as scaling, rotation, and shearing—to both the swipe trajectory and the keyboard layout simultaneously. This forces the neural network to learn the relationship between the gesture and the key positions, rather than the absolute coordinates of a QWERTY board.

The resulting FUTO Swipe model demonstrates the ability to generalize to keyboard layouts that were never seen during the training process, sometimes even outperforming models trained specifically on those layouts. By combining the flexibility of traditional algorithmic template-matching with the high accuracy of modern neural networks, this approach significantly lowers the barrier to deploying swipe typing on diverse or custom keyboard layouts. To address the scarcity of public data, the authors also released the swipe.futo.org corpus, which contains over 1 million MIT-licensed swipe samples, providing a valuable resource for the research community.