RT-DocLayout: Real-Time End-to-End Document Layout Analysis with Reading Order in the Wild

Document layout analysis (DLA) is essential for parsing complex documents, yet current systems often struggle with the trade-off between accuracy and computational efficiency. Traditional cascaded pipelines are prone to error propagation, while large-scale generative Transformers are too slow for real-time applications. Furthermore, existing methods often rely on axis-aligned bounding boxes, which fail to accurately isolate content in warped or skewed documents. This paper asks: can we build a unified, compact, and end-to-end architecture that simultaneously handles detection, segmentation, and reading order while maintaining real-time speed?

The authors propose RT-DocLayout, a 33M-parameter framework built upon the RT-DETR architecture. The model reformulates DLA as a unified, mask-centric task. Instead of using separate modules for different stages, RT-DocLayout uses a single query-based Transformer decoder to concurrently perform classification, bounding box regression, pixel-level mask generation, and reading order prediction in one forward pass. To improve robustness against real-world distortions like page curling or perspective shifts, the authors introduce a two-stage data augmentation pipeline that simulates intrinsic surface deformations and extrinsic projective transformations. The model is trained using a multi-task loss function that weights reading order prediction heavily to ensure logical sequence consistency.

RT-DocLayout sets a new state-of-the-art across multiple benchmarks, achieving an overall accuracy of 92.46% while maintaining an inference speed of 132.1 FPS on an NVIDIA A100 GPU. The model significantly outperforms previous methods like PP-DocLayoutV2 and MinerU2.5 in both accuracy and efficiency. By providing pixel-level masks rather than simple bounding boxes, the model effectively mitigates background interference and content overlap, leading to higher quality downstream document reconstruction. The unified multi-task design eliminates the need for redundant post-processing, making it a highly scalable solution for industrial document intelligence.