Chao Yin, Hongzhe Yue, Qing Han, Difeng Hu, Zhenyu Liang, Fangzhou Lin, Bing Sun, Boyu Wang, Mingkai Li, Wei Yao, Jack C.P. Cheng
9 min
Abstract
Automated semantic understanding of dense point clouds is a prerequisite for Scan-to-BIM pipelines, digital twin construction, and as-built verification--core tasks in the digital transformation of the construction industry. Yet for industrial mechanical, electrical, and plumbing (MEP) facilities, this challenge remains largely unsolved: TLS acquisitions of water treatment plants, chiller halls, and pumping stations exhibit extreme geometric ambiguity, severe occlusion, and extreme class imbalance that architectural benchmarks (e.g., S3DIS or ScanNet) cannot adequately represent. We present Industrial3D, a terrestrial LiDAR dataset comprising 612 million expertly labelled points at 6 mm resolution from 13 water treatment facilities. At 6.6x the scale of the closest comparable MEP dataset, Industrial3D provides the largest and most demanding testbed for industrial 3D scene understanding to date. We further establish the first industrial cross-paradigm benchmark, evaluating nine representative methods across fully supervised, weakly supervised, unsupervised, and foundation model settings under a unified benchmark protocol. The best supervised method achieves 55.74% mIoU, whereas zero-shot Point-SAM reaches only 15.79%--a 39.95 percentage-point gap that quantifies the unresolved domain-transfer challenge for industrial TLS data. Systematic analysis reveals that this gap originates from a dual crisis: statistical rarity (215:1 imbalance, 3.5x more severe than S3DIS) and geometric ambiguity (tail-class points share cylindrical primitives with head-class pipes) that frequency-based re-weighting alone cannot resolve. Industrial3D, along with benchmark code and pre-trained models, will be publicly available at https://github.com/pointcloudyc/Industrial3D.
Sam: Weakly supervised methods use tiny fractions of labels—like 0.1% of points marked. They spread that knowledge by assuming similar nearby areas belong together. SQN, for example, propagates those sparse hints efficiently. The paper shows SQN with minimal labels outperforms fully supervised methods by a clear margin.
Alex: Huh—that flips the usual idea. So less labeling can beat more? How do no-label methods stack up?
Sam: Unsupervised methods skip labels by training on shape contrasts—like pulling similar views close and pushing different ones apart, sorting puzzle pieces by edge fits. But pure geometry can't split valves from pipes. Foundation models, pre-trained on building scans, hit around 16% accuracy. The tests reveal a hierarchy: sparse labels outperform full supervision, but all face the dual crisis.
Alex: Yeah... with sparse labels working best so far, how did they create this massive dataset? Labeling millions of points sounds like a nightmare.
Sam: They used a four-step process. First, they sliced full scans into smaller chunks. Next, people outlined groups of points matching one object, like circling dots for a valve. Then they stitched chunks back and cleaned up edges. This took 754 person-hours, mostly on outlining and checking.
Alex: Human effort made it possible, but divided smartly. What makes these scans tougher than office building ones?
Sam: The scans cover 13 zones in water plants, over 15,000 square meters. Big classes like pipes take 77% of points, tiny ones like strainers just 3%. Rare parts cluster in specific spots. Test areas are separate zones with 14% of data, mimicking unseen factories. Unlike simpler scans, these have parts blocking each other, long pipes everywhere, valves buried in pipe jungles, and similar metal colors—so shape alone must tell them apart.
Alex: That explains why pipes fool everything into lumping valves with them.
Sam: Exactly. This setup lets Scan-to-BIM check installs against plans accurately. The paper positions it as a tough new standard, 3.5 times more imbalanced than prior sets.
Alex: How did they measure those gaps? What shows the head-tail split most clearly?
Sam: They used mean intersection over union, or mIoU—like judging a pizza slicer by how neatly it cuts every slice, even tiny ones. Best fully trained methods averaged 56% mIoU. Big parts like pipes scored near 88%, tiny ones like valves only 30%—a nearly 60-point gap.
Alex: mIoU reveals the imbalance because it treats rare parts fairly. Why does that gap matter for digital twins?
Sam: That gap breaks Scan-to-BIM, where scans turn into models to verify if parts match plans—like checking a Lego set against instructions. Missing a rare strainer invalidates the plumbing model, risking failures in plants. Tests used separate zones, with 86% for training and 14% held out.
Alex: Even top methods ace pipes but flop on valves, dooming the twins.
Sam: Precisely. It highlights needs like better shape modeling for ambiguities and label tricks for rarity.
Alex: What specific changes help tail classes most, and why does a huge gap stick around?
Sam: One approach gives extra weight to rare classes, like paying more attention to quiet kids in a loud classroom. It uses focal loss to zoom in on mistakes, and boundary-aware modeling to check edges where objects meet, like seams in a quilt. This lifts tail-class scores from 24% to 30% mIoU—a clear improvement. But the head-tail gap stays nearly 60 points because pipe-like shapes in valves confuse features deeply.
Alex: Weighting and edge focus help a bit, but geometry wins out. How does that play into no-label methods?
Sam: Unsupervised clustering spots basic shapes like cylinders, then grows groups outward, like puzzle corner pieces adding neighbors. But on these scans, it hits just 12% mIoU because 86% of tail points share cylinder shapes with pipes—models lump them together. Tail classes like valves score zero.
Alex: Geometry shortcut explains the collapse. And foundation models—do prompts fix the pipe confusion?
Sam: Point-SAM uses prompts like clicking points. In realistic mode with examples per class, it gets 16% mIoU, tails at 5%. Cylinders blur boundaries like flange-pipe joins. Even ideal prompts reach only 21%, 35 points behind supervised—pre-training on rooms misses industrial cylinders.
Alex: The hierarchy holds because each level fights rarity and shape harder. Imbalance is the root—how bad compared to other scans?
Sam: Imbalance here is 215-to-1, over three times worse than building datasets at 62-to-1. Models ace heads at 88% but ignore tails without fixes.
Alex: Pulling it all together, sparse labels beat full ones, but everyone struggles with rare pipe-lookalikes. What does that mean for using these scans in factories?
Sam: The paper highlights three takeaways. First, foundation models lag 40 points behind supervised because room-shape training doesn't transfer to cylinders. Second, sparse supervision with 0.1% labels outperforms full labeling by cutting overfitting. Third, the dual crisis keeps tail classes low even after tweaks.
Alex: It's not just more data—it's smarter handling of boundaries and shifts. Any spots where this benchmark falls short?
Sam: Yes. It covers only semantic labeling per point, not distinguishing separate instances—which Scan-to-BIM needs. Data sticks to water plants, so others might differ. It measures problems but doesn't solve them.
Alex: Fair points... keeps it realistic. For engineers scanning plants today?
Sam: This sets a solid testbed for Scan-to-BIM, verifying builds and maintaining digital twins. It flags paths like fine-tuning foundation models to close the 40-point gap, enabling reliable AI for infrastructure.
Alex: That's a clear step ahead. Thanks, Sam—this makes the challenges and fixes feel tangible. Thanks for joining us on ResearchPod.