Automated Semantic Fault Localization in SysML v2: A Human-in-the-Loop Framework Using Knowledge-Graph Augmented LLMs

As Model-Based Systems Engineering (MBSE) adopts SysML v2, engineers face a critical challenge: while compilers can catch syntactic errors, they cannot detect semantic faults—models that are syntactically valid but violate physical or engineering constraints (e.g., connecting a fluid port to an electrical interface). This paper investigates whether fine-tuned small language models (SLMs), augmented with domain-specific knowledge graphs, can effectively localize and repair these subtle semantic errors.

The researchers developed a framework that treats fault repair as a patch-generation task rather than a full code-rewriting task. They first constructed a dataset of 8,301 SysML v2 models, including both syntactically and semantically faulty examples. To generate semantic faults, they used a domain knowledge graph (KG) to systematically inject physical violations into valid models. During inference, the KG provides relevant engineering constraints to the SLM, which then outputs a unified diff patch. This approach keeps the human engineer in the loop, allowing for review of the proposed repairs before they are applied to the model.

The study demonstrates that fine-tuning SLMs (specifically Qwen2.5-Coder-1.5B and DeepSeek-Coder-6.7B) significantly enhances their ability to handle domain-specific constraints. The fine-tuned models achieved a semantic fault repair success rate of over 91%, compared to less than 3% for baseline models. Furthermore, by training the models to output unified diff patches rather than full code blocks, the framework reduced the required token output by over 60%, making the process more computationally efficient and easier for engineers to verify.

This work provides a practical, scalable method for improving the reliability of MBSE models. By automating the detection of semantic errors that currently propagate through the design process and cause costly integration failures, this framework helps bridge the gap between formal modeling languages and real-world physical engineering constraints. It offers a path toward AI-assisted verification that respects the engineer's role in the design process.