Integrating High-Level Requirements to Low-Level Tests with Machine-Readable V&V Specifications

Modern software development has matured significantly in low-level testing, with tools like pytest and JUnit enabling rapid, automated verification. However, systems engineering remains largely disconnected from these developer workflows. High-level requirements—often found in procurement documents or regulatory standards—are rarely linked to the actual code tests that verify them. This manual, fragmented process is increasingly problematic for AI-enabled and cyber-physical systems, where regulators require rigorous, audit-ready evidence that specific requirements have been met.

VNVSpec addresses this disconnect by treating verification and validation (V&V) specifications as first-class, machine-readable data. Instead of living in isolated documents, requirements, hazards, and operational design domains are defined as typed, version-controlled objects (JSON, YAML, or TOML). The framework enforces a directed acyclic graph (DAG) structure, ensuring that every requirement can be traced from its high-level definition down to specific module-level metrics and, ultimately, to concrete test evidence.

VNVSpec is designed to meet developers where they are. It does not replace existing test runners; rather, it consumes their outputs. Through plugins for pytest, JUnit XML ingestion, and specialized adapters for AI models (e.g., PyTorch, HuggingFace), the framework collects test results and maps them to the corresponding requirements. It includes a quality checker that flags vague or unverifiable requirements at authoring time and provides automated exporters to generate compliance matrices, GSN assurance cases, and regulatory documentation. By integrating into CI/CD pipelines, VNVSpec allows teams to treat compliance as a continuous, automated process rather than a manual, error-prone hurdle.