Natural Language Access to Domain-Specific Metadata: A Reusable Framework for LLM Query Generation

The authors introduce the Natural Language Knowledge Graph Query (NLKGQ) system, a framework designed to bridge the gap between domain-specific metadata archives and researchers who lack expertise in structured query languages like SPARQL or SQL. The core of the approach is an 'ontology-first' development process: instead of relying on complex multi-agent systems or fine-tuning, the researchers formalize domain vocabulary and semantics into a Web Ontology Language (OWL) ontology. This ontology serves as the ground truth for an LLM, which translates natural language questions into structured queries.

The framework includes a web interface for end-users and a domain-agnostic harness that manages the interaction between the LLM and the knowledge graph. To validate the system, the authors applied it to a large-scale neuroimaging archive containing metadata for 2,000 MRI experiments. They conducted an extensive ablation study across eight different ontology representations and multiple LLM variants. By systematically stripping semantic information—such as comments and labels—from the ontology, they identified that these annotations are the most critical factors for LLM accuracy, often mattering more than the specific model architecture or prompt engineering techniques used.

A key contribution of this work is the direct comparison between SPARQL and SQL as query backends. Using an automated conversion process, the authors transformed their OWL ontology into a relational SQL schema. Their results indicate that the structural features of OWL provide a substantial advantage for LLM-driven query generation compared to traditional SQL Data Definition Language (DDL). This suggests that knowledge graphs are inherently better suited for natural language interfaces than relational databases when the schema is complex.