RAGAL: A Frugal, Fully Local Retrieval-Augmented Assistant for Technical Support at a Government Agency

This paper presents RAGAL, a retrieval-augmented generation system designed for a Romanian government agency under strict constraints: zero data egress, a read-only mandate, and limited hardware (a single 8 GB consumer laptop). The system supports technical staff by retrieving precedents from 15,073 resolved support tickets and internal documents to draft SQL correction scripts.

The author demonstrates that investing in retrieval engineering—specifically hybrid dense-sparse retrieval and intent routing—yielded the highest performance gains, improving internal evaluation scores from 62% to 81%. Further improvements were achieved by fine-tuning the bge-m3 embedder on ticket data. The paper provides a practical, reproducible recipe for full embedder fine-tuning within 8 GB of VRAM, utilizing 8-bit optimizers and gradient checkpointing to avoid silent memory spills that occur on Windows consumer hardware.

A critical finding is that fine-tuning a retriever on one domain (tickets) can silently degrade performance on another (documents). This regression remained invisible until the author constructed a per-domain evaluation set. To address this, the author implemented a zero-egress repair strategy using locally generated synthetic queries (GenQ). By training on a combined dataset of real ticket pairs and synthetic document pairs, the system successfully restored document retrieval performance to or above baseline levels while maintaining the significant gains made on the ticket domain.

The paper also highlights two counter-intuitive design choices: masking personally identifiable information (PII) improved generation quality by preventing the model from copying real user values into templates, and a structural "anchor distillation" scheme—where the LLM only writes prose around verbatim SQL—made SQL hallucination impossible by construction. These methods, combined with rule-based interceptors that bypass the LLM for sensitive lookups, ensure the system remains safe and reliable for public sector use.