Lightweight large language models hold promise for scientific applications but remain limited in rule-constrained domains because they can reproduce linguistic patterns without following scientific principles, leading to hallucinations. Here, we show that G-Frame, an adaptive multi-agent framework inspired by Bayesian game and team game principles, enables a closed-loop process for automated data generation, model evaluation, and training. By assigning complementary roles to multiple agents, G-Frame generates structured reasoning data and constructs a specialized corpus containing 363,045 chain-of-thought examples and 199,589 question–answer pairs. Using this corpus together with domain pre-training and adaptive training, the 7-billion-parameter model OmniChem achieves competitive performance across the reported benchmarks and reduces the ChemJudge hallucination rate from 41.95% for the base model to 12.91%. Case studies further illustrate potential applications in molecular design and selected synthesis-planning tasks. These results support G-Frame as an extensible game theory-driven multi-agent framework for improving the reliability of lightweight scientific language models. Lightweight AI models can be unreliable in specialized scientific tasks. Here, the authors introduce G-Frame, which uses game theory to coordinate multiple AI agents, automate data generation and training, and reduce hallucinations.