Programmable logic controllers (PLCs) run industrial plants, and large language models can already generate independent program organization units (POUs) for them. Whether such logic integrates into an existing PLC project and then runs correctly has been checked only in limited tests. We present \textsc{SemaPLC}, a project-grounded and verification-gated agent harness assembled from conventional tools but governed by a strict completion rule. Rather than stopping when the model judges its own output adequate, \textsc{SemaPLC} declares a task complete only when logged external checks confirm it. Those checks cover the specification, the compilation, and the behavior on a live runtime. On 117 independent-POU tasks matching existing benchmarks, it attains the highest strict verified pass rate on all seven models (72.6\% mean). On a project-context track of 65 tasks whose generated logic must compile and run inside a real project, it attains the highest mean on integrated compilation, static behavior, and dynamic behavior. Of the three layers, dynamic behavior is the most revealing. We measure it by deploying the generated and the reference logic to a live PLC runtime and comparing their executed traces. All methods fall within 10 static points of one another, whereas dynamic scores separate them sharply, from 22.4 to 31.4 for the baselines against 52.2 for \textsc{SemaPLC}. Overall, our verification-gated harness raises the mean at every layer and most sharply at runtime. Execution, not static scoring, is the faithful test of whether generated control logic actually works. \textsc{SemaPLC} is open-sourced at https://github.com/midea-ai/SemaPLC.
Alex: Welcome to another episode of ResearchPod. Sam, what are we looking at today?
Sam: We're looking at a paper about how artificial intelligence writes software for industrial control systems — the kind of software that runs factory floors, power plants, and automated machinery. The central claim is that current AI tools can write isolated code snippets without much trouble, but they routinely fail silently when placed inside massive, real-world industrial projects.
Alex: "Fail silently" — what does that mean exactly?
Sam: It means the code looks fine on the surface. It compiles, it passes basic checks, and nobody raises an alarm. But when the program actually runs on real equipment, something goes wrong — a timer fires at the wrong moment, a safety interlock doesn't trigger — and the system either damages equipment or behaves in ways the engineers never intended. No error message. No warning. Just a quiet, dangerous failure.
Alex: So the paper is asking: why does AI-generated code break when it meets a real factory floor, and how do we catch those hidden problems before they cause damage?
Sam: Exactly. And the authors' answer is a framework they built called SEMAPLC. Think of it like a flight simulator with a very strict rule: the pilot isn't allowed to land the plane until external instruments confirm every safety light is green. The AI model isn't allowed to declare its code finished until logged, external evidence confirms the program actually behaves correctly.
Alex: That's a meaningful shift. Most code generation tools let the AI decide when it's done.
Sam: Right, and that's the core problem they're solving. The system uses a strict completion rule — the AI is blocked from finishing unless external tool logs confirm success. Not the model's own judgment. Not a visual inspection. Actual machine-readable evidence from outside the model.
Alex: So how does it gather that evidence? What does the checking process actually look like?
Sam: It runs in layers. First, an automated audit reviews the code clause by clause against the original written requirement — like a teacher checking your essay against the assignment brief. Then it checks whether the code compiles correctly, with no syntax errors. Finally, and this is the critical part, it deploys the code to a live runtime environment and actually runs it.
Alex: And what happens during that live run?
Sam: The system puts the program through test scenarios and watches what the machine actually does — which variables change, when they change, in what order. It then compares that observed behavior against what the requirement said should happen. If there's a mismatch, the system doesn't just flag an error — it diagnoses the type of mismatch. A signal that never changes at all points to a wiring problem. A signal that changes too late points to a timer configuration issue. The failure itself becomes useful information.
Alex: So the errors aren't dead ends — they're clues that get fed back into the repair process.
Sam: Precisely. And the repair process has strict rules of its own. Each checking stage allows up to two repair attempts. But here's the part that really matters: any time the AI modifies the code — even a small change — every prior test result is immediately wiped out. The system starts the verification sequence from scratch.
Alex: Why is that important? Why not just re-run the specific test that failed?
Sam: Because a change in one part of the code can affect behavior somewhere else entirely. If you only re-check the thing you just fixed, you might miss a new problem you accidentally introduced. By resetting everything, the system guarantees that the final delivered program is the exact version that earned every single passing result — not a patched-together version where some tests apply to an older draft.
Alex: That prevents the AI from quietly claiming victory on a version of the code that was never fully tested.
Sam: That's the guarantee the framework is built around. And it matters because industrial control systems don't have much tolerance for partial correctness.
Alex: So does this actually work in practice? What did the tests show?
Sam: The results were meaningful. The runtime verification step — actually running the code and checking its behavior — caught failures that static compilation alone missed entirely. Programs that looked correct on paper turned out to behave incorrectly when deployed. The harness consistently produced higher reliability on the project-level tasks that most closely resemble real factory conditions.
Alex: Are there limits to what this approach can catch?
Sam: Yes, and the paper is clear about this. The dynamic checks only cover the test scenarios the engineers designed in advance. If a rare edge case wasn't included in the test set — an unusual plant condition, an unexpected sequence of inputs — the framework won't catch a failure there. The tests only find what the engineers thought to look for.
Alex: So it's not a guarantee of perfect safety. It's a significant improvement over the current standard, with known blind spots.
Sam: That's a fair summary. The paper also notes that the advantage of the framework narrows when using the most capable underlying AI models — so it's not a fixed margin of improvement across every situation. And there's a computational cost: the system makes substantially more requests to the AI model per task than a simpler approach would. Though the authors note that overall time stays comparable, because the checks run in a distributed way.
Alex: So you're trading more computational work for meaningfully higher reliability at runtime.
Sam: That's the trade-off. And the broader point the paper makes is this: static compilation — just checking whether code is grammatically correct — cannot reveal deep execution flaws in industrial control logic. True reliability only comes when the AI's completion is conditioned on external runtime evidence, not its own assessment of whether the job is done.
Alex: It's a bit like the difference between a student saying "I think my answer is right" versus actually checking it against the answer key and showing your work.
Sam: That's a good way to put it. And the authors suggest that future work in this area will likely move toward end-to-end development environments where writing, generating, and deploying code are all governed by this same verification gate — so the safety check isn't a separate step bolted on at the end, but built into the entire process from the start.
Alex: That seems like the logical direction. Thanks, Sam — this was a clear look at a genuinely important problem.
Sam: Thanks for listening to ResearchPod.