Mustafa S. Aljumaily, Hayder Kareem Abed, Nawar S. Alseelawi
4 min
Cybersecurity literature frequently advocates for "Defense-in-Depth" and "Human-AI collaboration" but often lacks a formal, quantitative framework to guide implementation. This paper seeks to bridge this gap by creating a falsifiable mathematical model that connects Defense-in-Depth theory, AI-driven pattern recognition, and human-AI triage capacity constraints.
The authors formalize the security architecture using three components:
They validate this model using Monte Carlo simulations with parameters representing a typical security stack (network IDS, endpoint EDR, and OT/SCADA monitors).
The study yields two primary insights. First, AI augmentation provides the greatest marginal gains in systems where traditional layering is already saturating, rather than providing a flat, additive benefit. Second, the authors find an "interior optimum" for human-AI collaboration. Contrary to the assumption that more human oversight is always better, the model shows that if an analyst's accuracy is imperfect, reviewing every AI-flagged alert can actually decrease the overall system-level detection probability compared to a filtered, automated approach. This suggests that SOC designers should target a specific capacity ratio rather than aiming for 100% human coverage.
This research moves the conversation around "responsible AI" in cybersecurity from qualitative, often vague recommendations toward concrete, testable design rules. For organizations managing IT/OT-converged critical infrastructure, this model provides a principled way to allocate scarce human expertise and optimize AI thresholds based on actual operational constraints.
Cybersecurity literature has extensively documented the operational benefits of artificial intelligence (AI) for threat detection, incident response, and prevention, while raising qualitative concerns about over-automation, algorithmic bias, and analyst-skill erosion. What remains largely absent is a formal, falsifiable model connecting three constructs that recur across this literature: Defense-in-Depth Theory, the Artificial Intelligence Theory of Pattern Recognition, and human-AI collaboration in security operations. This paper develops such a model. We formalize layered defense as a Bernoulli detection cascade in which AI augmentation enters multiplicatively across layers; we formalize each layer's pattern-recognition behavior as a Neyman-Pearson/Bayesian detector with a derived closed-form optimal threshold; and we formalize human-AI triage as a capacity-constrained cascade with an explicit, quantifiable trade-off between detection probability and false-alarm ("alert fatigue") rate. A Monte Carlo/analytical simulation evaluated at illustrative but realistic operating points shows that (i) AI augmentation compounds across defense layers, delivering its largest marginal gains exactly where traditional layering saturates, and (ii) full human review of AI-flagged alerts is not optimal: increasing analyst capacity toward 100% coverage cuts false alarms by roughly 20-fold but simultaneously lowers system-level detection probability, because imperfect analyst accuracy is then applied to every alert rather than a filtered subset. These results give the widely repeated qualitative recommendation of "balanced human-AI collaboration" a precise, testable form and suggest an interior-optimum capacity ratio as a concrete design target for security operations centers (SOCs), including those securing IT/OT-converged critical infrastructure.
Alex: [pace picking up, connecting dots] That moves the conversation from qualitative advice — "balance automation with oversight" — to an actual design target a manager could set. But how much should we trust these numbers, given they come out of a simulation? [[RP_SECTION:limitations-of-simulation-models|Limitations of simulation models]]
Sam: [steady, acknowledging the weight] That's the central limitation, and it's worth sitting with. The detectors in the model are synthetic — cleanly separated statistical distributions rather than logs pulled from a live SOC. The authors also assume the layers behave independently of each other, which is a strong simplification. In a real breach, an attacker who compromises the network layer often gains the access needed to slip past endpoint detection too, which means the layers are likely correlated in ways this model doesn't yet capture.
Alex: [reflective] So it's a compelling framework for thinking about the trade-off, but it still needs to be tested against messier, correlated real-world attack data before anyone sets thresholds by it.
Sam: [quiet conviction] Precisely. The real contribution here is the formalization itself — defining an alert fatigue index and a capacity-constrained cascade gives SOC designers a shared language for reasoning about this trade-off, rather than relying on intuition alone. Future work could bring in reinforcement learning to adapt these thresholds as the threat environment shifts, or model how human accuracy itself degrades under the alert fatigue the system is trying to manage. [[RP_SECTION:engineering-security-operations|Engineering security operations]]
Alex: [deliberate, summarizing] It's a shift from hand-wavy best practice to something closer to a falsifiable engineering problem. If a team can calibrate these parameters to their own environment, they'd have an actual roadmap instead of a gut feeling.
Sam: [calm] That's the core of it. The paper gives human-AI collaboration in security operations the mathematical scaffolding to be treated as an optimization problem rather than a philosophical one — shifting the goal from maximizing human involvement to maximizing system-level utility. For anyone designing critical infrastructure protection, that's a considerably more grounded place to start.