The Human Review Threshold for LLM-Suggested Infrastructure Changes

Abstract

Language-model suggestions can make infrastructure change requests easier to draft but may hide hazardous assumptions in fluent explanations. To address this, the study will identify which uncertainty and risk thresholds justify mandatory human approval for AI-proposed changes. The research design centres on a mixed-methods decision study combining standardised change tickets and blinded operator reviews. The choice of evaluation units and exclusion criteria is informed by the broader methodological record [3–7]. The analysis will contrast ungated suggestions, suggestions with confidence displays and suggestions with policy-based review requirements. The design prioritises precise operational definitions and a straightforward baseline before introducing more complex alternatives. Related enterprise-storage studies inform the assessment of operational reliability [1] and the deployment assumptions adopted for system-level comparison [2]. Assessment will focus on unsafe approvals, decision time, disagreement quality and the ability to recover from incorrect recommendations. The supporting evidence base motivates the chosen comparator conditions and robustness criteria [8–12]. Critical sensitivity checks address over-trust, incomplete dependency information, access permissions and high-impact irreversible actions. Further systems and modelling literature supplies context for the proposed sensitivity analyses [13–17]. Primary analyses would be reported by the main deployment or participant subgroup so that averaged results do not conceal systematic failures. The expected output is a reusable evaluation framework with transparent uncertainty and reporting rules, informed by wider reproducibility literature [18–23].