The Human Review Threshold for LLM-Suggested Infrastructure Changes: A Cross-Site Validation Agenda

Abstract

Language-model suggestions can make infrastructure change requests easier to draft but may hide hazardous assumptions in fluent explanations. An evaluation that overlooks these constraints may reach conclusions that cannot be reproduced. The planned investigation seeks to identify which uncertainty and risk thresholds justify mandatory human approval for AI-proposed changes. Relevant methodological and data-quality literature informs the design of the sampling and validation strategy [3–7]. The protocol specifies a mixed-methods decision study combining standardised change tickets and blinded operator reviews. Comparators will include ungated suggestions, suggestions with confidence displays and suggestions with policy-based review requirements. Prior work on storage operations provides technical context for validation [1], while complementary systems evidence motivates service-continuity checks [2]. The cross-site extension holds out an entire institution or deployment location rather than relying only on random test splits. Statistical and computational sources help define the reproducibility checks and comparison procedures [8–12]. The primary outcomes are unsafe approvals, decision time, disagreement quality and the ability to recover from incorrect recommendations. Potential biases include over-trust, incomplete dependency information, access permissions and high-impact irreversible actions. Between-site heterogeneity would be examined through prespecified subgroup estimates and transparent domain-shift diagnostics. Additional research informs the prespecified stress tests and limits on interpretation [13–17]. The intended contribution is an auditable research protocol rather than an unsupported claim of effectiveness; related work frames its transferability limits [18–23].