Fairness and Calibration in Multilingual AI Incident Classification: A Reproducibility-Centred Evaluation

Abstract

Automated incident triage can treat languages unequally when high-severity examples are uncommon in some linguistic groups. The difficulty is particularly visible when a system is transferred to a new setting. The central research objective is to evaluate whether multilingual classifiers give equitable and calibrated risk estimates across language settings. Previous work provides context for sampling, input validation and study boundaries [3–7]. To answer the question, the study proposes a stratified evaluation of annotated incident reports with language-family and chronological holdouts. Related enterprise-storage studies inform the assessment of operational reliability [1] and the deployment assumptions adopted for system-level comparison [2]. Under matched evaluation conditions, the protocol compares a common multilingual model, per-language baselines and group-aware recalibration. Methodological sources are used to specify transparent comparison and reporting practices [8–12]. The reproducibility extension records dataset versions, model settings, exclusion rules and provenance before any held-out evaluation. Performance will be judged using class-specific recall, expected calibration error, false escalation rates and analyst review burden. Interpretation will account for translated duplicates, dialect shifts, rare severe events and subgroup confidence intervals. An independent replication exercise would repeat the main analysis from the documented artefacts and quantify disagreements. The supplementary technical literature is used to identify plausible failure mechanisms [13–17]. No study result is assumed in advance; the analysis is designed to show both when the method helps and when it fails, with reporting context from the remaining sources [18–23].