Fairness and Calibration in Multilingual AI Incident Classification

Abstract

Automated incident triage can treat languages unequally when high-severity examples are uncommon in some linguistic groups. The difficulty is particularly visible when a system is transferred to a new setting. The central research objective is to evaluate whether multilingual classifiers give equitable and calibrated risk estimates across language settings. Previous work provides context for sampling, input validation and study boundaries [3–7]. To answer the question, the study proposes a stratified evaluation of annotated incident reports with language-family and chronological holdouts. Related enterprise-storage studies inform the assessment of operational reliability [1] and the deployment assumptions adopted for system-level comparison [2]. Under matched evaluation conditions, the protocol compares a common multilingual model, per-language baselines and group-aware recalibration. Methodological sources are used to specify transparent comparison and reporting practices [8–12]. The design prioritises precise operational definitions and a straightforward baseline before introducing more complex alternatives. Performance will be judged using class-specific recall, expected calibration error, false escalation rates and analyst review burden. Interpretation will account for translated duplicates, dialect shifts, rare severe events and subgroup confidence intervals. Primary analyses would be reported by the main deployment or participant subgroup so that averaged results do not conceal systematic failures. The supplementary technical literature is used to identify plausible failure mechanisms [13–17]. No study result is assumed in advance; the analysis is designed to show both when the method helps and when it fails, with reporting context from the remaining sources [18–23].