AI-Assisted Audit of Intensive-Care Risk Scores under Missing Laboratory Data: Evidence, Failure Modes and Deployment Trade-offs
DOI:
https://doi.org/10.66687/rx6q9692Keywords:
Artificial Intelligence, Machine Learning, Medical Informatics, Decision Support Systems, Clinical, Reproducibility of ResultsAbstract
Intensive-care risk scores can become unreliable when laboratory results arrive late or are missing at the moment clinicians need a prediction. An evaluation that overlooks these constraints may reach conclusions that cannot be reproduced. This study aims to test how missing-data mechanisms affect the calibration and usefulness of AI-based deterioration alerts. The evaluation would follow a retrospective, patient-level validation protocol using de-identified longitudinal ICU records, with hospital and calendar-period holdouts. The choice of evaluation units and exclusion criteria is informed by the broader methodological record [3–7]. The analysis will contrast complete-case scoring, imputation and missingness-aware prediction under matched alert budgets. The evidence-mapping component examines why a successful result in one technical environment may fail elsewhere. The technical pipeline must remain auditable: prior work on storage operations provides system-level context [1], while a second source identifies infrastructure conditions relevant to data continuity [2], rather than patient-level outcomes. Assessment will focus on calibration error, sensitivity at a fixed alert rate, subgroup performance and time to actionable warning. The supporting evidence base motivates the chosen comparator conditions and robustness criteria [8–12]. Critical sensitivity checks address patient-level leakage, changes in test-ordering practices and clinicians’ review of ambiguous outcomes. Further systems and modelling literature supplies context for the proposed sensitivity analyses [13–17]. Stress tests would distinguish outright failures, reduced confidence and cases where a simpler baseline is preferable. The expected output is a reusable evaluation framework with transparent uncertainty and reporting rules, informed by wider reproducibility literature [18–23].
References
[1] Mohammed Nazir. (2022). Performance Comparison of AWS EBS, EFS and S3 for Enterprise Cloud-Storage Workloads. International Journal of Business & Computational Sciences, 2(1), 26–33. https://ijbcs.org/index.php/IJBCS/article/view/2022-01-05
[2] Mohammed Nazir. (2021). RTO and RPO Optimization Using Asynchronous Replication in Multi-Petabyte Enterprise Storage Environments. Journal of Computing, Intelligence and Information Sciences, 2(1). https://jciis.com/index.php/jciis/article/view/2020-01-04
[3] Esteva, A., Kuprel, B., Novoa, R. A., et al. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542, 115–118. https://www.nature.com/articles/nature21056
[4] Johnson, A., Pollard, T., & Mark, R. (2016). MIMIC-III Clinical Database, version 1.4. PhysioNet. https://physionet.org/content/mimiciii/1.4/
[5] Begum, M. H., Talukder, S. I., Ahmed, M. J., Khan, M. F. I., & Azam, M. A. (2025). XAI-Driven Multimodal Deep Learning for Early Sepsis Prediction in ICU. Journal of Science Technology and Social Transformation, 1(1), 30–41. https://jstst.info/j/article/view/25
[6] Jumper, J., Evans, R., Pritzel, A., et al. (2021). Highly accurate protein structure prediction with AlphaFold. Nature, 596, 583–589. https://www.nature.com/articles/s41586-021-03819-2
[7] Liu, X., Faes, L., Kale, A. U., et al. (2020). Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nature Medicine, 26, 1364–1374. https://www.nature.com/articles/s41591-020-1034-x
[8] Singhal, K., Azizi, S., Tu, T., et al. (2023). Large language models encode clinical knowledge. Nature, 620, 172–180. https://www.nature.com/articles/s41586-023-06291-2
[9] McKinney, S. M., Sieniek, M., Godbole, V., et al. (2020). International evaluation of an AI system for breast cancer screening. Nature, 577, 89–94. https://www.nature.com/articles/s41586-019-1799-6
[10] Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems, 30. https://arxiv.org/abs/1706.03762
[11] Guo, C. et al. (2017). On Calibration of Modern Neural Networks. Proceedings of Machine Learning Research, 70, 1321–1330. https://proceedings.mlr.press/v70/guo17a.html
[12] Li, T. et al. (2024). MinFlow: High-performance and Cost-efficient Data Passing for I/O-intensive Stateful Serverless Analytics. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/li
[13] Lundberg, S. M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems, 30. https://arxiv.org/abs/1705.07874
[14] Patel, T. et al. (2020). Uncovering Access, Reuse, and Sharing Characteristics of I/O-Intensive Files on Large-Scale Production HPC Systems. USENIX FAST '20. https://www.usenix.org/conference/fast20/presentation/patel-hpc-systems
[15] Swanson, M. et al. (2010). Contingency Planning Guide for Federal Information Systems (NIST SP 800-34 Rev. 1). NIST. https://doi.org/10.6028/NIST.SP.800-34r1
[16] Lu, R. et al. (2022). NVMe SSD Failures in the Field: the Fail-Stop and the Fail-Slow. USENIX ATC '22. https://www.usenix.org/conference/atc22/presentation/lu
[17] Mahmud, T. et al. (2023). ConfD: Analyzing Configuration Dependencies of File Systems for Fun and Profit. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/mahmud
[18] Li, Q. et al. (2023). More Than Capacity: Performance-oriented Evolution of Pangu in Alibaba. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/li-qiang-deployed
[19] Maneas, S. et al. (2022). Operational Characteristics of SSDs in Enterprise Storage Systems: A Large-Scale Field Study. USENIX FAST '22. https://www.usenix.org/conference/fast22/presentation/maneas
[20] Dai, Y. et al. (2024). Symbiosis: The Art of Application and Kernel Cache Cooperation. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/dai
[21] Wong, D.L.-K. et al. (2024). Baleen: ML Admission & Prefetching for Flash Caches. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/wong
[22] Zhou, W. et al. (2025). 3L-Cache: Low Overhead and Precise Learning-based Eviction Policy for Caches. USENIX FAST '25. https://www.usenix.org/conference/fast25/presentation/zhou-wenbin
[23] Jiao, B., Goel, A. & Wang, A.-I.A. (2025). Silhouette: Leveraging Consistency Mechanisms to Detect Bugs in Persistent Memory-Based File Systems. USENIX FAST '25. https://www.usenix.org/conference/fast25/presentation/jiao
Published
Issue
Section
License
Copyright (c) 2026 Luciana Salaverry

This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain copyright in their published work. Unless an individual Version of Record identifies different terms, JMRIS articles are distributed under the Creative Commons Attribution 4.0 International License (CC BY 4.0): https://creativecommons.org/licenses/by/4.0/. This permits sharing and adaptation, including commercial use, with appropriate attribution, a link to the license, and indication of changes. Third-party material may be subject to separate terms. Existing articles retain the license originally granted to their Version of Record; a policy update does not retrospectively replace that license.