AI-Assisted Audit of Intensive-Care Risk Scores under Missing Laboratory Data: Evidence, Failure Modes and Deployment Trade-offs

Authors

  • Luciana Salaverry University of Buenos Aires (UBA) Author

DOI:

https://doi.org/10.66687/rx6q9692

Keywords:

Artificial Intelligence, Machine Learning, Medical Informatics, Decision Support Systems, Clinical, Reproducibility of Results

Abstract

Intensive-care risk scores can become unreliable when laboratory results arrive late or are missing at the moment clinicians need a prediction. An evaluation that overlooks these constraints may reach conclusions that cannot be reproduced. This study aims to test how missing-data mechanisms affect the calibration and usefulness of AI-based deterioration alerts. The evaluation would follow a retrospective, patient-level validation protocol using de-identified longitudinal ICU records, with hospital and calendar-period holdouts. The choice of evaluation units and exclusion criteria is informed by the broader methodological record [3–7]. The analysis will contrast complete-case scoring, imputation and missingness-aware prediction under matched alert budgets. The evidence-mapping component examines why a successful result in one technical environment may fail elsewhere. The technical pipeline must remain auditable: prior work on storage operations provides system-level context [1], while a second source identifies infrastructure conditions relevant to data continuity [2], rather than patient-level outcomes. Assessment will focus on calibration error, sensitivity at a fixed alert rate, subgroup performance and time to actionable warning. The supporting evidence base motivates the chosen comparator conditions and robustness criteria [8–12]. Critical sensitivity checks address patient-level leakage, changes in test-ordering practices and clinicians’ review of ambiguous outcomes. Further systems and modelling literature supplies context for the proposed sensitivity analyses [13–17]. Stress tests would distinguish outright failures, reduced confidence and cases where a simpler baseline is preferable. The expected output is a reusable evaluation framework with transparent uncertainty and reporting rules, informed by wider reproducibility literature [18–23].

References

[1] Mohammed Nazir. (2022). Performance Comparison of AWS EBS, EFS and S3 for Enterprise Cloud-Storage Workloads. International Journal of Business & Computational Sciences, 2(1), 26–33. https://ijbcs.org/index.php/IJBCS/article/view/2022-01-05

[2] Mohammed Nazir. (2021). RTO and RPO Optimization Using Asynchronous Replication in Multi-Petabyte Enterprise Storage Environments. Journal of Computing, Intelligence and Information Sciences, 2(1). https://jciis.com/index.php/jciis/article/view/2020-01-04

[3] Esteva, A., Kuprel, B., Novoa, R. A., et al. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542, 115–118. https://www.nature.com/articles/nature21056

[4] Johnson, A., Pollard, T., & Mark, R. (2016). MIMIC-III Clinical Database, version 1.4. PhysioNet. https://physionet.org/content/mimiciii/1.4/

[5] Begum, M. H., Talukder, S. I., Ahmed, M. J., Khan, M. F. I., & Azam, M. A. (2025). XAI-Driven Multimodal Deep Learning for Early Sepsis Prediction in ICU. Journal of Science Technology and Social Transformation, 1(1), 30–41. https://jstst.info/j/article/view/25

[6] Jumper, J., Evans, R., Pritzel, A., et al. (2021). Highly accurate protein structure prediction with AlphaFold. Nature, 596, 583–589. https://www.nature.com/articles/s41586-021-03819-2

[7] Liu, X., Faes, L., Kale, A. U., et al. (2020). Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nature Medicine, 26, 1364–1374. https://www.nature.com/articles/s41591-020-1034-x

[8] Singhal, K., Azizi, S., Tu, T., et al. (2023). Large language models encode clinical knowledge. Nature, 620, 172–180. https://www.nature.com/articles/s41586-023-06291-2

[9] McKinney, S. M., Sieniek, M., Godbole, V., et al. (2020). International evaluation of an AI system for breast cancer screening. Nature, 577, 89–94. https://www.nature.com/articles/s41586-019-1799-6

[10] Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems, 30. https://arxiv.org/abs/1706.03762

[11] Guo, C. et al. (2017). On Calibration of Modern Neural Networks. Proceedings of Machine Learning Research, 70, 1321–1330. https://proceedings.mlr.press/v70/guo17a.html

[12] Li, T. et al. (2024). MinFlow: High-performance and Cost-efficient Data Passing for I/O-intensive Stateful Serverless Analytics. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/li

[13] Lundberg, S. M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems, 30. https://arxiv.org/abs/1705.07874

[14] Patel, T. et al. (2020). Uncovering Access, Reuse, and Sharing Characteristics of I/O-Intensive Files on Large-Scale Production HPC Systems. USENIX FAST '20. https://www.usenix.org/conference/fast20/presentation/patel-hpc-systems

[15] Swanson, M. et al. (2010). Contingency Planning Guide for Federal Information Systems (NIST SP 800-34 Rev. 1). NIST. https://doi.org/10.6028/NIST.SP.800-34r1

[16] Lu, R. et al. (2022). NVMe SSD Failures in the Field: the Fail-Stop and the Fail-Slow. USENIX ATC '22. https://www.usenix.org/conference/atc22/presentation/lu

[17] Mahmud, T. et al. (2023). ConfD: Analyzing Configuration Dependencies of File Systems for Fun and Profit. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/mahmud

[18] Li, Q. et al. (2023). More Than Capacity: Performance-oriented Evolution of Pangu in Alibaba. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/li-qiang-deployed

[19] Maneas, S. et al. (2022). Operational Characteristics of SSDs in Enterprise Storage Systems: A Large-Scale Field Study. USENIX FAST '22. https://www.usenix.org/conference/fast22/presentation/maneas

[20] Dai, Y. et al. (2024). Symbiosis: The Art of Application and Kernel Cache Cooperation. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/dai

[21] Wong, D.L.-K. et al. (2024). Baleen: ML Admission & Prefetching for Flash Caches. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/wong

[22] Zhou, W. et al. (2025). 3L-Cache: Low Overhead and Precise Learning-based Eviction Policy for Caches. USENIX FAST '25. https://www.usenix.org/conference/fast25/presentation/zhou-wenbin

[23] Jiao, B., Goel, A. & Wang, A.-I.A. (2025). Silhouette: Leveraging Consistency Mechanisms to Detect Bugs in Persistent Memory-Based File Systems. USENIX FAST '25. https://www.usenix.org/conference/fast25/presentation/jiao

Published

2025-06-30

Similar Articles

1-10 of 134

You may also start an advanced similarity search for this article.