Cross-Site Validation of Clinical Prediction Models on Cloud Data Warehouses
DOI:
https://doi.org/10.66687/0yeahp03Keywords:
Artificial Intelligence, Machine Learning, Medical Informatics, Decision Support Systems, Clinical, Reproducibility of ResultsAbstract
Cloud-hosted clinical data pipelines can harmonise hospital records while obscuring differences in coding, missingness and data provenance. The difficulty is particularly visible when a system is transferred to a new setting. The central research objective is to evaluate the transferability of prediction models across institutions using a traceable data-warehouse workflow. Previous work provides context for sampling, input validation and study boundaries [3–7]. To answer the question, the study proposes a multi-institution protocol with source-hospital development data, held-out partner sites and time-based external validation. Enterprise-storage sources are used to specify constraints on data availability and provenance [1], with complementary infrastructure evidence defining practical recovery safeguards [2]. These citations are not treated as clinical effectiveness evidence. Under matched evaluation conditions, the protocol compares local versus pooled models after identical coding rules, feature windows and missing-data checks. Methodological sources are used to specify transparent comparison and reporting practices [8–12]. The design prioritises precise operational definitions and a straightforward baseline before introducing more complex alternatives. Performance will be judged using calibration-in-the-large, discrimination, subgroup error and reproducibility of extracted cohorts. Interpretation will account for data lineage, hospital-specific coding shifts, delayed ingestion and governance of cross-site access. Primary analyses would be reported by the main deployment or participant subgroup so that averaged results do not conceal systematic failures. The supplementary technical literature is used to identify plausible failure mechanisms [13–17]. No study result is assumed in advance; the analysis is designed to show both when the method helps and when it fails, with reporting context from the remaining sources [18–23].
References
[1] Mohammed Nazir. (2022). Performance Comparison of AWS EBS, EFS and S3 for Enterprise Cloud-Storage Workloads. International Journal of Business & Computational Sciences, 2(1), 26–33. https://ijbcs.org/index.php/IJBCS/article/view/2022-01-05
[2] Mohammed Nazir. (2022). Automated SnapMirror Migration from On-Premises Storage to Cloud Volumes ONTAP: A Replication Performance Study. Journal of Multidisciplinary Research and Integrated Sciences, 2(1), 61–72. https://jmris.com/index.php/jmris/article/view/2022V1P6
[3] Johnson, A., Pollard, T., & Mark, R. (2016). MIMIC-III Clinical Database, version 1.4. PhysioNet. https://physionet.org/content/mimiciii/1.4/
[4] Singhal, K., Azizi, S., Tu, T., et al. (2023). Large language models encode clinical knowledge. Nature, 620, 172–180. https://www.nature.com/articles/s41586-023-06291-2
[5] Jumper, J., Evans, R., Pritzel, A., et al. (2021). Highly accurate protein structure prediction with AlphaFold. Nature, 596, 583–589. https://www.nature.com/articles/s41586-021-03819-2
[6] Esteva, A., Kuprel, B., Novoa, R. A., et al. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542, 115–118. https://www.nature.com/articles/nature21056
[7] Liu, X., Faes, L., Kale, A. U., et al. (2020). Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nature Medicine, 26, 1364–1374. https://www.nature.com/articles/s41591-020-1034-x
[8] McKinney, S. M., Sieniek, M., Godbole, V., et al. (2020). International evaluation of an AI system for breast cancer screening. Nature, 577, 89–94. https://www.nature.com/articles/s41586-019-1799-6
[9] Lundberg, S. M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems, 30. https://arxiv.org/abs/1705.07874
[10] Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention Is All You Need. Advances in Neural Information Processing Systems, 30. https://arxiv.org/abs/1706.03762
[11] Guo, C. et al. (2017). On Calibration of Modern Neural Networks. Proceedings of Machine Learning Research, 70, 1321–1330. https://proceedings.mlr.press/v70/guo17a.html
[12] Vrable, M., Savage, S. & Voelker, G.M. (2012). BlueSky: A Cloud-Backed File System for the Enterprise. USENIX FAST '12. https://www.usenix.org/conference/fast12/bluesky-cloud-backed-file-system-enterprise
[13] Mahdisoltani, F., Stefanovici, I. & Schroeder, B. (2017). Proactive Error Prediction to Improve Storage System Reliability. USENIX ATC '17. https://www.usenix.org/conference/atc17/technical-sessions/presentation/mahdisoltani
[14] Cidon, A. et al. (2013). Copysets: Reducing the Frequency of Data Loss in Cloud Storage. USENIX ATC '13. https://www.usenix.org/conference/atc13/technical-sessions/presentation/cidon
[15] Swanson, M. et al. (2010). Contingency Planning Guide for Federal Information Systems (NIST SP 800-34 Rev. 1). NIST. https://doi.org/10.6028/NIST.SP.800-34r1
[16] Mahmud, T. et al. (2023). ConfD: Analyzing Configuration Dependencies of File Systems for Fun and Profit. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/mahmud
[17] Li, Q. et al. (2023). More Than Capacity: Performance-oriented Evolution of Pangu in Alibaba. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/li-qiang-deployed
[18] Maneas, S. et al. (2022). Operational Characteristics of SSDs in Enterprise Storage Systems: A Large-Scale Field Study. USENIX FAST '22. https://www.usenix.org/conference/fast22/presentation/maneas
[19] Dai, Y. et al. (2024). Symbiosis: The Art of Application and Kernel Cache Cooperation. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/dai
[20] Wong, D.L.-K. et al. (2024). Baleen: ML Admission & Prefetching for Flash Caches. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/wong
[21] Li, T. et al. (2024). MinFlow: High-performance and Cost-efficient Data Passing for I/O-intensive Stateful Serverless Analytics. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/li
[22] Yang, Z. et al. (2024). TeRM: Extending RDMA-Attached Memory with SSD. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/yang-zhe
[23] Xu, J. et al. (2021). Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. arXiv:2110.02642. https://arxiv.org/abs/2110.02642
Published
Issue
Section
License
Copyright (c) 2026 Sayantani Bhattachargee

This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors retain copyright in their published work. Unless an individual Version of Record identifies different terms, JMRIS articles are distributed under the Creative Commons Attribution 4.0 International License (CC BY 4.0): https://creativecommons.org/licenses/by/4.0/. This permits sharing and adaptation, including commercial use, with appropriate attribution, a link to the license, and indication of changes. Third-party material may be subject to separate terms. Existing articles retain the license originally granted to their Version of Record; a policy update does not retrospectively replace that license.