Conformal Monitoring of Storage-Failure Predictors during Workload Drift: A Cross-Site Validation Agenda

Authors

DOI:

https://doi.org/10.66687/3m24pm51

Keywords:

Machine Learning, Algorithms, Computer Simulation, Reproducibility of Results, Models, Statistical

Abstract

Predictive maintenance systems often report confident failure probabilities even after the underlying workload distribution changes. An evaluation that overlooks these constraints may reach conclusions that cannot be reproduced. The planned investigation seeks to quantify when conformal uncertainty monitoring can warn of deteriorating failure-prediction reliability. Relevant methodological and data-quality literature informs the design of the sampling and validation strategy [3–7]. The protocol specifies a chronologically blocked simulation that calibrates uncertainty on historical storage telemetry and tests later hardware cohorts. Comparators will include fixed-score alarms, probability-calibrated models and rolling conformal warning policies. Prior work on storage operations provides technical context for validation [1], while complementary systems evidence motivates service-continuity checks [2]. The cross-site extension holds out an entire institution or deployment location rather than relying only on random test splits. Statistical and computational sources help define the reproducibility checks and comparison procedures [8–12]. The primary outcomes are empirical coverage, alert lead time, false-positive workload and stability under distribution shift. Potential biases include serial dependence, rare-event labelling, replacement hardware and uncertainty during outages. Between-site heterogeneity would be examined through prespecified subgroup estimates and transparent domain-shift diagnostics. Additional research informs the prespecified stress tests and limits on interpretation [13–17]. The intended contribution is an auditable research protocol rather than an unsupported claim of effectiveness; related work frames its transferability limits [18–23].

References

[1] Mohammed Nazir. (2021). Capacity Reclamation and Thin-Provisioning Efficiency in Large-Scale Enterprise Storage Systems. International Journal of Business & Computational Sciences, 1(1), 15–22. https://ijbcs.org/index.php/IJBCS/article/view/2021P4

[2] Mohammed Nazir. (2023). FlexGroup and FlexCache Optimization for AWS-Based CAE/HPC File Workloads: A Latency and Throughput Evaluation. International Journal of Business & Computational Sciences, 3(1), 25–31. https://ijbcs.org/index.php/IJBCS/article/view/2023-01-04

[3] Darban, Z.Z. et al. (2022). Deep Learning for Time Series Anomaly Detection: A Survey. arXiv:2211.05244. https://arxiv.org/abs/2211.05244

[4] Mahdisoltani, F., Stefanovici, I. & Schroeder, B. (2017). Proactive Error Prediction to Improve Storage System Reliability. USENIX ATC '17. https://www.usenix.org/conference/atc17/technical-sessions/presentation/mahdisoltani

[5] Xu, J. et al. (2021). Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy. arXiv:2110.02642. https://arxiv.org/abs/2110.02642

[6] Guo, C. et al. (2017). On Calibration of Modern Neural Networks. Proceedings of Machine Learning Research, 70, 1321–1330. https://proceedings.mlr.press/v70/guo17a.html

[7] Pang, G. et al. (2020). Deep Learning for Anomaly Detection: A Review. arXiv:2007.02500. https://arxiv.org/abs/2007.02500

[8] Azam, M. A., Hossain, I., & Islam, M. (2024). Advancing Explainable Artificial Intelligence and Predictive Business Analytics to Strengthen Digital Transformation, Operational Resilience, and Productivity among U.S. Small and Medium-Sized Enterprises. European Journal of Management, Economics and Business, 1(1), 70–94. https://ejmeb.com/index.php/journal/article/view/323

[9] Li, X. et al. (2023). ParaRC: Embracing Sub-Packetization for Repair Parallelization in MSR-Coded Storage. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/li-xiaolu

[10] Bai, W. et al. (2023). Empowering Azure Storage with RDMA. USENIX NSDI '23. https://www.usenix.org/conference/nsdi23/presentation/bai

[11] Mihailescu, M., Soundararajan, G. & Amza, C. (2013). MixApart: Decoupled Analytics for Shared Storage Systems. USENIX FAST '13. https://www.usenix.org/conference/fast13/technical-sessions/presentation/mihailescu

[12] Jiao, Z. et al. (2024). The Design and Implementation of a Capacity-Variant Storage System. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/jiao

[13] Wu, K. et al. (2021). The Storage Hierarchy is Not a Hierarchy: Optimizing Caching on Modern Storage Devices with Orthus. USENIX FAST '21. https://www.usenix.org/conference/fast21/presentation/wu-kan

[14] Li, N. et al. (2022). Speculative Recovery: Cheap, Highly Available Fault Tolerance with Disaggregated Storage. USENIX ATC '22. https://www.usenix.org/conference/atc22/presentation/li-nanqinqin

[15] Cao, Z. et al. (2020). Characterizing, Modeling, and Benchmarking RocksDB Key-Value Workloads at Facebook. USENIX FAST '20. https://www.usenix.org/conference/fast20/presentation/cao-zhichao

[16] McKinney, S. M., Sieniek, M., Godbole, V., et al. (2020). International evaluation of an AI system for breast cancer screening. Nature, 577, 89–94. https://www.nature.com/articles/s41586-019-1799-6

[17] Chalapathy, R. & Chawla, S. (2019). Deep Learning for Anomaly Detection: A Survey. arXiv:1901.03407. https://arxiv.org/abs/1901.03407

[18] Lundberg, S. M., & Lee, S.-I. (2017). A Unified Approach to Interpreting Model Predictions. Advances in Neural Information Processing Systems, 30. https://arxiv.org/abs/1705.07874

[19] Cao, Z., Kuenning, G. & Zadok, E. (2020). Carver: Finding Important Parameters for Storage System Tuning. USENIX FAST '20. https://www.usenix.org/conference/fast20/presentation/cao-zhen

[20] Cidon, A. et al. (2013). Copysets: Reducing the Frequency of Data Loss in Cloud Storage. USENIX ATC '13. https://www.usenix.org/conference/atc13/technical-sessions/presentation/cidon

[21] Nachman, A., Yadgar, G. & Sheinvald, S. (2020). GoSeed: Generating an Optimal Seeding Plan for Deduplicated Storage. USENIX FAST '20. https://www.usenix.org/conference/fast20/presentation/nachman

[22] Alagappan, R. et al. (2019). Protocol-Aware Recovery for Consensus-Based Storage. USENIX ATC '19. https://www.usenix.org/conference/atc19/presentation/alagappan

[23] Yang, Z. et al. (2023). λ-IO: A Unified IO Stack for Computational Storage. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/yang-zhe

Published

2024-03-15

Similar Articles

1-10 of 134

You may also start an advanced similarity search for this article.