Explainable Anomaly Detection for HPC Storage during Computational Modelling

Authors

  • Muntasir Almomani Binghamton University (SUNY) Author

DOI:

https://doi.org/10.66687/d5kqnm63

Keywords:

Artificial Intelligence, Computer Simulation, Algorithms, Cloud Computing, Information Storage and Retrieval

Abstract

Scientific-computing storage experiences bursts of parallel reads and checkpoint writes that can resemble genuine infrastructure anomalies. The difficulty is particularly visible when a system is transferred to a new setting. To address this, the study will separate workload-driven congestion from storage faults while maintaining interpretable alerts. The research design centres on a trace-replay experiment using computational-model I/O patterns and injected failures under controlled concurrency. The choice of evaluation units and exclusion criteria is informed by the broader methodological record [3–7]. The analysis will contrast static queue thresholds, robust multivariate detectors and explanation-producing anomaly models. The design prioritises precise operational definitions and a straightforward baseline before introducing more complex alternatives. Related enterprise-storage studies inform the assessment of operational reliability [1] and the deployment assumptions adopted for system-level comparison [2]. Assessment will focus on tail latency, false alerts per workload, fault lead time and root-cause agreement with diagnostic logs. The supporting evidence base motivates the chosen comparator conditions and robustness criteria [8–12]. Critical sensitivity checks address cache warm-up, burst timing, shared-file interference and operator interpretation of alarms. Further systems and modelling literature supplies context for the proposed sensitivity analyses [13–17]. Primary analyses would be reported by the main deployment or participant subgroup so that averaged results do not conceal systematic failures. The expected output is a reusable evaluation framework with transparent uncertainty and reporting rules, informed by wider reproducibility literature [18–23].

References

[1] Mohammed Nazir. (2023). FlexGroup and FlexCache Optimization for AWS-Based CAE/HPC File Workloads: A Latency and Throughput Evaluation. International Journal of Business & Computational Sciences, 3(1), 25–31. https://ijbcs.org/index.php/IJBCS/article/view/2023-01-04

[2] Mohammed Nazir. (2021). Capacity Reclamation and Thin-Provisioning Efficiency in Large-Scale Enterprise Storage Systems. International Journal of Business & Computational Sciences, 1(1), 15–22. https://ijbcs.org/index.php/IJBCS/article/view/2021P4

[3] Cidon, A. et al. (2013). Copysets: Reducing the Frequency of Data Loss in Cloud Storage. USENIX ATC '13. https://www.usenix.org/conference/atc13/technical-sessions/presentation/cidon

[4] Maneas, S. et al. (2022). Operational Characteristics of SSDs in Enterprise Storage Systems: A Large-Scale Field Study. USENIX FAST '22. https://www.usenix.org/conference/fast22/presentation/maneas

[5] Vrable, M., Savage, S. & Voelker, G.M. (2012). BlueSky: A Cloud-Backed File System for the Enterprise. USENIX FAST '12. https://www.usenix.org/conference/fast12/bluesky-cloud-backed-file-system-enterprise

[6] Mihailescu, M., Soundararajan, G. & Amza, C. (2013). MixApart: Decoupled Analytics for Shared Storage Systems. USENIX FAST '13. https://www.usenix.org/conference/fast13/technical-sessions/presentation/mihailescu

[7] Jiao, Z. et al. (2024). The Design and Implementation of a Capacity-Variant Storage System. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/jiao

[8] Patel, T. et al. (2020). Uncovering Access, Reuse, and Sharing Characteristics of I/O-Intensive Files on Large-Scale Production HPC Systems. USENIX FAST '20. https://www.usenix.org/conference/fast20/presentation/patel-hpc-systems

[9] Cao, Z., Kuenning, G. & Zadok, E. (2020). Carver: Finding Important Parameters for Storage System Tuning. USENIX FAST '20. https://www.usenix.org/conference/fast20/presentation/cao-zhen

[10] Mahdisoltani, F., Stefanovici, I. & Schroeder, B. (2017). Proactive Error Prediction to Improve Storage System Reliability. USENIX ATC '17. https://www.usenix.org/conference/atc17/technical-sessions/presentation/mahdisoltani

[11] Yang, Z. et al. (2023). λ-IO: A Unified IO Stack for Computational Storage. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/yang-zhe

[12] Guo, C. et al. (2017). On Calibration of Modern Neural Networks. Proceedings of Machine Learning Research, 70, 1321–1330. https://proceedings.mlr.press/v70/guo17a.html

[13] Li, X. et al. (2023). ParaRC: Embracing Sub-Packetization for Repair Parallelization in MSR-Coded Storage. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/li-xiaolu

[14] Bai, W. et al. (2023). Empowering Azure Storage with RDMA. USENIX NSDI '23. https://www.usenix.org/conference/nsdi23/presentation/bai

[15] Wu, K. et al. (2021). The Storage Hierarchy is Not a Hierarchy: Optimizing Caching on Modern Storage Devices with Orthus. USENIX FAST '21. https://www.usenix.org/conference/fast21/presentation/wu-kan

[16] Li, N. et al. (2022). Speculative Recovery: Cheap, Highly Available Fault Tolerance with Disaggregated Storage. USENIX ATC '22. https://www.usenix.org/conference/atc22/presentation/li-nanqinqin

[17] Rodriguez, L.V. et al. (2021). Learning Cache Replacement with CACHEUS. USENIX FAST '21. https://www.usenix.org/conference/fast21/presentation/rodriguez

[18] Darban, Z.Z. et al. (2022). Deep Learning for Time Series Anomaly Detection: A Survey. arXiv:2211.05244. https://arxiv.org/abs/2211.05244

[19] Nachman, A., Yadgar, G. & Sheinvald, S. (2020). GoSeed: Generating an Optimal Seeding Plan for Deduplicated Storage. USENIX FAST '20. https://www.usenix.org/conference/fast20/presentation/nachman

[20] Alagappan, R. et al. (2019). Protocol-Aware Recovery for Consensus-Based Storage. USENIX ATC '19. https://www.usenix.org/conference/atc19/presentation/alagappan

[21] Mahmud, T. et al. (2023). ConfD: Analyzing Configuration Dependencies of File Systems for Fun and Profit. USENIX FAST '23. https://www.usenix.org/conference/fast23/presentation/mahmud

[22] Pang, G. et al. (2020). Deep Learning for Anomaly Detection: A Review. arXiv:2007.02500. https://arxiv.org/abs/2007.02500

[23] Wong, D.L.-K. et al. (2024). Baleen: ML Admission & Prefetching for Flash Caches. USENIX FAST '24. https://www.usenix.org/conference/fast24/presentation/wong

Published

2024-03-15

Similar Articles

1-10 of 121

You may also start an advanced similarity search for this article.