Skip to main content

GPU Lifetimes on Titan Supercomputer: Survival Analysis and Reliability

    Mallikarjun Shankar | Oak Ridge National Laboratory
    George Ostrouchov | Oak Ridge National Laboratory
    Don Maxwell | Oak Ridge National Laboratory
    James Rogers | Oak Ridge National Laboratory
    Rizwan Ashraf | Oak Ridge National Laboratory
Download Dataset on Globus

Description

George Ostrouchov, Don Maxwell, Rizwan Ashraf, Mallikarjun Shankar, and James Rogers. 2020. GPU Lifetimes on Titan Supercomputer: Survival Analysis and Reliability. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '20). Association for Computing Machinery, New York, NY, USA. Data and code for SC20 paper about Titan GPU reliability analysis: https://github.com/olcf/TitanGPULife. Includes R code to generate graphics for paper and additional analyses. See code/README for instructions. Includes original Titan GPU reliability data on over 100,000 collective hours of operation: data/titan.gpu.history.txt - history data, data/titan.service.txt - service nodes for exclusion. Includes output data files produced by code/TitanGPUmodel.Rmd: data/gc_full.csv - cleaned up data (see paper and R code); data/gc_summary_loc.csv - one record per GPU (variables: SN, time, nlife, nloc, last, col, row, cage, slot, node, max_loc_events, time_max_loc, dbe, dbe_loc, otb, otb_loc, out, batch, days, years, dead, dead_otb, dead_dbe) (see paper and R code). Includes .Rmd analysis document as TitanGPUmode.html. Includes Python code to process data/gc_full.csv into graphics from time-between-failure analyses: See code/tbf-analyses/README for instructions.

Funding Information

DOE Contract Number

DE-AC05-00OR22725

Originating Research Organization

Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization

Office of Science (SC), Advanced Scientific Computing Research (ASCR) (SC-21)

Details

Release Date

September 2, 2020

Subject

42 ENGINEERING, 47 OTHER INSTRUMENTATION

Keywords

GPU, reliability, supercomputer, NVIDIA, Cray, large-scale systems, Kaplan-Meier survival, Cox regression

Dataset

Dataset Type

ND Numeric Data

Software

R, python

Cite This Dataset:

Shankar, M., Ostrouchov, G., Maxwell, D., Rogers, J., Ashraf, R., Engelmann, C. (2020). GPU Lifetimes on Titan Supercomputer: Survival Analysis and Reliability. Oak Ridge National Laboratory. https://doi.org/10.13139/ORNLNCCS/1657202.

Acknowledgements

This work was carried out [in part] at Oak Ridge National Laboratory, managed by UT-Battelle, LLC for the U.S. Department of Energy under contract DE-AC05-00OR22725.