April 2020 Darshan counters from the Summit supercomputer
-
Ahmad Maroof Karimi | Oak Ridge National Laboratory
Bing Xie | Oak Ridge National Laboratory
Arnab K. Paul | Oak Ridge National Laboratory
Sarp Oral | Oak Ridge National Laboratory
Feiyi Wang | Oak Ridge National Laboratory
Description
This dataset is the Darshan counters collected from the Summit supercomputer in a month of April 2020. 1. Description of methods used for collection/generation of data: Job submitted on Summit HPC system when completed successfully and has made I/O calls (captured by Darshan tool) writes a Darshan log file on alpine filesystem. One job can have multiple `jsrun` commands and Darshan will generate separate logs each log corresponding to an `jsrun` command, so a job can have one or more Darshan logs associated with it. 2. Methods for processing the data: To process the data, we first use `darshan-util` tool to parse the Darshan logs. Then we restructure the logs and merge data from multiple Darshan logs if they belong to the same Summit job.
Funding Information
DOE Contract Number
DE-AC05-00OR22725Originating Research Organization
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)Sponsoring Organization
Office of Science (SC)Details
Release Date
May 3, 2022Subject
97 MATHEMATICS AND COMPUTINGKeywords
Supercomputer I/O subsystem, Summit supercomputer, Access Patterns, Darshan logDataset
Dataset Type
ND Numeric DataSoftware
There are several ways to read CSV files, we recommend to use Python framework and pandas library.Cite This Dataset:
Karimi, A., Xie, B., Paul, A., Oral, S., Wang, F. (2022). April 2020 Darshan counters from the Summit supercomputer. Oak Ridge National Laboratory. https://doi.org/10.13139/OLCF/1865904.
Acknowledgements
This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Advanced Scientific Computing Research programs in the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.