April 2020 Darshan counters from the Summit supercomputer
- Karimi, Ahmad Maroof | Oak Ridge National Laboratory
- Xie, Bing | Oak Ridge National Laboratory
- Paul, Arnab K. | Oak Ridge National Laboratory
- Oral, Sarp | Oak Ridge National Laboratory
- Wang, Feiyi | Oak Ridge National Laboratory
Overview
Description
This dataset is the Darshan counters collected from the Summit supercomputer in a month of April 2020. 1. Description of methods used for collection/generation of data: Job submitted on Summit HPC system when completed successfully and has made I/O calls (captured by Darshan tool) writes a Darshan log file on alpine filesystem. One job can have multiple `jsrun` commands and Darshan will generate separate logs each log corresponding to an `jsrun` command, so a job can have one or more Darshan logs associated with it. 2. Methods for processing the data: To process the data, we first use `darshan-util` tool to parse the Darshan logs. Then we restructure the logs and merge data from multiple Darshan logs if they belong to the same Summit job.
Funding resources
DOE contract number
DE-AC05-00OR22725Originating research organization
Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)Sponsoring organization
Office of Science (SC)Details
DOI
10.13139/OLCF/1865904Release date
May 3, 2022Dataset
Dataset type
ND Numeric DataSoftware
There are several ways to read CSV files, we recommend to use Python framework and pandas library.Acknowledgements
Users should acknowledge the OLCF in all publications and presentations that speak to work performed on OLCF resources:
This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.
Category
- 97 MATHEMATICS AND COMPUTING
Keywords
- Supercomputer I/O subsystem,
- Summit supercomputer,
- Access Patterns,
- Darshan log