Skip to main content

April 2020 Darshan counters from the Summit supercomputer

    Ahmad Maroof Karimi | Oak Ridge National Laboratory
    Bing Xie | Oak Ridge National Laboratory
    Arnab K. Paul | Oak Ridge National Laboratory
    Sarp Oral | Oak Ridge National Laboratory
    Feiyi Wang | Oak Ridge National Laboratory
Download Dataset on Globus

Description

This dataset is the Darshan counters collected from the Summit supercomputer in a month of April 2020. 1. Description of methods used for collection/generation of data: Job submitted on Summit HPC system when completed successfully and has made I/O calls (captured by Darshan tool) writes a Darshan log file on alpine filesystem. One job can have multiple `jsrun` commands and Darshan will generate separate logs each log corresponding to an `jsrun` command, so a job can have one or more Darshan logs associated with it. 2. Methods for processing the data: To process the data, we first use `darshan-util` tool to parse the Darshan logs. Then we restructure the logs and merge data from multiple Darshan logs if they belong to the same Summit job.

Funding Information

DOE Contract Number

DE-AC05-00OR22725

Originating Research Organization

Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring Organization

Office of Science (SC)

Details

Release Date

May 3, 2022

Subject

97 MATHEMATICS AND COMPUTING

Keywords

Supercomputer I/O subsystem, Summit supercomputer, Access Patterns, Darshan log

Dataset

Dataset Type

ND Numeric Data

Software

There are several ways to read CSV files, we recommend to use Python framework and pandas library.

Cite This Dataset:

Karimi, A., Xie, B., Paul, A., Oral, S., Wang, F. (2022). April 2020 Darshan counters from the Summit supercomputer. Oak Ridge National Laboratory. https://doi.org/10.13139/OLCF/1865904.

Acknowledgements

This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Advanced Scientific Computing Research programs in the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.