Skip to main content

April 2020 Darshan counters from the Summit supercomputer

  • Karimi, Ahmad Maroof | Oak Ridge National Laboratory
  • Xie, Bing | Oak Ridge National Laboratory
  • Paul, Arnab K. | Oak Ridge National Laboratory
  • Oral, Sarp | Oak Ridge National Laboratory
  • Wang, Feiyi | Oak Ridge National Laboratory
Download dataset
Overview

Description

This dataset is the Darshan counters collected from the Summit supercomputer in a month of April 2020. 1. Description of methods used for collection/generation of data: Job submitted on Summit HPC system when completed successfully and has made I/O calls (captured by Darshan tool) writes a Darshan log file on alpine filesystem. One job can have multiple `jsrun` commands and Darshan will generate separate logs each log corresponding to an `jsrun` command, so a job can have one or more Darshan logs associated with it. 2. Methods for processing the data: To process the data, we first use `darshan-util` tool to parse the Darshan logs. Then we restructure the logs and merge data from multiple Darshan logs if they belong to the same Summit job.

Funding resources

DOE contract number

DE-AC05-00OR22725

Originating research organization

Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States)

Sponsoring organization

Office of Science (SC)

Details

DOI

10.13139/OLCF/1865904

Release date

May 3, 2022

Dataset

Dataset type

ND Numeric Data

Software

There are several ways to read CSV files, we recommend to use Python framework and pandas library.

Acknowledgements

Users should acknowledge the OLCF in all publications and presentations that speak to work performed on OLCF resources:

This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC05-00OR22725.

Category

  • 97 MATHEMATICS AND COMPUTING

Keywords

  • Supercomputer I/O subsystem,
  • Summit supercomputer,
  • Access Patterns,
  • Darshan log