Modular hpc i/o characterization with darshan

S Snyder, P Carns, K Harms, R Ross… - 2016 5th workshop …, 2016 - ieeexplore.ieee.org
2016 5th workshop on extreme-scale programming tools (ESPT), 2016ieeexplore.ieee.org
Contemporary high-performance computing (HPC) applications encompass a broad range
of distinct I/O strategies and are often executed on a number of different compute platforms
in their lifetime. These large-scale HPC platforms employ increasingly complex I/O
subsystems to provide a suitable level of I/O performance to applications. Tuning I/O
workloads for such a system is nontrivial, and the results generally are not portable to other
HPC systems. I/O profiling tools can help to address this challenge, but most existing tools …
Contemporary high-performance computing (HPC) applications encompass a broad range of distinct I/O strategies and are often executed on a number of different compute platforms in their lifetime. These large-scale HPC platforms employ increasingly complex I/O subsystems to provide a suitable level of I/O performance to applications. Tuning I/O workloads for such a system is nontrivial, and the results generally are not portable to other HPC systems. I/O profiling tools can help to address this challenge, but most existing tools only instrument specific components within the I/O subsystem that provide a limited perspective on I/O performance. The increasing diversity of scientific applications and computing platforms calls for greater flexibility and scope in I/O characterization.In this work, we consider how the I/O profiling tool Darshan can be improved to allow for more flexible, comprehensive instrumentation of current and future HPC I/O workloads. We evaluate the performance and scalability of our design to ensure that it is lightweight enough for full-time deployment on production HPC systems. We also present two case studies illustrating how a more comprehensive instrumentation of application I/O workloads can enable insights into I/O behavior that were not previously possible. Our results indicate that Darshan's modular instrumentation methods can provide valuable feedback to both users and system administrators, while imposing negligible overheads on user applications.
ieeexplore.ieee.org
Showing the best result for this search. See all results