Avetec

Advanced Virtual Engine Test Cell

Avetec Case Studies

Parallel File System Comparison Project

Overview

File system performance is a key component of user application efficiency in high performance computing (HPC) data centers. Storage administrators and data center directors must choose wisely to select the correct file system to meet their users’ requirements and properly adjust settings for optimal performance. Until recently, there has been no direct correlation between parallel file systems, and many of the decisions have been based upon vendor marketing. The DICE program, in partnership with the Department of Energy’s Sandia National Laboratories, is conducting a project to develop a comprehensive benchmarking framework for evaluating file systems.

Challenge

The DICE (Data Intensive Computing Environment) team’s challenge is to determine which file systems best meet data center users’ requirements. This is expected to aid data centers in configuring file systems for optimum performance.

Objective

This project is seeking to develop a framework for evaluating the performance of parallel file systems and to develop a normalization factor between different file system architectures. HPC community involvement is key to providing a viable tool and understanding all the factors for an accurate comparison, especially for file systems at large scale.

Results

Through a web-based survey conducted in late Spring of 2010, the DICE team gathered information from HPC data center representatives concerning parallel file systems, benchmarks and I/O measurement, and trace collection. The results and analysis of the survey questions will be available in a report at www.diceprogram.org/projects/pfs. Using the survey data, the team is currently building a benchmarking framework for evaluating file systems. This framework will include a searchable tool for determining proper benchmarks and a repository of benchmarks to be used for testing. The team is also planning to extend this work by developing a normalization factor between different file system architectures.

Ultimate Value

Through analyzing trace and benchmark data, a normalization factor and a comprehensive benchmark framework for evaluating file systems are being developed. This information will help storage administrators and data center directors determine the file system solution to meet their customers’ needs for user application efficiency.

 

Case Studies Home