Versatile Communication Algorithms for Data Analysis

Authors: 
T. Peterka and R. Ross
Name of Publication: 
Proceedings of EuroMPI’12
Type of Publication: 
Conference Proceedings
Abstract: 
Large-scale parallel data analysis, where global information from a variety of problem domains is resolved in a distributed memory space, relies on communication. Three communication algorithms motivated by data analysis workloads--merge based reduction, swap based reduction, and neighborhood exchange--are presented, and their performance is benchmarked. These algorithms communicate custom data types among blocks assigned to processes in flexible ways, and their performance is optimized by tunable parameters. Performance is compared with an MPI implementation and with previous communication algorithms on an IBM Blue Gene/P supercomputer at a variety of message sizes and process counts.
URL of Published Paper: 
http://dx.doi.org/10.1007/978-3-642-33518-1_33
Conference Location: 
Vienna, Austria
Page Numbers: 
275-284
DOI: 
10.1007/978-3-642-33518-1_33
Published Date: 
September, 2012