Hier finden Sie wissenschaftliche Publikationen aus den Fraunhofer-Instituten.

MapReduce in GPI-space

: Rotaru, T.; Rahn, M.; Pfreundt, F.-J.


Euro-Par 2013. Parallel Processing Workshops : BigDataCloud, DIHC, FedICI, HeteroPar, HiBB, LSDVE, MHPC, OMHI, PADABS, PROPER, Resilience, ROME, and UCHPC 2013, Aachen, Germany, August 26-27, 2013; revised selected papers
Berlin: Springer, 2014 (Lecture Notes in Computer Science 8374)
ISBN: 978-3-642-54419-4 (Print)
ISBN: 978-3-642-54420-0 (Online)
ISBN: 3-642-54419-3
International Conference on Parallel Processing (Euro-Par) <19, 2013, Aachen>
Workshop on Big Data Management in Clouds (BigDataCloud) <2, 2013, Aachen>
Conference Paper
Fraunhofer ITWM ()

The computing power of modern high performance systems cannot be fully exploited using traditional parallel programming models. On the other hand, the growing demand for processing big data volumes requires a better control of the workflows, an efficient storage management, as well as a fault-tolerant runtime system. Trying to offer our proper solution to these problems, we designed and developed GPI-Space, a complex but flexible software development and execution platform, in which the data coordination of an application is decoupled from the programming of the algorithms. This allows the domain user to focus on the implementation of its problem only, while the fault tolerant runtime framework automatically runs the application in parallel in complex environments. We discuss the advantages and the disadvantages of our approach by comparison with the most popular MapReduce implementation, Hadoop. The tests performed on a multicore cluster with the wordcount use case showed that GPI-Space is almost three times faster than Hadoop when strictly the execution times are considered, and more than six times faster when the data loading time is also considered.