BucDoop: Bottom Up Computation of Iceberg Data Cubes with Hadoop

Abstract

Big Data analysis has been a key matter during the recent years for the study of various phenomena in various science contexts as well as in business intelligence. Furthermore it appears for good reason to remain in focus for the future. Online Analytical processing methods and Data Cubes need to be further studied in order to reduce time used for efficient data analysis. This study introduces BucDoop, a novel algorithm that exploits the parallelism benefits of Hadoop Map Reduce, for the efficient iceberg data cube creation in reasonable time. BucDoop includes the use of the Bottom Up Computation (BUC) idea in the context of iceberg cube data lattice traversal, managing to reduce the amount of data handled with early pruning architecture and producing the portion of the cube needed for analysis purposes (iceberg problem). Experiments conducted herein present an efficient scalability factor for the creation of the iceberg cube for very big data, by-passing the data explosion and memory constraints problem while using only commodity hardware.