Analysis and Evaluation of Visual Data Exploration Algorithms for Big Data

Nowadays in industrial and internet process, the data volume is getting much bigger than ever. The distributed file system and data process platform Hadoop brings various possibilities and conveniences for data process and analysis. In addition, more and more visualization tools sprung up. How to efficiently take advantage of these tools and possibilities are becoming the hot topic. This thesis aims at analyzing and evaluation of Big Data algorithms, which supports a visual Data exploration platform, which is on the basis of a Big Data Kappa Architecture. The platform should automatically analyze and visualize data. The analysis algorithms should offer the basic evaluation functions for any numerical data, which should be without a-priory expert knowledge. The algorithms should be able to detect and evaluate the data format of each data column. After automatic analysis of data content and format the platform needs to make automatic evaluations for data reduction. With suitable web based visualizations user should be able to interactively navigate within results. The typical data processing consists of five steps: data acquisition, data storage, data analysis, data visualization and data exploration. This thesis focuses on analysis, visualization and exploration. Suitable algorithms need to be researched and tested for each of these steps. The evaluation of the algorithms takes place on known data sets for which the domain knowledge is present and the information contained is known.