Building a scalable and fault-tolerant cloud architecture for the distributed execution of workflows
Due to the massive amount of data that accumulates during the collection of geospatial data, it is not possible to process the data on a single system. To solve this problem, numerous frameworks for distributing the workload to multiple computing nodes have been developed in the past years. However, most of the frameworks provide their own paradigm or even language to solve problems. Therefore it's often necessary to reimplement the algorithms of existing libraries. Furthermore, the use of a particular framework often leads to a vendor lock-in. Moreover, most of the tools, are designed to be executed on a static grid leaving the opportunities of modern cloud environments unused. This thesis proposes a highly scalable and fault tolerant solution to distribute work throughout a dynamically growing cluster of computing nodes. Unlike other solution, it does not force the user into a particular framework. It's a modular architecture which allows the integration of already existing frameworks to enable the execution of workows.
Gießen, TH Mittelhessen, Master Thesis, 2017