Design and Evaluation of a Generic Orchestrator that executes an AI pipeline

The IT industry has seen significant adoption of microservice architecture in recent years. Microservices interact with each other using different protocols. SOAP was integrated with an ever-growing set of protocols called WS-I to promote interoperability. This large set of protocols with SOAP quickly became unpopular as it caused a considerable effort on the user side to make it interoperable. JSON replaced the XML as the serialization technology, and REST replaced SOAP as the communication protocol between web services. REST provided more flexibility, was more efficient and faster as compared to SOAP. However, it uses third-party tools such as swagger to auto-generate code for API calls in various languages, lacks support for streaming, needs semantic versioning whenever the API contract changes. This lead to its limited interoperability. As a variant of RPC architecture, Google created gRPC as a new communication protocol that solved most of the SOAP and REST issues. gRPC uses protocol buffers, usually called protobuf, for the serialization of data. It made it possible to clearly define the clean interfaces between services and supported in-built code generation for various programming languages. Automatic code generation makes it possible to use stubs and skeleton to call the services implemented on the server-side. In this thesis work, we use an open-source framework called Acumos, designed to make it easy to build, share, and deploy AI apps. Acumos has a design studio where users can compose an AI pipeline. Acumos has different orchestrators for different programming languages. However, there is no functionality to execute a generic pipeline that is implemented in multiple programming languages. Using gRPC communication, docker as a containerization tool, Kubernetes as a deployment environment, We propose to design a generic orchestrator capable of running any generic pipeline composed according to AI4EU container specification. This design of a generic orchestrator capable of executing pipelines, it has never seen before, is cutting edge and goes beyond state of the art.