Platforms and Infrastructure to Operate GenAI in Your Company's Basement

In recent years, generative AI has revolutionized various industries by enabling the creation of highly sophisticated and creative outputs. However, the journey to harnessing the full potential of generative AI has just begun, especially for organizations opting to self-host these solutions. Unlike in the case where cloud resources from OpenAI, Microsoft Azure or their competitors are used, self-hosting requires extensive computational power, substantial data storage, and robust infrastructure - but also offers tempting benefits like data privacy and cost efficiency. In this article we discuss self-hosting of generative AI and report on the technical and operational hurdles involved. Additionally, we will provide detailed information on how we have built our own platform for multi-modal foundation models, offering insights into the necessary steps and considerations for successful implementation.