Simplified Training, Tuning and Deployment of Foundational Models as a Service with Red Hat OpenShift AI

Emerging AI/ML workflows, like the end-to-end cycle to train and deploy Foundation Models, are increasingly complex tasks with wide-ranging compute and data requirements. In this fundamental paradigm shift, a single and typically very large model is trained and adapted to work on many different specific tasks. From data cleaning and preparation, to large-scale distributed training, to fine-tuning and validation, to scalable inferencing, the current reality is that working with foundation models is a remarkably complex task, involving fragmented software tooling that often requires extensive expertise to be deployed and operated at large scale. In this talk, we will show how we simplified the end-to-end life cycle of foundation models with a cloud-native, and scalable stack for training, fine- and prompt-tuning, and inferencing, realized with Red Hat OpenShift Data Science (RHODS). We will give an overview of how we are introducing new open source components, like CodeFlare SDK, Multi-cloud App Dispatcher, and InstaScale and integrating with Ray and PyTorch to enable large scale data preparation, training and validation. Once models are validated, we will show how our inference stack, based on ModelMesh and KServe, can be used to deploy models in production with RHODS model serving, and how we use Data Science Pipelines to orchestrate models, including versioning and tracking in production deployment. We will also show how we operate this stack, from public cloud to on-premise, and how we are leveraging it to enable and accelerate the value of foundation models in a range of use cases, including success stories highlighting the benefits of this full stack for end-to-end life cycle of foundation models.