BentoML or RayServe, You Can Choose Both with BentoRay

Ray and BentoML are two popular libraries in the machine learning ecosystem, both beloved by developers for their unique strengths. Ray is particularly appreciated for its powerful architecture, which enables developers to parallelize and scale Python code with ease, without having to worry about the complexities of distributed computing. Meanwhile, developers love BentoML for its simplicity in developing, managing, and collaborating on AI applications. With BentoML, developers can build AI applications once and deploy them anywhere, without having to modify any code. The industry is now seeking a solution that can seamlessly integrate both libraries, enabling developers to streamline ML model training and AI application development.

In this talk, we will introduce BentoRay, a new project that enables developers to effortlessly run BentoML workloads (Bentos) on the RayServe architecture leveraging Ray's resource management and autoscaling capabilities in a Ray cluster. We will dive deep into the technical details of how BentoML's distributed runner architecture is transformed into Ray's deployment graph, and how BentoML's dependency environment is transparently mapped and bootstrapped in Ray's runtime environment. We will also demonstrate the simple steps involved in moving from a BentoML single container deployment to a distributed microservice based BentoRay deployment in a Ray cluster, allowing developers to seamlessly scale their workloads all without having to modify any code.

Overall, attendees will gain a comprehensive understanding of how BentoRay can help streamline the deployment of machine learning models and applications on Ray's powerful distributed computing framework. With BentoRay, developers can leverage the best of both worlds, combining the simplicity and collaboration capabilities of BentoML with the power and scalability of Ray.