Training 6: Practical Data and Evaluation Considerations for Building Production-Ready LLM Applications with LlamaIndex and Ray (3 hours)
Hands-on Lab
Software Engineer, ML Practitioner, ML Engineer
Ray Data, Train, Tune, ServeLarge Language Models (LLMs) are beginning to revolutionize how users can search for, interact with, and generate new content. Some recent stacks and toolkits around Retrieval-Augmented Generation (RAG) have emerged, enabling users to build applications such as chatbots using LLMs on their private data.
However, while setting up a naive RAG stack is straightforward, addressing a long tail of quality evaluation and scalability challenges is essential for software engineers to render their applications production-ready.
This hands-on training will guide you through using LlamaIndex and Ray to implement reliable evaluation methods for LLMs. You'll learn how to design experiments to optimize key application components and utilize scalable evaluation workflows to quantitatively compare them.
Concluding the training, you'll learn how to take the best-performing configuration and transition it into production, along with discussing the path toward continuous learning.
Learning Outcomes
- Gain an understanding of data-related design decisions and challenges when working on LLM-powered (RAG) applications.
- Acquire practical experience in using LlamaIndex and Ray to build scalable, robust, and reliable LLM software systems.
- Design reliable and evaluable workflows and run large tuning jobs with them to quantifiable decide the best version to productionize.
- Learn how to productionize an LLM application and gather the right feedback for fine-tuning
- Extend the experience to your own specific context, applications, stack and scale
Prerequisites
- Experience as a software engineer, ML practitioner and ML engineer
- Intermediate programming skills with Python
- Basic understanding of LLMs-enabled use cases
- Basic experience with LlamaIndex or Ray is helpful but not required
