Training 3: Learn How to Build and Productionize LLM-Powered Applications with Ray and Anyscale (3 hours)
9:00 AM - 12:00 PM
Level: Beginner to Intermediate
Hands-on Lab
Software Developer, ML Engineer, Data Scientist
Ray Data, ServeLearn to build LLM-powered apps with Python and learn to scale and deploy them with Ray using open-source models.
In this workshop, you'll learn the why as well as the how, as you gain hands-on experience implementing common designs for LLM systems including the retrieval-augmented generation ("RAG") patterns that power the most successful AI apps.
We'll cover all of the fundamentals as well as demystify the more advanced operational elements like dealing with large datasets (think: an archive of customer service interactions or insurance claims), autoscaling on GPUs, and more.
Learning Outcomes
- Learn to build AI applications that can answer questions intelligently from rich context
- More importantly, learn where these patterns came from and why, by starting from the basics in Python without libraries
- Understand when, why, and how to move from remote APIs where others host the model (e.g., OpenAI) to your own systems with open models
- Learn how to move from a local prototype to a production-grade scalable implementation with Ray
- Discover tools from the LLMs ecosystem -- and how to think about scaling them -- including vector databases, embedding models, and model optimization libraries
Prerequisites
- Basic familiarity with user-facing functionality of language models like chatbots, answerbots, or translation apps
- Intermediate programming skills with Python
- Basic experience with Ray is helpful but not required
- No knowledge of language model internals is required
