5:00 p. m. - 6:00 p. m. (UTC)

Join us for a comprehensive session exploring the integration of Weights & Biases (W&B) with GitOps in the context of Microsoft's Phi-2 language model. We'll delve into the intricacies of fine-tuning language models, employing W&B for tracking and analysis, and leveraging GitOps for enhanced continuous integration and delivery (CI/CD) in machine learning operations (MLOps). The workshop includes practical demonstrations on how to utilize W&B for meticulous tracking, analysis, and comparison of model performance. This technical session is designed to provide attendees with actionable insights and advanced knowledge for optimizing NLP models and MLOps workflows.

5:00 p. m. - 6:00 p. m. (UTC)

Evaluating LLM outputs accurately is critical to being able to iterate quickly on a LLM system. Human annotations can be slow and expensive and using LLMs instead promises to solve this. However, aligning a LLM Judge with human judgements is often hard with many implementation details to consider. In this workshop we will explore: Evaluating specialized LLMs using Weave Productionizing the latest LLM-as-a-judge research Improving on your existing judge Building annotation UIs