Weights & Biases

5:00 p.m. - 6:00 p.m. (UTC)

Join us for a comprehensive session exploring the integration of Weights & Biases (W&B) with GitOps in the context of Microsoft's Phi-2 language model. We'll delve into the intricacies of fine-tuning language models, employing W&B for tracking and analysis, and leveraging GitOps for enhanced continuous integration and delivery (CI/CD) in machine learning operations (MLOps). The workshop includes practical demonstrations on how to utilize W&B for meticulous tracking, analysis, and comparison of model performance. This technical session is designed to provide attendees with actionable insights and advanced knowledge for optimizing NLP models and MLOps workflows.

5:00 p.m. - 6:00 p.m. (UTC)

Evaluating LLM outputs accurately is critical to being able to iterate quickly on a LLM system. Human annotations can be slow and expensive and using LLMs instead promises to solve this. However, aligning a LLM Judge with human judgements is often hard with many implementation details to consider. In this workshop we will explore: Evaluating specialized LLMs using Weave Productionizing the latest LLM-as-a-judge research Improving on your existing judge Building annotation UIs

6:00 p.m. - 7:00 p.m. (UTC)

Join us for a technical deep dive into fine-tuning GPT models for agent-based tasks using Weights & Biases (W&B) and Azure OpenAI Service. Learn how to effectively track, evaluate, and monitor model performance, with practical insights on autonomous code generation and applications from the Meta Hacker Cup AI Track at NeurIPS 2024