5:00 ÖS - 6:00 ÖS (UTC)

Evaluating LLM outputs accurately is critical to being able to iterate quickly on a LLM system. Human annotations can be slow and expensive and using LLMs instead promises to solve this. However, aligning a LLM Judge with human judgements is often hard with many implementation details to consider. In this workshop we will explore: Evaluating specialized LLMs using Weave Productionizing the latest LLM-as-a-judge research Improving on your existing judge Building annotation UIs

6:00 ÖS - 7:00 ÖS (UTC)

Join us for a technical deep dive into fine-tuning GPT models for agent-based tasks using Weights & Biases (W&B) and Azure OpenAI Service. Learn how to effectively track, evaluate, and monitor model performance, with practical insights on autonomous code generation and applications from the Meta Hacker Cup AI Track at NeurIPS 2024