05 June 2025 15:00 - 15:20
LLM System Evaluation: The only moat in the GenAI Era
Strategic LLM evaluation offers a competitive edge by ensuring AI quality, facilitating rapid iteration, and streamlining model selection.
A robust framework, incorporating defined objectives, expert-reviewed golden datasets, and an evolving human-to-automated evaluation process, is the key.
This talk covers:
1. How to implement such a framework effectively within an organization,
2. How to leverage internal knowledge to build and maintain a robust evaluation pipeline
3. An overview of techniques for LLM evaluation and how they directly feed into improving your LLM based system