28 January 2027 14:30 - 15:00
Panel | Building Eval Pipelines That Actually Catch Regressions
An eval suite that passes every time doesn't necessarily mean the system's solid, it might just mean nobody's testing the right thing anymore.
Can you actually trust an AI system to judge itself, or does real rigor only come from keeping a human, or a much stricter process, in the loop?
This session gets practitioners who've actually built eval pipelines in production to compare what's working against what just looks good on paper.
Through real examples of regressions that slipped past a clean eval run, attendees get a clear picture of what a credible testing process actually requires.
→ Whether LLM as judge can actually be trusted, or if it's judging its own homework
→ Whether evals should be able to block a deploy, or just flag problems after the fact
→ What it takes to build a testing process rigorous enough to catch real regressions, not just obvious ones