Sign In
Register

Partner with us

Secure your seat

Call to action
Your text goes here. Insert your content, thoughts, or information in this space.
Button

Back to speakers

Jeffrey
Pincus
Data Science Manager | Syndication Insights
Comcast
Jeffrey Pincus is a Data Science Manager at Comcast NBCUniversal, where he leads a team of six data scientists focused on using AI and data science to drive strategic priorities and build better products. Previously, he spent four years at General Motors, leading Test & Learn initiatives and developing econometric and AI/ML models to forecast customer behavior and evaluate business strategies. His earlier experience includes data science and analytics roles at Booz Allen Hamilton, Technomics, and AECOM, working across AI/ML, experimentation, automation, and large-scale data analysis.
Button
28 January 2027 14:30 - 15:00
Panel | Building Eval Pipelines That Actually Catch Regressions
An eval suite that passes every time doesn't necessarily mean the system's solid, it might just mean nobody's testing the right thing anymore. Can you actually trust an AI system to judge itself, or does real rigor only come from keeping a human, or a much stricter process, in the loop? This session gets practitioners who've actually built eval pipelines in production to compare what's working against what just looks good on paper. Through real examples of regressions that slipped past a clean eval run, attendees get a clear picture of what a credible testing process actually requires. → Whether LLM as judge can actually be trusted, or if it's judging its own homework → Whether evals should be able to block a deploy, or just flag problems after the fact → What it takes to build a testing process rigorous enough to catch real regressions, not just obvious ones