Partnerships

Register now

Call to action
Your text goes here. Insert your content, thoughts, or information in this space.
Button

Back to speakers

Ryan
Kemmer
Machine Learning Engineer
Prime Video & Amazon Studios
Ryan Kemmer is an ML engineer at Amazon Prime Video, focused on agentic AI for search and content discovery. He's currently working on the architecture and evaluation of LLM-powered search systems, making it easy for customers to find something great to watch. Previously, he built generative brand-identity tools at Amazon, and before that worked as a data scientist researching fraud detection. He lives in the East Village of NYC.
Button
04 June 2026 16:00 - 16:30
Panel | Evaluating autonomous agents: Bridging the gap between testing and real-world performance
Evaluating autonomous agents is harder than static models or prompt-based systems. Their behavior unfolds over sequences, interacts with tools and environments, and can shift in live conditions in ways offline tests miss. In this panel, engineers share how they measure agent behavior, analyze trajectories over single outputs, and identify the signals that matter in real-world contexts, offering candid insights on what works, what doesn’t, and remaining evaluation challenges.