Sign In
Register

Partnerships

Request your invite

Call to action
Your text goes here. Insert your content, thoughts, or information in this space.
Button

Back to speakers

Nikolai
Liubimov
Co-Founder & CTO
HumanSignal
Nikolai Liubimov is the Co-founder and CTO of HumanSignal, the company behind Label Studio, where he leads the development of data-centric tooling that powers machine learning workflows across industry and research. With a PhD in computer science and over a decade building speech, language, and deep learning systems at companies like Yandex and Huawei, he has deep experience in designing scalable human-in-the-loop infrastructure. His work focuses on bridging the gap between human judgment and AI automation - building annotation, data creation, and curation platforms that enable teams to train safer, more reliable, and more performant models.
Button
02 December 2025 11:30 - 12:00
Measuring what matters: Benchmarking and human-in-the-loop for reliable generative AI
When it comes to deploying generative AI in the enterprise, two questions loom large: Can we trust this model in production? and How do we keep it aligned with human values over time? In this 30-minute talk, we’ll explore two critical themes: how to measure model quality effectively and how to embed human-in-the-loop (HITL) systems into your pipelines. You’ll learn: Why standard benchmarks fall short, and how to design evaluations that reflect your real-world data. Practical approaches to testing and monitoring models before (and after) they go live. How HITL feedback loops can capture nuance, enforce quality, and drive continuous improvement. Best practices for balancing automation with human oversight to build trust at scale. Through real-world examples and practical guidance, you’ll leave with a concise toolkit for evaluating and governing GenAI systems that’s rigorous enough for the enterprise and flexible enough for fast-moving teams.