This role owns how AI and LLM capabilities are applied across YouGov Sport’s data products, ensuring solutions are robust, well-governed, and delivering real client value.

Rather than building models, this person acts as a smart client, challenger, and orchestrator across AWS, third-party partners, and internal teams. They define what “good” looks like, stress test outputs, ensure data integrity, and shape how AI is deployed across products and workflows.

This is a product-first AI role, focused on validation, application, and scalability — not engineering.

🧩 Core responsibilities

1. AI product ownership & use case definition

• Define and prioritise AI/LLM use cases across platforms (e.g. QuickSight-powered tools, Theia)
• Translate client and commercial needs into clear AI product requirements
• Define expected behaviours, boundaries, and outputs for AI features
• Ensure features are grounded in real data constraints and use cases

2. Testing, evaluation & quality assurance (CRITICAL FOCUS AREA)

• Design structured test frameworks for LLM-powered features
• Develop and maintain evaluation datasets and benchmark queries
• Introduce repeatable / automated evals where possible

3. Vendor & partner leadership (AWS + third parties)

• Act as a confident, informed counterpart to AWS and AI vendors
• Push for better outputs, clearer explanations, and improved performance
• Coordinate delivery across:
  • AWS (e.g. QuickSight AI capabilities)
  • third-party LLM partners (e.g. Theia)
  • internal stakeholders

4. Data alignment & governance

• Ensure AI tools pull from the correct datasets, respect access boundaries, avoid restricted or irrelevant sources; work with internal data experts to validate logic and outputs
• Define guidelines for trustworthy and explainable outputs

5. Strategic AI capability development

• Identify where AI can add value across internal and other products
• Build a repeatable approach to deploying AI
• Shape longer-term scalability, governance, and reuse of components

6. Cross-functional collaboration & client alignment

• Work closely with commercial teams (client-facing insight), product teams; data & BI specialists
• Conduct user interviews and feedback loops
• Ensure AI outputs are understandable, relevant, and usable by clients

Essential

• Experience as a Product Manager / Lead for AI or data products
• Strong understanding of LLM capabilities and limitations in real-world applications
• Proven experience:
  • defining use cases
  • writing product requirements
  • working with engineers and vendors
• Experience designing or contributing to:
  • test cases / evaluation approaches
  • quality validation frameworks
• Strong critical thinking and challenge mindset
• Experience working with external technology partners
• Comfortable interfacing with client-facing/commercial teams

Highly desirable

• Experience with:
  • BI or data platforms (e.g. dashboards, query tools)
  • AWS ecosystem (even if not hands-on)
• Exposure to:
  • conversational interfaces / chat agents
  • computer vision / image recognition (for Theia)
• Experience creating:
  • LLM evaluation sets
  • guardrails / constraints for AI systems

Why join YouGov?
Join our global team to help us achieve our social mission: to make millions of people’s opinions heard for the benefit of our local, national, and international communities.

Understanding diversity of opinion requires diversity of background. Although our global panel of millions of people worldwide powers our research, our biggest asset is our people. If our research is to be truly representative of what the world thinks, we need people from all walks of life to be part of the team to bring their perspective to the work we do. 

Life at YouGov
We are driven by a set of shared values. We are fast, fearless, and innovative. We work diligently to get it right. We are guided by accuracy, ethics, and proven methodologies. We respect and trust each other, bringing these values into everything that we do.

We strive to provide YouGovers with best-in-class benefits to support their physical, financial, and emotional wellbeing. We want our employees to have a sense of belonging and uniqueness in a supportive workplace, so they can bring their full selves to work. 
 
Equal Opportunity Employer
As an Equal Opportunity Employer, qualified applicants will receive consideration for employment without regard to race, color, religion, sex (including pregnancy, gender identity or expression, and sexual orientation), parental status, national origin, marital status, age, disability, genetic information, HIV status, political affiliation, socioeconomic background, veteran status or any other characteristic protected by law or in line with our responsibilities as a fair and ethic employer. All employment decisions are made based on occupational qualifications, merit, and business need.

Data Privacy
To find out how we collect and use your personal data when you apply for a role at YouGov, please read our privacy notice at https://jobs.yougov.com/privacy