AI Quality and Agent Operations Analyst - AI Solutions

NVIDIA pioneers computer graphics, gaming, AI, and accelerated computing. We are looking for an AI Quality & Operations Analyst to join our dedicated team and play a crucial role in improving Sales AI applications. This position offers the opportunity to work on innovative AI solutions as they move from development into production. You will collaborate with skilled team members to establish consistent evaluation, quality assurance, and operational practices across WWFO!
 

What you’ll be doing:

• Collaborate with application teams to replace ad hoc testing with consistent AI evaluation and quality assurance standards and workflows.

• Help establish test scenarios, assessment standards, quality metrics, and acceptance thresholds.

• Evaluate AI outputs for accuracy, relevance, consistency, safety, and alignment with business workflows.

• Analyze evaluation results, service metrics, production data, and user feedback to monitor application quality.

• Investigate failures, contributing factors, recurring patterns, and performance trends. Collaborate with technical teams on prioritized improvements and communicate findings to technical and business teams.

• Support production readiness and launches through testing, validation, documentation, monitoring, issue tracking, and quality checks before and after launch.

• Develop evaluation datasets, test scenarios, quality reports, and operational documentation that support continuous improvement.

• Assess risks from AI-generated outputs and downstream actions. Identify when additional controls, testing, or human review are needed.

What we need to see:

• 2+ years of experience in software quality, application operations, data analysis, AI or machine learning, technical support, business analysis, or a related field.

• Experience testing, evaluating, monitoring, or supporting applications in development or production.

• Familiarity with generative AI, machine learning concepts, APIs, data workflows, or enterprise software.

• Experience analyzing test results, application metrics, user feedback, or other data to identify issues and opportunities.

• Solid attention to detail and experience developing structured test cases, evaluation criteria, documentation, or quality reports.

• Ability to collaborate effectively with Engineering, Product, Data, Solution Architecture, and business teams.

• Ability to prioritize multiple assignments and support time-sensitive production readiness and launch activities.

• A bachelor’s degree or equivalent experience in Computer Science, Engineering, Data Science, Information Systems, or a related technical field is encouraged.

Ways to stand out from the crowd:

• Background with generative AI applications, AI assistants, retrieval-augmented generation, agentic workflows, prompt evaluation, model behavior, hallucination detection, or responsible AI practices.

• Experience developing evaluation datasets, benchmarks, test harnesses, quality scorecards, or automated testing approaches.

• Experience with Python, SQL, APIs, dashboards, data analysis, or quality monitoring tools.

• Background with enterprise application launches, incident management, post-launch operations, observability, service metrics, root-cause analysis, or operational troubleshooting.

NVIDIA is widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us. If you're creative and autonomous, we want to hear from you!

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 124,000 USD - 195,500 USD.Applications for this job will be accepted at least until August 31, 2026.

This posting is for an existing vacancy. 

NVIDIA uses AI tools in its recruiting processes.

NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.