Principal Quality Assurance Engineer – Generative AI & LLM Platforms 1212256

Job Description:

 

We are seeking a Senior Quality Engineer to own quality strategy and hands-on validation for enterprise products built with Generative AI, large language models, distributed systems, and modern React-based user interfaces.

This role combines strong software testing fundamentals with AI evaluation expertise. The successful candidate will design automated quality frameworks, test deterministic and probabilistic behaviour, uncover complex cross-layer defects, influence architecture for testability, and help engineering teams deliver secure, reliable, accessible, high-performance products at speed.

 

Responsibilities:

• Define risk-based test strategies, quality gates, release criteria, traceability, and measurable objectives across AI, UI, API, cloud, and network layers.
• Evaluate model quality, groundedness, safety, privacy, robustness, tool use, permissions, failure recovery, latency, and cost.
• Validate React interfaces, web standards, accessibility, security, APIs, asynchronous workflows, and enterprise integrations.
• Test AWS deployments, distributed systems, traffic behavior, scaling, failover, disaster recovery, and adverse network conditions.
• Build reusable Python and pytest frameworks and execute performance, scale, reliability, and end-to-end testing with CI/CD integration.
• Use telemetry, incidents, and user feedback to detect regressions, strengthen coverage, and improve preventive controls.
• Partner across disciplines, use AI-assisted testing responsibly, mentor engineers, and promote shared ownership of quality.

 

Education and Experience Required:

• Bachelor's or Master's degree in Computer Science, Machine Learning, Artificial Intelligence, or a related discipline.
• 12+ years’ experience with at least 3 years of experience in a lead role

 

Knowledge and Skills:

• Quality Engineering: Enterprise test strategy, automation, quality governance
• Automation: Python, pytest, UI/API testing, mocking, diagnostics, CI/CD
• Web Testing: React, responsive design, accessibility, cross-browser, web security,
• API testing: JMeter, SoapUI, Postman
• GenAI Evaluation: LLM testing, groundedness, hallucination, safety, regression, metrics, human review, LangSmith, LangGraph, Langfuse, MCP
• Networking & Protocols: IP clos fabric, EVPN, VXLAN, BGP, MPLS, NETCONF, RESTCONF, gRPC, SNMP, LLDP, traffic simulators such as Ixia, Spirent etc…
• Performance Engineering: Performance, scale, functional, integration, E2E, regression, security, accessibility, reliability, compatibility
• Problem-Solving: Architecture analysis, cross-layer debugging, risk assessment, root-cause communication
• AI quality and observability: Experience with agent evaluation, prompt regression, model comparison, tracing, and production monitoring.
• Performance and resilience: Proficiency in API and browser performance testing, cloud-scale resilience, and chaos testing.
• Security and complex platforms: Knowledge of OWASP risks, threat modelling, adversarial validation, and testing multi-tenant or distributed enterprise architectures.
• AWS and monitoring: Familiarity with AI services, container platforms, and observability tools such as Datadog.
• Credentials and leadership: Relevant networking certifications and demonstrated leadership across automation, cloud quality, performance, security, or responsible AI.