- Location
- Doha, Doha Municipality, Qatar · published
- Type
- Contract
- Department
- IT
- Seniority
- Senior
- Source
- Workable
Description
We are looking for a Senior QA Engineer to own quality across our AI products. You will test LLM-powered features end to end, build automated test and evaluation suites, and help ensure every release is accurate, safe, and reliable.
Responsibilities
• Define test strategies and plans for AI features across web, API, and backend.
• Evaluate model outputs for accuracy, consistency, hallucinations, bias, and safety.
• Build evaluation datasets, including edge cases and adversarial / red-team prompts.
• Develop and maintain automated UI, API, and regression tests integrated into CI/CD.
• Run regression testing on model, prompt, and configuration changes.
• Test performance, latency, error handling, and fallback behavior.
• Report defects clearly and provide release-readiness sign-off.
• Test non-deterministic outputs using scoring rubrics, LLM-as-judge, human evaluation, and defined pass/fail thresholds.
• Perform AI security testing, including prompt injection, jailbreaks, and PII / data leakage across chatbots, RAG, and agents.
• Work with ML engineers and product managers to define quality metrics and acceptance criteria.
• Where applicable, monitor production quality and model drift after release.