QA Engineer - AI Systems III

About Us:

Proofpoint is a global leader in human- and agent-centric cybersecurity. We protect how people, data, and AI agents connect across email, cloud, and collaboration tools. Over 80 of the Fortune 100, 10,000 large enterprises, and millions of smaller organizations trust Proofpoint to stop threats, prevent data loss, and build resilience across their people and AI workflows. Our mission is simple: safeguard the digital world and empower people to work securely and confidently. Join us in our pursuit to defend data and protect people.

How We Work:

At Proofpoint you’ll be part of a global team that breaks barriers to redefine cybersecurity guided by our BRAVE core values: 

Bold in how we dream and innovate

Responsive to feedback, challenges and opportunities

Accountable for results and best in class outcomes

Visionary in future focused problem-solving

Exceptional in execution and impact

We're looking for an AI Engineer who is passionate about both building AI-powered applications and ensuring they work reliably in production. This role combines AI engineering with quality engineering, focusing on developing LLM-based solutions while creating automated testing and evaluation frameworks to measure quality, accuracy, safety, and performance. You will work closely with engineering, product, and business teams to deliver AI solutions that are not only innovative, but also measurable, stable, and production-ready.

Responsibilities:
- AI Engineering: Build and maintain AI-powered applications using LLMs, RAG pipelines, and agent-based workflows.
- Develop integrations with foundation models and AI services using Python and modern AI frameworks.

- Design prompt workflows, structured outputs, tool calling, and retrieval-based systems.

- Build APIs, backend services, and internal tools using FastAPI or similar frameworks.

- Collaborate with stakeholders to translate business requirements into AI solutions.

- AI Quality & Automation Design and execute automated test suites for AI applications and agentic workflows.

- Create and maintain evaluation datasets, benchmark scenarios, and regression test cases.

- Build pipelines to measure response quality, hallucinations, accuracy, latency, and cost.

- Implement LLM-as-a-judge evaluations and automated scoring frameworks using tools such as DeepEval, RAGAS, or PromptFoo.

- Develop AI-powered test generation for functional, edge-case, and negative testing scenarios.

- Perform adversarial testing including prompt injection, jailbreak attempts, and safety validation.

- Monitor production AI systems through logging, tracing, and quality dashboards.

- Work with engineering teams to establish quality gates before AI features are released.

Required Qualifications:
- 2- 4 years of software engineering, QA automation, AI engineering, or related experience.

- Strong Python and Golang programming skills and understanding of software engineering fundamentals.

- Familiarity with LLM applications, prompt engineering, RAG, or agent-based AI systems.

- Experience in test automation using frameworks such as PyTest or equivalent tools.

- Understanding of API testing, backend services, and REST architectures.

- Knowledge of AI evaluation concepts such as hallucination detection, response quality measurement, benchmark testing, and regression testing.

- Experience with SQL and basic cloud platforms (AWS, Azure, or GCP).
- Strong analytical and problem-solving skills.

Why Proofpoint?

At Proofpoint, we believe that an exceptional career experience includes a comprehensive compensation and benefits package. Here are just a few reasons you’ll love working with us:

• Competitive compensation

• Comprehensive benefits

• Career success on your terms

• Flexible work environment

• Annual wellness and community outreach days

• Always on recognition for your contributions

• Global collaboration and networking opportunities

Our Culture:

Our culture is rooted in values that inspire belonging, empower purpose and drive success-every day, for everyone.

We encourage applications from individuals of all backgrounds, experiences, and perspectives. If you need accommodation during the application or interview process, please reach out to [email protected].

How to Apply

Interested? Submit your application along with any supporting information- we can’t wait to hear from you!