Hiring.Camp

Principal AI Engineer - Agentic Systems and Evaluation Platform

Amgen is committed to unlocking

·

Apr 17, 2026

Location
India - Hyderabad
Workplace
Onsite
Type
Full-time
Department
Engineering
Seniority
Lead
Experience
10+ years
Education
Master
Source
Workday

Description

Career Category

Information Systems

Job Description

Job Description


About Amgen 

Amgen harnesses the best of biology and technology to fight the world’s toughest diseases, and make people’s lives easier, fuller and longer. We discover, develop, manufacture and deliver innovative medicines to help millions of patients. Amgen helped establish the biotechnology industry more than 40 years ago and remains on the cutting-edge of innovation, using technology and human genetic data to push beyond what’s known today. 

About the Role 

As a Principal AI Engineer, you will own the architecture, engineering, and ongoing operation of evaluations as a shared component of our enterprise AI platform. Your primary mandate is to build and manage the services, integrations, and infrastructure that enable teams to evaluate agentic AI systems consistently, securely, and at scale. 

You will establish a common evaluation capability across models, agent frameworks, and application types, with self-service interfaces, reusable framework integrations, and consistent management of evaluation configurations, datasets, runs, and results. Application teams and domain experts will use this capability to define and maintain the evaluation content and acceptance criteria for their use cases. 

As the hands-on technical lead for a small engineering team (~2-3 engineers), you will own the component roadmap and production lifecycle, partnering with platform, application, infrastructure, security, and governance teams. The role combines strong platform engineering with a deep understanding of how agentic systems are evaluated. 

What You Will Do 

  • Own the evaluation platform component from architecture and delivery through adoption, operations, and continuous improvement; define its roadmap, service boundaries, integration contracts, and engineering standards. 
  • Build self-service APIs, SDKs, developer integrations, and user interfaces for configuring evaluations, submitting and tracking runs, and accessing results within existing engineering workflows. 
  • Create an extensible integration layer for evaluation frameworks, scorers, model endpoints, and agent runtimes, enabling teams to use supported tools and add evaluators through consistent interfaces. 
  • Engineer reliable evaluation execution services with asynchronous orchestration, scheduling, concurrency management, retries, cancellation, and workload isolation across projects and teams. 
  • Provide versioning and lifecycle management for datasets, evaluation configurations, evaluator definitions, and result artifacts, with metadata and lineage that make runs reproducible and comparable. 
  • Support agentic evaluation through common contracts for inputs, outputs, traces, tool calls, and execution trajectories; provide controlled execution and replay patterns where appropriate. 
  • Build reusable evaluator components and configuration patterns for output quality, retrieval, tool use, task completion, safety, latency, and cost, while enabling application teams to supply domain-specific evaluation content. 
  • Integrate evaluation execution and results with CI/CD, experiment tracking, observability, and promotion workflows; enable configurable quality thresholds and evidence collection for release and governance review. 
  • Deliver consistent results schemas, comparison views, scorecards, and reporting APIs so teams can examine trends and compare model, prompt, and agent versions within project context. 
  • Embed authentication, authorization, project isolation, auditability, and data retention controls, working with security and governance partners to protect evaluation datasets, traces, and results. 
  • Operate the component as a production service, defining service objectives, monitoring reliability and execution performance, managing capacity and cost visibility, and leading incident resolution and upgrades. 
  • Drive adoption through documentation, onboarding patterns, reference integrations, and technical support; use platform telemetry and feedback to prioritize improvements and reduce integration effort. 

What We Expect of You 

  • Own evaluations as an enduring platform capability, with clear interfaces, dependable operations, and a roadmap informed by multiple teams. 
  • Turn recurring application requirements into reusable capabilities and extension points, balancing standardization with flexibility across agent frameworks and workloads. 
  • Measure success through platform adoption, service reliability, reproducibility, evaluation turnaround time, and developer experience. 
  • Provide hands-on technical leadership through architecture and code reviews, mentoring, and collaboration across engineering and governance teams. 

Basic Qualifications 

  • Bachelor's or master's degree in computer science, AI, or a related field 
  • 10+ years of experience in software engineering, AI engineering, ML systems, or platform engineering 
  • Strong Python and software engineering skills, including the design of production APIs, reusable libraries, and distributed backend services 
  • Experience building and operating shared platform services or infrastructure consumed by multiple applications or engineering teams 
  • Hands-on experience integrating AI evaluation frameworks, experiment tracking, or automated AI quality workflows into production engineering environments 
  • Understanding of LLM and agentic architectures, including orchestration, RAG, tool use, state management, and the evaluation data and traces these systems produce 
  • Experience with asynchronous processing, cloud services, CI/CD, observability, versioning, and secure access to enterprise systems 
  • Ability to define service contracts, make architecture tradeoffs, and lead engineering delivery and operations across teams 

Preferred Qualifications 

  • Experience owning an evaluation platform component, shared testing infrastructure, or an ML platform service through design, adoption, and production operations 
  • Experience integrating tools such as MLflow, RAGAS, DeepEval, or Promptfoo through reusable adapters and common configuration and results interfaces 
  • Experience designing plugin architectures, workload isolation, job orchestration, artifact storage, and versioned APIs or schemas 
  • Experience connecting agent traces, production feedback, and human review systems to shared evaluation infrastructure 
  • Familiarity with the strengths and limitations of deterministic checks, LLM-as-judge scoring, and human assessment, and their implications for platform design 
  • Experience with enterprise security and governance controls, usage metering, and capacity and cost management for AI services 

Soft Skills 

  • Strong technical leadership and mentoring, with ownership from architecture through production support 
  • Ability to translate needs across application teams into a coherent platform roadmap 
  • Clear communication of service capabilities, integration patterns, operating standards, and technical tradeoffs 
  • Practical engineering judgment and a strong focus on developer experience and service reliability 

Equal Opportunity Statement 

Amgen is an Equal Opportunity employer and will consider you without regard to your race, color, religion, sex, sexual orientation, gender identity, national origin, protected veteran status, or disability status.


.

Skills

PythonCI/CDLLMRAG

Similar Jobs

30

AI Engineer Principal

Gdit·US·Remote

Today

Principal AI Engineer

Mymoose·California, US·Remote

1d ago

Principal AI Engineer

Outsystems·US·Hybrid

1d ago

Principal AI Engineer

Gsk·India·Remote

2d ago

Principal AI Engineer

Mymoose·India

4d ago

Principal AI Engineer

Ecolab·India

2w ago

Principal AI Engineer

Ecolab·India

2w ago

Principal AI Engineer

Truvio·Berlin, State of Berlin +4·Remote

2w ago

Principal AI Engineer

Mdtkangaroo·Northridge, California +1

2w ago

Principal AI Engineer

Ecolab·India

2w ago

Principal AI Engineer

Hearst Magazines·Texas, US·Hybrid

3w ago

Principal AI Engineer

Wk·US·Hybrid

3w ago

Principal AI Engineer

Mastercard·India +1

1mo ago

Principal AI Engineer

Sedgwick is the world·US·Remote

1mo ago

AI Principal Engineer

Join Us·Malaysia

1mo ago

Principal AI Engineer

Qualified Health Pbc·Palo Alto, California·Hybrid

1mo ago

Principal AI Engineer

Netcompany·Athens, Attica·Hybrid

1mo ago

Principal AI Engineer

Aveva·Mumbai, Maharashtra

1mo ago

Principal AI Engineer

vertexinc·Pennsylvania, US·Remote

1mo ago

Principal AI Engineer

Innovaccer Analytics·San Francisco, California

Principal Engineer - AI

Circana·Pune, Maharashtra

1mo ago

Principal Engineer - AI

Circana·India +1

1mo ago

Principal Engineer, AI

Thomson Reuters·Bengaluru, Karnataka·Hybrid

1mo ago

Principal AI Engineer

Danaher·IND - Bangalore - Beckman Coulter India Private Limited·Onsite

1mo ago

Principal AI Engineer

Ecolab·India

2mo ago

Principal AI Engineer

Outsystems·Lisbon, Portugal

2mo ago

Principal AI Engineer

Obin Ai·NYC-Metro·Hybrid

2mo ago

Principal AI Engineer

Phizenix·Livermore, California·Hybrid

2mo ago

Principal AI Engineer

Velsera·New York City, New York

Principal AI Engineer

Mirakllabs·Bordeaux, Nouvelle-Aquitaine +1·Onsite

2mo ago