Hiring.Camp

Senior Inference Engineer

Careers Home

·

Oct 15, 2026

Salary
$200k – $350k
Location
USA - CA - Palo Alto, United States of America
Type
Full-time
Department
Engineering
Seniority
Senior
Source
Workday

Description

Uniphore is the Business AI company. Our sovereign, composable and secure AI platform connects enterprise data, fine-tunes AI models and deploys agentic AI across the enterprise. We empower every worker to boost productivity and help businesses grow faster, operate smarter and reduce costs. Trusted by more than 2,000 businesses globally, and recognized on the Deloitte Fast 500, Uniphore delivers on the promise of AI as a transformative force for business.


Job Description:
 

Staff Inference Engineer –

Uniphore is building an enterprise AI platform where agents operate across voice, video, text, and structured data in real time. We are looking for a Senior or Staff Inference Engineer to own the inference layer — combining deep model inference expertise with systems engineering to deliver a context-aware runtime serving finetuned SLMs across cloud, on-premises, and widely used GPU platforms.


This is a hands-on individual contributor role working closely with Uniphore’s research, model engineering, knowledge systems, agent, and pla4orm teams.

What You Will Build Context-aware runtime: Integrate context selection, compression, conversation compaction, and persistent task state with model execution. Adapt context budgets, model selection, and reasoning effort to each task.

KV-cache and computation optimization: Build paged KV-cache and prompt-prefix-cache management, cache-aware routing, retention, eviction, and offloading. Preserve correctness as context, permissions, model versions, and adapters change.

Inference performance: Implement and tune speculative decoding, aQention kernels, continuous batching, quantization, chunked prefill, and multi-GPU parallel execution.

SLM and adapter lifecycle: Build model and adapter lifecycle management, multi-adapter serving, zero-downtime hot swapping, and resource scheduling.

Production reliability: Implement observability (DCGM, Prometheus, Grafana), tenant isolation, overload protection, failure recovery, and safe rollout and rollback.

Cross-environment deployment: Build portable runtime integrations across NVIDIA and AMD GPU pla4orms, cloud and on-premises environments, and single- or multi-GPU configurations.


Technical Expertise

Model inference internals: Autoregressive generation, prefill vs. decoding, aQention architectures, compute- vs. memory-bound execution, numerical precision.

Speculative decoding: DraY-and-verify execution, acceptance/rejection sampling, adaptive speculation, and throughput tradeoffs.

GPU kernels: FlashAQention-style kernels, tiling, operator fusion, memory access paQerns.

CUDA, HIP, Triton, or CUTLASS proficiency.

KV and prefix caching: Paged KV storage, fragmentation, cache quantization, offloading, cross request prefix reuse, tenant isolation.

Context-window management: Token budgeting, context selection, prompt compression, conversation compaction, sliding window and sparse aQention.

Serving and scaling: Continuous batching, chunked prefill, admission control, quantization, multi-GPU parallelism, prefill/decode disaggregation.

 

Qualifications Required:

• 5+ years engineering, including 3+ years hands-on with GPU systems, distributed

inference, or ML serving

• Production experience with vLLM, SGLang, or TensorRT-LLM

• Strong Python and C++ (C++17/20)ti ability to read and debug CUDA C++ or HIP kernels

• Hands-on GPU profiling and performance optimization

 

Preferred:

• Custom GPU kernel development (CUDA, HIP, Triton, CUTLASS)

• PyTorch internals — torch. Compile, custom operators, MoE or disaggregated

prefill/decode

• LoRA/SLM adapter serving and zero-downtime hot swapping

• NVIDIA and AMD Instinct hardware in production

• Telemetry with DCGM, AMD SMI, Prometheus, and Grafana


Comp Range/OTE: 174- 218K USD


 


Hiring Pay Range:

$200,000 - $350,000


Benefits:


In addition to competitive base pay, this position also includes an annual incentive opportunity based on target achievement,  pre-IPO stock options, benefits including medical, dental, vision, 401(k) with a match, and more, plus generous paid time off, paid holidays, paid day off for your birthday and other paid leave policies to support employees through all phases of life.


Location preference:

USA - CA - Palo Alto

Uniphore is an equal opportunity employer committed to diversity in the workplace. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, disability, veteran status, and other protected characteristics.
 
For more information on how Uniphore uses AI to unify—and humanize—every enterprise experience, please visit www.uniphore.com.

Skills

PythonC++PyTorch

Similar Jobs

30

Senior Inference Engineer

techire ai·06·Remote

8mo ago

Sr. Inference Engineer (LLM Serving)

GoodData·Prague, Czechia·Hybrid

2d ago

Lead/Senior Inference Engineer

Thomson Reuters·Ann Arbor, Michigan +4·Remote

3d ago

Lead/Senior Engineer, Model Inference

Thomson Reuters·London, England +1·Hybrid

3d ago

Senior ML Systems Engineer - Productionization & Inference

Digitalturbine·Warsaw, Mazovia·Hybrid

3d ago

Senior Software Engineer Ai Foundry Core Bulk Inference

Accounts·Remote

Senior Software Development Engineer in Test - Akamai Inference Cloud (Remote/Poland)

Akamai·Poland·Remote

4d ago

Senior Software Engineer - Akamai Inference Cloud (Remote/Poland)

Akamai·Poland·Remote

4d ago

Architecture & Performance Modeling Engineer, AI Inference Silicon (Senior / Principal)

Positron·US·Remote

1w ago

Performance Modeling Engineer, AI Inference Deployments (Senior/Principal)

Positron·US·Remote

1w ago

Senior Software Engineer, Inference Performance

Tensordyne·Berlin, State of Berlin +1

1w ago

Senior AI Research Engineer - LLM Training & Inference Infrastructure

Datadog·Paris, Île-de-France·Hybrid

Sr. Software Engineer, Training/Inference (C++)

xAI·Palo Alto, California +1

2w ago

Senior / Staff AI Research Engineer, Real-Time Inference

Roboforce·Milpitas, California·Onsite

2w ago

Senior Staff Software Engineer - AI/ML Inference Engineering

ServiceNow·Kirkland, Washington·Hybrid

2w ago

Senior Staff Software Engineer - AI/ML Inference Engineering

ServiceNow·San Diego, California·Hybrid

2w ago

Senior Staff Software Engineer - AI/ML Inference Engineering

ServiceNow·Santa Clara, California·Hybrid

2w ago

Senior Machine Learning Engineer, Inference

Aida Projektai Mb·Mountain View, California·Remote, Onsite

2w ago

Senior Software Engineer — ML Inference & Numerics

Tensordyne·Berlin, State of Berlin +1

2w ago

Senior Inference Optimization ML Engineer

Rhoda Ai·Mountain View, California

2w ago

Senior AI/ML Engineer – SLM Optimization & Inference

Cnx·Nebraska, US·Remote

3w ago

(Senior/Junior AI Inference Reliability Engineer, Compute Infra

Together·San Francisco, California

3w ago

Senior Machine Learning Engineer, Inference

Inworld AI·Mountain View, California

4w ago

Senior Software Engineer, GPU Code Generation for Machine Learning Inference

Nvidia·Santa Clara, California +1·Remote

1mo ago

Senior Software Engineer, GPU Code Generation for Machine Learning Inference

Nvidia·New York City, New York +1·Remote

1mo ago

(Staff / Senior Staff) AI Infra Engineer, Inference Infrastructure

Together·San Francisco, California

1mo ago

Senior IA Inference Engineer (x/f/m)

Doctolib·Paris, Île-de-France·Hybrid

1mo ago

Senior II Platform Engineer Lead - Akamai Inference Cloud - Remote/Poland

Akamai·Poland·Remote

1mo ago

Senior Staff Inference Performance Engineer

D Matrix·Santa Clara, Villa Clara Province·Hybrid

1mo ago

Senior/Staff Back-End Software Engineer, Inference

Cohere·Montréal, Quebec +4·Remote

1mo ago