Hiring.Camp

Machine Learning Engineer- Inference Optimization | Experienced Hire

Sig

·

May 28, 2026

Location
Bala Cynwyd (Philadelphia Area), PA, US
Department
Engineering
Closing date
Today
Source
iCIMS

Description

Overview

We are looking for a Machine Learning Engineer focused on low-latency inference optimization to help build, tune, and productionize high-performance model serving systems. This role sits at the intersection of machine learning, systems engineering, and GPU performance. You will work on inference workloads where latency, throughput, reliability, and hardware efficiency all matter, and where a deep understanding of modern inference runtimes can meaningfully improve production outcomes.

 

You will work closely with quantitative researchers and engineers to understand model structure, identify inference bottlenecks, and turn research ideas into efficient production systems. The work may involve other types of models, but focuses on transformer-style architectures, and structured inference workloads. You will evaluate and tune frameworks and related serving or compilation systems, while also reasoning about GPU execution, memory layout, batching strategies, precision tradeoffs, and end-to-end latency.

What you'll do

  • Design, build, and optimize low-latency inference systems for production machine learning workloads.
  • Profile model inference pipelines across model execution, runtime configuration, batching, memory movement, serialization, networking, and I/O.
  • Evaluate, integrate, and tune inference runtime systems.
  • Improve latency, throughput, GPU utilization, for production inference workloads.
  • Build and support benchmarking and profiling tools to compare model variants, hardware targets, runtime configurations, and deployment strategies.
  • Debug performance issues involving GPU memory, compute saturation, kernel behavior, CPU/GPU coordination, data movement, and serving-layer overhead.
  • Help shape model and system design choices so that research models are efficient to deploy under real latency constraints.
  • Where necessary, collaborate with lower-level systems or GPU specialists on custom operators, kernel-level optimization, or hardware-specific performance work.

What we’re looking for

  • Experience deploying, optimizing, or operating machine learning inference workloads in production or production-like environments.
  • Programming experience in Python, Java, C# etc. and at least one systems language such as C, C++, Rust, or Go
  • Solid understanding of modern ML frameworks such as PyTorch, including model execution, export, tracing, compilation, and performance profiling.
  • Ability to reason about latency, throughput, batching, memory use, GPU utilization, and reliability under real workloads.
  • Strong practical judgment around tradeoffs between model quality, latency, throughput, implementation complexity, and maintainability.

Preferred qualifications

  • Experience optimizing inference for latency-sensitive or high-throughput applications.
  • Experience with model optimization techniques such as quantization, pruning, distillation, operator fusion, graph lowering, custom operators, or model compilation.
  • Exposure to CUDA, Triton language, ROCm, PTX, CuTe, CUTLASS, FlashInfer, or similar low-level GPU programming tools.
  • Experience running inference workloads on Kubernetes or GPU clusters, including scheduling, autoscaling, observability, and resource management.
  • Background in mathematics, physics, computer science, engineering, statistics, quantitative finance, or another technical field.
  • Demonstrated ability to improve real-world inference performance beyond a baseline framework implementation.

 

If you're a recruiting agency and want to partner with us, please reach out to [email protected]. Any resume or referral submitted in the absence of a signed agreement will not be eligible for an agency fee.

Skills

PythonJavaRustKubernetesMachine LearningPyTorch

Similar Jobs

30

Machine Learning Engineer

Modjo·Paris, Île-de-France

Today

Machine Learning Engineer

Workday·Pleasanton, California·Hybrid

1d ago

Machine Learning Engineer

Weekday AI·Delhi, India·Onsite

1d ago

Machine Learning Engineer

Trainingbasket·Noida, Uttar Pradesh

1d ago

Machine learning Engineer

SIA·Maastricht, Limburg·Hybrid

3d ago

Machine Learning Engineer

SIA·Brussels, Brussels Capital·Hybrid

3d ago

Machine learning Engineer

SIA·Rotterdam, South Holland·Hybrid

3d ago

Machine Learning Engineer

SIA·Antwerp, Ohio·Hybrid

3d ago

Machine learning Engineer

SIA·Amsterdam, North Holland·Hybrid

3d ago

Machine Learning Engineer

Convin.AI·Bengaluru, Karnataka

5d ago

Machine Learning Engineer

Ford·Dearborn, Michigan·Hybrid

5d ago

Machine Learning Engineer

USA01 - USA Automotive·Dearborn, Michigan·Hybrid

5d ago

Machine Learning Engineer

Fabuwood·Newark, New Jersey

5d ago

Machine Learning Engineer

Claylabs·San Francisco, California·Hybrid

1w ago

Machine Learning Engineer

"Arena Intelligence, Inc."·SF Bay Area·Hybrid

1w ago

Machine Learning Engineer

"Voyager Technologies, Inc."·El Segundo, California

1w ago

Machine Learning Engineer

The Faculty·London, England·Hybrid

1w ago

Machine Learning Engineer

Poshmark·IN Tamil Nadu - Office

1w ago

Machine Learning Engineer

Endava·Córdoba, Cordoba·Hybrid

1w ago

Machine Learning Engineer

Endava·Rosario, Santa Fe·Remote

1w ago

Machine Learning Engineer

Cisco·India·Remote

1w ago

Machine Learning Engineer

Vitol·Houston, Texas·Onsite

1w ago

Machine Learning Engineer

Imc·Remote

1w ago

Machine Learning Engineer

PlayStation Global·Zaventem, Flanders·Remote, Hybrid

1w ago

Machine Learning Engineer

Experian·Sofia, Sofia-Capital

1w ago

Machine Learning Engineer

Coframe·SF Bay Area·Onsite

1w ago

Machine Learning Engineer

Arabic.AI·Riyadh, Riyadh Region

Machine Learning Engineer

The Faculty·Melbourne, Victoria +1·Hybrid

1w ago

Machine Learning Engineer

Averis·Kuala Lumpur, Malaysia

2w ago

Machine Learning Engineer

Central Insurance Company·Alpharetta, Georgia +4·Hybrid

2w ago