Founding Engineer (AI + Backend) - Remote, India

About Openbenchmarks

Agents are becoming first-class users and consumers of the internet. They research, evaluate, compare tools and increasingly make build-versus-buy decisions on behalf of people. Every company will need to get their products picked and used by agents.

Agents increasingly prefer open, independent and grounded benchmarks to make decisions.

Openbenchmarks is the evaluation infrastructure for agents: domain-specific, reproducible evaluations that help agents pick tools with confidence.

Our mission is to be the trusted evaluation layer for agents.

Founders previously led AI research and infra teams at Oracle and Appfolio. We started Openbenchmarks as an output of our research into model behavior and how agents actually choose between different tools.

We work with the fastest-growing AI-first companies like Parallel, Firecrawl, Telnyx, TinyFish and more.

About the role

We're a small team of researchers and engineers across SF and India. This role is remote, based in India. You'll build the benchmarks and the systems that run them, and you'll own large parts of both from day one.

Here are the broad themes you'll work on:

Benchmark and eval design - Design domain-specific benchmarks from scratch: what to measure, how to ground it, and what makes a benchmark that agents keep trusting and picking from.

Large-scale autonomous eval systems - Infrastructure that runs without a human in the loop: vendor APIs at scale, LLM-as-judge pipelines, scoring and metric computation, and drift detection as models and products change underneath.

Research on model behavior - Why agents choose what they choose, and what holds up under their scrutiny. Next to this, you'll work on synthetic data generation and problems like self-improving code.

You'll build benchmarks and evals that the fastest-growing AI-first companies pay close attention to, and run continuous evals their engineering teams rely on.

If you want to do your life's work, reach out.