Member of Technical Staff: Python, TypeScript & LLM Agent Evals | On-site San Francisco | Up to $400K + Equity

"Good morning! And in case I don't see ya..."Truman Burbank, The Truman Show

Every AI agent thinks it lives in a perfect little world. Clean prompts, polite users, happy paths only. Sunny Seahaven, every day.

Your job is to sail out to the horizon and find where the sky is painted on.

The opportunity

A well-funded, seed-stage San Francisco startup is building the trust layer for AI agents, making sure they're safe and reliable before they're handed real responsibility in high-stakes industries. It's backed by top-tier AI investors, and the team is small, senior and stacked with people who have built companies before.

These agents are already in production at some of the most widely used AI products on the planet. Somebody has to find out how they break before the real world does. That's you.

This is a senior individual contributor seat with a rare mix: hardcore engineering, frontier AI research turned into product, and real contact with the customer teams building the agents everyone uses.

Downtown San Francisco after dark. Your new office is somewhere in that glow.

What you'll actually do

Own red-team engagements end to end on major customers' agents: scoping, recon, attack execution, remediation and sign-offSharpen the core engine that powers every evaluation the company runsTurn bleeding-edge adversarial research into repeatable productBe the palate for eval quality. If a test is weak, you send it back to the kitchenAutomate your own workflow with agents, because doing it twice by hand is a crimeWork out how to test the next wave, like computer-use agents, before anyone has a playbook for them

The tech you'll live in

Python is the engine room. Evals, attack pipelines and grading logic all run here, so you'll write production-grade Python fluently. That means async code that fires thousands of conversations at an agent in parallel, clean typed data models, solid tests, and the judgment to know when a script needs to grow up into a system.

TypeScript is where the work meets people. Customers need to see results, findings and reports, and you'll ship that user-facing side yourself. If you're strong in React and Node, typed end to end and built with the same care as the backend, you'll feel at home. This one is non-negotiable: they want engineers who already write both languages well, not people planning to learn one.

LLMs and agents are your raw material. You should be comfortable with the major model APIs, tool and function calling, structured outputs, retrieval, multi-turn agent loops, and protocols like MCP that connect agents to the outside world. Hands-on time with agent frameworks and orchestration is a big plus.

Evals are the craft. Designing test suites that actually catch failures. Using LLM-as-judge grading without fooling yourself. Building golden datasets, generating synthetic adversarial scenarios at scale, and proving whether a fix really worked or just moved the problem somewhere else.

The Bay Bridge at night. Every light is a test case.

Adversarial AI is the fun bit. Prompt injection, jailbreaks, tool misuse, data leakage, harmful outputs, and agents going off-script under multi-turn pressure. If you've read the latest red-teaming papers and itched to try them on a real production system, this is the job.

Integration is the glue. Every customer's agent is different, so you'll work across SDKs, API endpoints, webhooks and the occasional ugly legacy system, often getting a new integration running in days. Browser automation and computer-use tooling will matter more and more as agents start clicking around the web on their own.

Must have

Real production experience shipping user-facing features, not just backend plumbingStrong, hands-on Python and TypeScript. Both, not "happy to pick it up"5+ years as a software engineer, owning high-stakes technical work from start to finishA career arc that keeps climbing in title and scopeA CS degree or equivalent technical backgroundAgentic AI tools baked into how you work every dayClear communication with engineers and non-technical stakeholders alikeOn-site in San Francisco, 5 days a week

Bonus points

Founder, founding engineer or early startup hire (extra credit if it was VC-backed)Customer-facing experienceHands-on with agentic frameworks and evalsA visible spike: a side project people talk about, open source, writing, a standout company, or a bold career move that shows real conviction

Probably not for you if

Your whole career has been at big tech with no early-stage timeYou're an engineering manager who's drifted away from writing codeYou're after a management or director title. This is a senior IC seat, and that's the point

Halfway to stopping cable cars, all the way to stopping rogue agents.

Why this one is worth your time

You'll work on one of the biggest open problems in AI: knowing whether an agent can be trusted before it does something that mattersYour work lands in production with customers whose agents are used by millionsSmall team, huge surface area, and a direct line to the foundersThe category is brand new, so the playbook you write could become the industry standard

The package

Up to $400K base + meaningful equityOn-site in SF with a small, sharp, very senior teamVisa transfers welcome (OPT, H-1B transfers)A quick, focused process: a short vibe check, a coding round, then an onsite covering systems design, algorithms and a deep-dive on AI evals

How to apply

Hit apply with your resume, along with one line on the most interesting thing you've ever broken.

"We accept the reality of the world with which we're presented."Christof, The Truman Show

The agents accept it. You won't.