- Salary
- $25k – $75k/yr
- Location
- Bengaluru, Karnataka, IN
- Workplace
- Remote
- Type
- Internship
- Department
- Engineering
- Seniority
- Internship
- Source
- Y Combinator
Description
We companies build and launch public benchmarks that measure what AI can do in real-world domains. Our work includes ITSMBench, which evaluates whether AI models can autonomously resolve IT tickets. We’re looking for a intern to help build domain-specific benchmarks, develop tools that make benchmarking easier, and research ways to create evaluations that are fair, challenging, and useful.
What you’ll work on
- Help experts build realistic tasks and environments for domain-specific AI benchmarks.
- Develop tooling for running evaluations and investigating model failures.
- Research and test ideas for constructing fair and difficult benchmarks.
What we’re looking for
- Strong programming fundamentals and experience building software through coursework, personal projects, open-source contributions, or prior work.
- Interest in evals research, and understanding where models succeed or fail.
Experience with evaluation systems or research projects is a plus, but isn’t required.
There is a potential opportunity to join full time after three months.