Hiring.Camp

AI Research Scientist – Scientific Evaluation (ML/AI)

Lightly AG

·

2 weeks ago

Salary
$50 – $65/hr
Location
Switzerland
Workplace
Remote
Type
Contract
Department
Education
Education
PhD
Source
JOIN

Description

Lightly AG is a Zurich-based AI company and ETH/HSG spin-off, backed by Y Combinator and top-tier investors. Our machine learning and computer vision technology is trusted by global leaders in autonomous driving, medical imaging, and visual inspection.

We’re looking for experienced ML/AI researchers with a strong publication track record at top-tier conferences (NeurIPS, ICML, ICLR) to support an advanced AI research evaluation project. You'll critically assess scientific papers and research outputs generated by autonomous AI research agents, evaluating their technical correctness, scientific novelty, experimental rigor, and overall research quality against the standards of leading ML/AI conferences.

This is a remote, project-based contractor opportunity with flexible working hours.

Tasks

What you'll be doing

  • What you'll be doing

  • Critically evaluate AI-generated ML/AI research papers, assessing their scientific contributions, technical correctness, methodology, experimental design, and conclusions

  • Assess scientific novelty and significance, determining whether proposed approaches represent meaningful advancements over existing state-of-the-art research

  • Evaluate experimental rigor and validity, including baseline comparisons, benchmark selection, evaluation metrics, statistical evidence, and ablation studies

  • Identify methodological flaws, unsupported claims, inconsistencies, or misleading results that undermine the scientific validity of proposed contributions

  • Assess reproducibility and technical soundness, including whether reported findings are sufficiently supported by experimental evidence and, where available, accompanying code and implementation details

  • Review and verify relevant academic literature, identifying overlooked prior work, inaccurate citations, and unsupported novelty claims

  • Evaluate whether AI-generated research meets the scientific standards of top-tier ML/AI conferences, such as NeurIPS, ICML, and ICLR

  • Provide structured, evidence-based assessments of research quality, including strengths, weaknesses, critical technical concerns, and overall scientific merit

  • Apply detailed evaluation rubrics consistently, providing clear, well-reasoned justifications for your assessments

The project focuses on evaluating advanced AI-generated scientific research, with an emphasis on technical validity, scientific novelty, experimental rigor, and overall research quality. Your role is to provide expert human judgment on whether the resulting research is scientifically valid, genuinely novel, and of a quality comparable to work accepted at leading ML/AI conferences.

Requirements

You're a strong candidate if you:

  • Have a PhD in Machine Learning, Artificial Intelligence, Computer Science, Statistics, or a closely related technical field, or equivalent demonstrated research expertise

  • Have published at least 3 peer-reviewed research papers at NeurIPS, ICML, and/or ICLR (hard requirement), ideally including first-author or significant research contributions

  • Have substantial hands-on ML/AI research experience, including developing novel methods, designing experiments, analyzing results, and publishing original scientific work

  • Have a strong understanding of scientific methodology and experimental rigor, including appropriate baselines, statistical validity, benchmark design, ablation studies, and reproducibility

  • Can independently assess the novelty, significance, and technical soundness of advanced ML/AI research, identifying both meaningful scientific contributions and fundamental methodological weaknesses

  • Have experience reviewing research papers for top-tier ML/AI conferences or journals, ideally NeurIPS, ICML, or ICLR — strongly preferred

  • Are highly familiar with state-of-the-art ML/AI research and can critically assess new methods against existing literature and established benchmarks

  • Are comfortable verifying scientific claims, including reviewing technical arguments, experimental evidence, citations, and, where relevant, code implementations

  • Have strong analytical and scientific writing skills and can articulate nuanced technical judgments through concise, evidence-based feedback

  • Can consistently apply detailed evaluation guidelines and scoring rubrics while exercising independent expert judgment

  • Have exceptional attention to detail, particularly when identifying subtle methodological flaws, unsupported conclusions, or discrepancies between claims and evidence

Benefits

  • Fully remote and flexible — work from anywhere

  • Part-time contractor role with flexible hours

  • Work on the evaluation of cutting-edge autonomous AI systems capable of conducting end-to-end scientific research

  • Apply your research expertise to help assess and advance the scientific capabilities of frontier AI agents

Send us your CV along with a brief note about your research background, areas of expertise, and publication record. Please include links to at least 3 peer-reviewed publications at NeurIPS, ICML, and/or ICLR, as well as any relevant academic peer-review experience.

If applicable, we'd also love to know which ML/AI research areas you're most familiar with and whether you've served as a reviewer, area chair, or in a similar capacity at major research conferences.

We look forward to hearing from you!

Skills

Machine LearningComputer Vision

Similar Jobs

30

AI Research Scientist

MakiPeople·London, England +1·Hybrid

4d ago

AI research scientist

Writer·New York City, New York +1·Hybrid

5d ago

AI Research Scientist

Trainingbasket·Noida, Uttar Pradesh

3w ago

AI Research Scientist

hum.ai·San Francisco, California·Remote

3w ago

AI Research Scientist

Nvidia·Seoul, South Korea

3w ago

AI Research Scientist

Nvidia·Seoul, South Korea

3w ago

AI Research Scientist

Tanium·Addison, Illinois +1·Hybrid

3w ago

AI Research Scientist

Abglobal·Nashville, Tennessee

1mo ago

AI Research Scientist

Pacific Defense·Fairfax, Virginia

AI Research Scientist

The Zebra·GBR - London, UK·Hybrid

1mo ago

AI Research Scientist

Embedding Vc·Milpitas, California·Onsite

3mo ago

AI Research Scientist

Sprinter Health·California, US +1·Hybrid

4mo ago

Research Scientist - AI

Nvidia·Singapore

4mo ago

AI Research Scientist

Nvidia·Singapore·Remote

6mo ago

AI Research Scientist

Webai·Austin, Texas·Onsite

6mo ago

Research Scientist, AI

Biohub·Redwood City, California·Hybrid

7mo ago

AI Research Scientist

Nvidia·Singapore·Remote

AI Research Scientist

Pangram Labs·Brooklyn, New York·Onsite

9mo ago

Research Scientist (AI)

Physical Superintelligence·Boston, Massachusetts +1·Hybrid

9mo ago

AI Research Scientist

Webai·Austin, Texas·Remote

10mo ago

Research Scientist, AI

Substrate·San Francisco, California·Onsite

1y+ ago

AI Research Scientist

Collate·San Francisco, California·Onsite

1y+ ago

Research Scientist Ai Safety And Alignment Deepmind

Accounts·Remote

Research Scientist – AI for Digital Circuit Design of Image Sensors

Sony·SEU - Switzerland - Schlieren·Hybrid

5d ago

Research Scientist – AI for Digital Circuit Design of Image Sensors

Sonyglobal·SEU - Switzerland - Schlieren·Hybrid

5d ago

Staff/Sr. Staff/Principal AI research scientist

Writer·New York City, New York +1·Hybrid

5d ago

Frontier Ai Research Scientist Deepmind

Accounts·Remote

Research Scientist, AI and Systems Security - PhD New College Grad 2026

Nvidia·Santa Clara, Villa Clara Province +1

1w ago

Research Scientist, AI and Systems Security - PhD New College Grad 2026

Nvidia·Santa Clara, California +1

1w ago

Audio AI Research Scientist — Audio Source Separation

Bose Career Site·Australia +11·Remote

1w ago