Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

Tailor your resume to this Member of Technical Staff, Evaluations role.

Rezi rewrites your resume against Stealth - AI x Bio's job description. Free.

Resume score gauge reading 58 out of 100

Don't guess if your resume is good enough.

See how it scores against the Member of Technical Staff, Evaluations posting at Stealth - AI x Bio — free, in seconds.

About the Role

Own the evaluation of AI models and agents, including biological models, agent planning, and execution. Develop benchmarks, design evaluation strategies, and manage training and evaluation data. Collaborate with scientists and engineers to ensure data quality and model performance.

Responsibilities

Own model and agent evaluation, covering biological models, agent planning, and execution.
Translate scientist workflows into benchmarks for long-horizon scientific tasks.
Develop checks to determine the correctness and data support of model outputs.
Run internal benchmarks to compare models and harnesses.
Design benchmarks that remain challenging as models improve.
Protect benchmarks against leakage, memorization, and gaming.
Report evaluation results with associated uncertainty.
Trace unexpected result shifts to their root cause (model, harness, or evaluation).
Own the data used for model training and evaluation.
Source external datasets and prepare internal experimental data.
Build tools for researchers to understand model failures.
Ensure evaluations account for noise, are easily rerunnable, and well-documented.
Report into the intelligence team and collaborate with RL, Bio ML, and infrastructure engineers, and scientists.

Requirements

Experience building evaluations, benchmarks, or task suites for language models, agents, or scientific models.
Proven ability to identify issues in results that others have overlooked.
Tendency to review traces before trusting scores and treat unexpected high results as potential bugs.
Experience with building evaluations, benchmarks, or task suites for language models or agents, including scoring code.
Experience with long-horizon agent tasks utilizing tools like search, databases, and code execution.
Experience detecting reward hacking, grader exploits, data contamination, and label leakage.
Proficiency in statistics for noisy results, including repeats, uncertainty, and determining significant differences between runs.
Experience with model-based grading and rubric design, including calibrating LLM judges against expert ratings.
Hands-on experience with evaluation frameworks such as Harbor, Inspect, or Prime Intellect's verifiers.
Experience building evaluations that rerun reproducibly.
Clear writing and communication skills regarding evaluation design, results, and limitations.
Strong Python skills and daily use of coding agents.

Skills

Python
Agent coding
Evaluation frameworks (Harbor, Inspect, Prime Intellect)
Statistics
Model-based grading
Rubric design
LLM calibration
Benchmark design
Data sourcing
Data preparation
Tool building
Communication

Location

London
San Francisco

Work Type

Full-time

Experience Level

Mid-level
Senior

Benefits

30 days of annual leave plus local public holidays
UK: Pension with 10% employer contribution
UK: Bupa private health cover
US: 401(k) with matching
US: Health cover

About the Company

Stealth startup in AI and bio, building infrastructure for data generation.
Focusing on producing critical data that does not yet exist.
Building infrastructure with quality and traceability from day one.
Team is small and elite, working on hard problems.
Foundations are being laid now, with opportunities for significant ownership.

Equal Opportunity

We are an equal opportunity employer.
Hiring decisions are made on merit, scope-fit, and the strength of the working relationship.
Applications are welcome from candidates of any background.