About the Role
Sampura Research is an AI safety non-profit focused on scalable oversight for frontier models. We are designing novel judge protocols and building high-quality evaluations to advance the science of judges. Backed by an $11M grant, we were founded in 2026 by members of DeepMind’s alignment and human data teams. As a founding Member of Technical Staff, you will help shape research and engineering direction and own end-to-end research and engineering problems.
Responsibilities
- Build diverse and high-quality evaluations for judge protocols by pulling from existing datasets/literature and creating novel datasets from scratch.
- Research and implement strategies for improving judge performance through hybridization, such as learning-to-defer, human assistance, or other mechanisms.
- Build the tools required to enable and scale our research agenda, including large-scale inference/evaluation pipelines, a human rating platform, and a robust public leaderboard.
- Scope ambiguous problems independently.
- Communicate progress clearly.
- Synthesize findings into papers, blog posts, and benchmark releases.
- Lead research and mentorship programs through paid fellowships or volunteer research focused on evaluation or methods work.
Requirements
- A Bachelor's degree or higher in Computer Science, Mathematics, or a related field or equivalent experience.
- Proficiency coding in Python or similar languages, as well as machine learning and analysis tooling (e.g., JAX, PyTorch, Pandas).
- Experience working with large language models, human-in-the-loop data annotation systems, or AI evaluation frameworks.
- A willingness to embrace AI tools intelligently, applying scrutiny and human review where necessary.
- A track record of exploring and resolving open-ended research questions with rigor, in academic or industry labs, fellowships, or volunteer research programs.
- Demonstrated ownership across multiple parts of the AI research lifecycle (experiment design and execution, infrastructure, data collection and processing, inference).
- Expertise in a highly relevant domain such as model confidence/calibration, interpretability, safety/alignment evaluation, human-computer interaction, or other fields related to scalable oversight.
- Hands-on experience with post-training methods such as reinforcement learning, adversarial fine-tuning, or distillation, especially applied to judges or reward models.
- Leadership as a mentor or team lead for related research, either inside a research lab or via programs like SPAR, MARS, or MATS.
Skills
- Python
- JAX
- PyTorch
- Pandas
- Large language models
- Human-in-the-loop data annotation systems
- AI evaluation frameworks
- AI tools
- Model confidence/calibration
- Interpretability
- Safety/alignment evaluation
- Human-computer interaction
- Scalable oversight
- Reinforcement learning
- Adversarial fine-tuning
- Distillation
- Judges
- Reward models
Location
- London
Work Type
- On-site
- Full-time
Experience Level
- Junior
- Senior
- Staff
Education Level
- Bachelor's degree or higher in Computer Science, Mathematics, or a related field or equivalent experience.
Salary/Compensations
- Junior hires: £100,000-£150,000 (~$136,000-$205,000)
- Senior hires: £150,000-£200,000 (~$205,000-$273,000)
- Staff hires: £200,000-£290,000 (~$273,000-$395,000)
Benefits
- Visa sponsorship
About the Company
- Sampura Research is an AI safety research non-profit focused on scalable oversight for frontier models through improved human/AI “judges”.
- Judges (LLM, human, or a hybrid) are essential for aligning highly capable models through their use in evaluation and training.
- Limitations in today's judges can lead to undesirable behavior such as reward hacking in training, or failure to spot misalignment in evaluation.
- As models become more capable and their data more complex, often beyond what the systems overseeing them can understand, we expect these limitations to grow in severity and to become harder to notice.
- We are designing novel judge protocols which leverage the strengths of both humans and models to outperform either alone, and building high-quality evaluations to explore judge performance across a diverse set of safety-related tasks.
- We believe that our focused approach will rapidly advance the science of judges by providing a much-needed systematic measurement of which protocols actually work, and producing state-of-the-art judges which can be used in real world training and evaluation pipelines.
- Backed by an $11M grant from Coefficient Giving ($7M for the first year, and another $4M pledged), Sampura Research was founded in 2026 by members of DeepMind’s alignment and human data teams.
Equal Opportunity
- Sampura Research is an equal opportunity employer.
- We are committed to building a team with diverse backgrounds and perspectives and believe this makes our research and workplace better.
- We do not discriminate on the basis of race, religion or belief, sex, sexual orientation, gender reassignment, age, disability, marriage or civil partnership, pregnancy or maternity, or any other protected characteristic.
- If you need adjustments at any stage of the application process, let us know and we'll make them.
