Impress employers and recruiters.
Choose from hundreds of resume examples.

Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

About the Role

We are looking for a researcher to help develop the next generation of internal evaluations and research signals for post-training. This role spans evaluation creation, usability, correctness, auditing, agentic evaluation, and the development of efficient signals for research and training. You will work closely with researchers and engineers across post-training and the broader research organization.

Responsibilities

• Create internal evaluations and research signals for capabilities and behaviors important to model research and post-training.
• Develop usability evaluations that measure whether models are genuinely useful in real research and product workflows, and partner with the data flywheel to turn evaluation insights into better data and training signals.
• Improve evaluation correctness, including grader reliability, ambiguous ground truth, evaluator disagreement, false positives and negatives, and gaps between measured and intended behavior.
• Build eval onboarding and auditing methodologies that help researchers understand, trust, and appropriately use internal evaluation signals.
• Develop evaluations that are sensitive to meaningful improvements, robust to gaming, and capable of generalizing beyond the specific benchmark or setup in which they were developed.
• Develop specialized agentic evaluations, including supporting harness development and studying cross-harness and cross-environment generalization.
• Work with post-training scaling to develop signal-bearing core sets that provide efficient, high-quality measurements for internal RL research.
• Develop evaluations for personalized preferences, biases, values, and other nuanced dimensions of model behavior in collaboration with post-training crafting.

Requirements

• Bachelor’s degree or equivalent experience in Computer Science, Machine Learning, Physics, Mathematics, or a related discipline with strong theoretical and empirical grounding.
• Experience designing, building, or analyzing evaluations, benchmarks, datasets, graders, or other measurement systems.
• Strong written and verbal communication skills, with the ability to collaborate effectively across research and engineering teams.
• Experience with LLMs, post-training, reinforcement learning, or agentic systems.
• Experience building AI evaluations, graders, benchmarks, or internal research signals.
• Experience with evaluation correctness, auditing, human/LLM-based evaluation, or open-ended task evaluation.
• Experience with agentic evaluation, harnesses, long-horizon tasks, or cross-environment generalization.
• Experience evaluating preferences, personalization, biases, values, or other nuanced model behaviors.
• Track record of developing new evaluation methodologies or research signals that meaningfully influenced model development.
• Proficiency in Python and familiarity with at least one deep learning framework (e.g., PyTorch, TensorFlow, or JAX).
• Comfortable with debugging distributed training and writing code that scales.
• Strong research judgment: clean ablations, honest baselines, and clear technical writing.
• PhD in Computer Science, Machine Learning, Physics, Mathematics, or a related discipline with strong theoretical and empirical grounding; or, equivalent industry research experience.

Skills

• Python
• PyTorch
• TensorFlow
• JAX
• LLMs
• Post-training
• Reinforcement Learning
• Agentic Systems
• AI Evaluations
• Graders
• Benchmarks
• Internal Research Signals
• Evaluation Correctness
• Auditing
• Human/LLM-based Evaluation
• Open-ended Task Evaluation
• Agentic Evaluation
• Harnesses
• Long-horizon Tasks
• Cross-environment Generalization
• Personalization
• Biases
• Values
• Model Behaviors
• Distributed Training
• Debugging
• Research Judgment
• Technical Writing

Education Level

• Bachelor’s degree or equivalent experience in Computer Science, Machine Learning, Physics, Mathematics, or a related discipline with strong theoretical and empirical grounding.
• PhD in Computer Science, Machine Learning, Physics, Mathematics, or a related discipline with strong theoretical and empirical grounding; or, equivalent industry research experience.

About the Company

• The mission of Thinking Machines is to build AI that extends human will and judgment.
• We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication.
• We believe the future worth building is human, and we're hiring people who want to build it.