Impress employers and recruiters.
Choose from hundreds of resume examples.
Impress employers and recruiters.
Choose from hundreds of resume examples.
About the Role
The Agent Transformation Accelerator (ATA) Product team is building Meta’s future of work: Metamate, an internal full-stack platform that lets people direct, review, and help agents improve their work across the entire company. This role owns how we measure and raise the quality of agentic systems, shaping the approach from the ground up in a space where the right metrics don't yet exist. You'll define what "good" means for agents and own the eval verdict that gates model upgrades, harness changes, and major launches. Expect to be constantly learning, working shoulder-to-shoulder with engineering, data science, and ML, including our partners across ATA and Meta Superintelligence Labs (MSL), to turn signal from real usage into a continuous improvement loop that makes the product measurably better every week.
Responsibilities
- Own the end-to-end feedback and improvement system: collection, analysis, prioritization, and resolution.
- Define how we measure agent quality across dimensions (accuracy, helpfulness, reliability, safety).
- Build product mechanisms that capture implicit and explicit user signal at scale.
- Partner with ML and data science to turn feedback into model improvements, eval frameworks, and training data.
- Drive the evals and measurement strategy: what "good" looks like, how we detect regressions, and how we track progress.
- Create transparency and accountability for quality across the ATA Product team.
Requirements
- Strong quantitative skills and experience defining complex metrics.
- Comfort with ambiguous problem spaces where the right metrics do not yet exist.
- 5+ years of relevant industry experience with at least 2 years in Product Management.
- Experience partnering closely with data science and ML engineering teams.
- Track record of driving product improvements through data and experimentation.
- Direct experience building evals for AI products end to end: task set construction, rubric design, instrumentation, and analysis.
- Experience running eval-driven development cycles, with a clear view of where evals are informative and where they mislead.
- Willingness to get into the weeds of eval data and rubrics, with high standards for eval rigor and an obsession with quality.
- Deep understanding of LLM capabilities and failure modes.
Skills
- AI tools
- Prompt/context engineering
- Agent orchestration
- Risk assessment
- Bias mitigation
- Quality and accuracy reviews
Experience Level
- 5+ years of relevant industry experience
- 2+ years in Product Management
Education Level
- Bachelor's degree (or relevant degree equivalent): STEM subject ideal but not essential (Computer Science, Engineering, Information Systems, Analytics, Mathematics, Physics, Applied Sciences)
Salary/Compensations
- $146,000/year to $204,000/year + bonus + equity + benefits