Impress employers and recruiters.
Choose from hundreds of resume examples.

Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

About the Role

Contra Labs is seeking a Project Operator to manage research projects focused on how creatives use AI. This role bridges the gap between clients, creative experts, and engineering teams, ensuring rigorous and useful results for AI labs and product companies. You will own the methodology, delivery, quality, analysis, systems, and narrative for client engagements.

Responsibilities

• Define methodology, including annotation protocols, rubrics, and evaluation designs for creative and multimodal outputs.
• Manage project delivery, including scope, evaluator staffing, quality routing, timelines, and client relationships from kickoff to final readout.
• Ensure data quality and inter-rater reliability, making real-time adjustments when necessary.
• Conduct qualitative and quantitative analysis, translating findings into actionable reports and recommendations.
• Develop and refine processes, partnering with engineering on tooling to improve efficiency and reliability.
• Create case studies and benchmark publications to enhance Contra Labs' market presence.

Requirements

• Based in NYC.
• In office 5 days a week (Williamsburg).

Skills

• Study design
• Evaluation execution
• Data analysis
• Presentation of findings
• Data labeling
• Annotation
• Human-in-the-loop systems
• AI and ML evaluation workflows
• Managing concurrent projects
• Understanding of creative quality
• Process optimization
• Qualitative research
• Quantitative research
• Statistical methods for evaluation work
• RLHF
• LLM evaluation
• Benchmark design
• Ethnography
• UX research
• Design-study methods

Location

• NYC

Work Type

• Onsite

Experience Level

• 3+ years designing studies, running evaluations, analyzing data, and presenting findings.
• Experience with data labeling, annotation, human-in-the-loop systems, or AI and ML evaluation workflows.
• A track record managing several concurrent projects, including annotation, evaluation, or QA work.
• Startup or 0-to-1 experience.

Education Level

• Master's or PhD in HCI, cognitive science, psychology, design research, or a related field is a plus, and so is equivalent depth from industry.

Salary/Compensations

• $150,000 to $200,000 USD plus equity

Benefits

• Medical, dental, vision
• 401k matching
• Company laptop on your start date

About the Company

• Contra Labs is a human-centered research lab studying creative AI usage and tool development.
• We connect creative talent with AI labs and product companies, grounding research in real creative workflows.
• Our work powers benchmarking, evaluation, and post-training for leading models and applications.
• Built on Contra, the professional network for independent creatives, we use human judgment to train and evaluate AI tools.

Equal Opportunity

• Contra communicates with applicants through contra.com domains only.
• We never ask candidates for money.
• For current postings, visit Contra Careers.