Freelance Agent Evaluation Engineer at Mindrift | DEU | Rezi

Freelance Agent Evaluation Engineer at Mindrift

Freelance Agent Evaluation Engineer

Mindrift · DEU

2 weeks ago

Freelance Agent Evaluation Engineer

Mindrift · DEU

21 days ago
Resume preview

Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

About the Role

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focusing on testing, evaluating, and improving AI systems. This role involves creating challenging tasks and evaluation criteria for AI coding agents within realistic simulated developer environments.

Responsibilities

  • Build realistic developer environments, including virtual companies with codebases, infrastructure, and context.
  • Design tasks from intermediate environment states, crafting prompts and defining solvability criteria for AI agents.
  • Write tests to verify AI agent solutions, accepting valid approaches and rejecting incorrect ones.
  • Iterate on tasks and tests based on QA feedback, refining them for fairness and robustness.

Requirements

  • 5+ years in software development
  • Experience writing tests (functional, integration)
  • English proficiency - B2+

Skills

  • Python (FastAPI)
  • JavaScript/TypeScript (React)
  • Docker
  • Postgres
  • Kafka
  • Redis

Location

  • Remote

Work Type

  • Project-based
  • Contract

Experience Level

  • 5+ years

Salary/Compensations

  • Up to $50/hr equivalent

About the Company

  • Mindrift connects specialists with project-based AI opportunities for leading tech companies.