Impress employers and recruiters.
Choose from hundreds of resume examples.
Impress employers and recruiters.
Choose from hundreds of resume examples.
Target Resume Now
About the Role
Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. This role involves creating challenging tasks and evaluation criteria within realistic simulated environments to evaluate AI coding agents.
Responsibilities
- Build virtual companies with codebases, infrastructure, and context to create realistic development environments.
- Assemble and calibrate tasks by crafting prompts, defining evaluation criteria, and ensuring solvability and fairness.
- Design tasks within isolated environments emulating a developer's workstation.
- Write tests that accept all correct solutions and reject incorrect ones.
- Iterate with an AI agent on tests to verify their effectiveness.
- Review code written by agents, analyze performance, and design edge cases and adversarial scenarios.
- Iterate based on feedback from expert QA reviewers.
Requirements
- Degree in Computer Science, Software Engineering, or related fields.
- 5+ years in software development, primarily Python (FastAPI, pytest, async/await, subprocess, file operations).
- Background in full-stack development with experience building React-based interfaces (JavaScript/TypeScript) and back-end systems.
- Experience writing functional and integration tests.
- Experience with Docker containers and infrastructure tools (Postgres, Kafka, Redis).
- Understanding of CI/CD (GitHub Actions as a user).
- Comfortable reading and reasoning about code across the stack.
- Submit CV in English and indicate English proficiency level.
Skills
- Python
- FastAPI
- pytest
- async/await
- subprocess
- file operations
- React
- JavaScript
- TypeScript
- Docker
- Postgres
- Kafka
- Redis
- GitHub Actions
Location
- Remote
Work Type
- Project-based
- Part-time
- Non-permanent
Experience Level
- 5+ years in software development
- Experienced developers
- Software engineers
- Test automation specialists
Education Level
- Degree in Computer Science
- Degree in Software Engineering
- Degree in related fields
Salary/Compensations
- Up to $50 per hour equivalent
About the Company
- Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.