Impress employers and recruiters.
Choose from hundreds of resume examples.

Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

About the Role

Crimson Education is the world’s leading college admissions consulting firm, helping ambitious students gain admission to top universities through expert-led guidance and data-driven strategies. We are backed by leading VC firms and operate in 21 markets worldwide.

Responsibilities

• Develop, maintain, and scale robust, end-to-end automation test frameworks (UI, API, and Integration layers).
• Integrate automated test suites into CI/CD pipelines for rapid deployments.
• Define, track, and scale test coverage metrics for regression-free deployments.
• Conduct API performance, load, and stress testing.
• Design testing strategies for non-deterministic systems, evaluating LLM outputs for accuracy, bias, safety, and hallucination rates.
• Establish benchmarks for prompt regression testing to ensure AI features remain stable.
• Collaborate with Data Scientists and ML Engineers to monitor model performance metrics in production.
• Simulate adversarial inputs to ensure AI features remain secure.

Requirements

• 4+ years of experience in Software QA Automation.
• 1–2 years specifically focused on testing AI/ML-powered applications or LLM integrations.
• Strong programming proficiency in Python or JavaScript/TypeScript.
• Hands-on experience with modern automation tools (e.g., Playwright, Cypress, Selenium).
• Expertise in API testing tools (e.g., Postman, RestAssured, PyTest).
• Familiarity with AI evaluation frameworks (e.g., Ragas, TruLens, Deepchecks) or ability to write custom evaluation scripts.
• Basic understanding of vector databases, embeddings, and ML model data pipelines.
• Bachelor’s degree in Computer Science, Engineering, or equivalent practical experience.
• Familiarity with cloud platforms (AWS, GCP, or Azure) and containerization (Docker, Kubernetes).
• A healthy obsession with AI safety and ethics.

Skills

• Python
• JavaScript/TypeScript
• Playwright
• Cypress
• Selenium
• Postman
• RestAssured
• PyTest
• Ragas
• TruLens
• Deepchecks
• Vector databases
• Embeddings
• Cloud platforms (AWS, GCP, Azure)
• Docker
• Kubernetes

Location

• US
• Canada
• UK
• Singapore
• Japan
• Hong Kong
• Australia
• New Zealand
• Remote

Work Type

• Flexible working environment
• Remote

Experience Level

• 4+ years of experience in Software QA Automation
• 1–2 years specifically focused on testing AI/ML-powered applications or LLM integrations

Education Level

• Bachelor’s degree in Computer Science, Engineering, or equivalent practical experience

Benefits

• Limitless development and exposure
• Internal promotions/role changes
• $1000 individual training budget per year
• Psychologist on staff
• Insightful fireside chats and workshops
• Radical Candor feedback approach

About the Company

• Crimson Education is the world’s leading college admissions consulting firm.
• Over 1,490 Ivy League offers and 2,410 to the US Top 15.
• Crimson students are 7x more likely to get into the Ivy League than their peers.
• Featured on the front page of the Wall Street Journal.
• Crimson is the only college admissions consultancy that brings together former Ivy League admissions officers, professors, PhD teaching fellows, ISEF winners/judges, and Capstone project mentors.
• Backed by leading VC firms, including Tiger Global, Heal Partners, IceHouse Ventures, and Movac.
• Recently closed a USD $40M Series D funding round at a USD $640M valuation.
• Operates across 21 markets worldwide.
• Rapidly growing start up.
• Global player with 28 markets (and growing) across the globe.