Impress employers and recruiters.
Choose from hundreds of resume examples.

Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

About the Role

The Platform team builds the technology that enables Spotify to learn quickly and scale easily, supporting rapid global growth. The Multilingual AI Data Quality team focuses on ensuring AI-powered experiences are accurate, culturally relevant, and trustworthy across various languages and markets by evaluating language quality and AI data quality.

Responsibilities

• Define quality frameworks, evaluation rubrics, thresholds, and methodologies for multilingual AI experiences.
• Design and execute structured evaluations for AI-generated, AI-translated, AI-curated, and recommendation-driven experiences.
• Lead multilingual dataset curation, annotation, enrichment, and ground-truth creation to support AI model development and evaluation.
• Analyze evaluation results, identify quality gaps, and provide actionable recommendations to improve multilingual AI quality.
• Support LLM-as-a-judge workflows, evaluator calibration, and human-AI agreement studies.
• Partner closely with Product, Engineering, Data Science, Research, Localization, vendors, and market experts to improve AI quality signals and inform launch decisions.
• Document best practices and help define quality standards across languages, markets, and AI use cases.
• Contribute to building scalable evaluation capabilities that support the next generation of AI-powered experiences across Spotify.

Requirements

• Experience in multilingual quality evaluation, localization, data curation, annotation, AI evaluation, or related fields, including text-to-text and text-to-speech experiences.
• Understanding of language quality, cultural relevance, content quality, and user experience across multiple languages and markets.
• Experience designing or conducting structured evaluations using quality rubrics, audits, annotation projects, or review methodologies.
• Ability to use qualitative and quantitative data to identify trends, measure quality, and make recommendations.
• Familiarity with large language models (LLMs), generative AI evaluation, human-in-the-loop workflows, or LLM-as-a-judge methodologies.
• Ability to work through ambiguity and translate complex quality challenges into practical evaluation strategies.
• Effective communication skills and ability to thrive in highly cross-functional environments, collaborating with technical and non-technical partners.

Skills

• Recommendation systems
• Personalization
• Search
• Ranking
• Machine translation
• Generative AI
• Dataset creation
• Annotation operations
• Evaluator calibration
• Prompt testing
• Model evaluation
• SQL
• Python
• Dashboards
• Annotation platforms

Location

• London
• Stockholm

Work Type

• Hybrid

About the Company

• Spotify is an equal opportunity employer.
• At Spotify, we are passionate about inclusivity and making sure our entire recruitment process is accessible to everyone.
• We have ways to request reasonable accommodations during the interview process and help assist in what you need.
• If you need accommodations at any stage of the application or interview process, please let us know - we’re here to support you in any way we can.

Equal Opportunity

• You are welcome at Spotify for who you are, no matter where you come from, what you look like, or what’s playing in your headphones.
• Our platform is for everyone, and so is our workplace.
• The more voices we have represented and amplified in our business, the more we will all thrive, contribute, and be forward-thinking!
• So bring us your personal experience, your perspectives, and your background.
• It’s in our differences that we will find the power to keep revolutionizing the way the world listens.