Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

Tailor your resume to this Senior AI Evals Engineer (AI Quality & Safety) - Libra-Legal AI Assistant (m/w/d) role.

Rezi rewrites your resume against Wolters Kluwer's job description. Free.

Resume score gauge reading 58 out of 100

Don't guess if your resume is good enough.

See how it scores against the Senior AI Evals Engineer (AI Quality & Safety) - Libra-Legal AI Assistant (m/w/d) posting at Wolters Kluwer — free, in seconds.

About the Role

As an Evals Engineer (m/w/d) at Libra, you will be the first dedicated hire on a new team responsible for measuring quality, including datasets, rubrics, judges, and harnesses that determine if an AI feature is ready for release. You will work in a lean, agile environment with AI Engineers, Legal Engineers, and Product, based at the Merantix AI Campus in Berlin.

Responsibilities

Build and own the eval platform, including datasets, judges, harnesses, regression gates, and cost/quality views using Python, FastAPI, and Langfuse.
Make the eval platform self-service for AI Engineers to evaluate and tune their own features.
Map the quality landscape, defining defensible measures for research, drafting, summarization, and retrieval across different jurisdictions.
Design rubrics and set standards for LLM-as-judge, translating expert judgment into version-controlled criteria.
Keep judges recalibrated as models and jurisdictions change.
Deep-dive into results and traces to identify failure points and automate fixes.
Instrument the application to allow automated optimization of prompts, hyperparameters, and harness architecture.
Run automated optimization (GEPA, DSPy) against a trusted fitness function.
Treat quality per euro as a first-class metric, instrumented per call.
Find configurations where smaller models match or exceed the performance of expensive ones.
Own and design guardrails against ungrounded advice, invented citations, jurisdiction leakage, and prompt injection.
Red-team guardrails and prove their effectiveness.

Requirements

Minimum 5 years in software engineering, with at least 1 year building LLM-powered products in production.
Strong Python skills, including FastAPI, modern tooling, and the data stack (pandas, numpy, notebooks).
Hands-on experience designing evaluations: datasets, rubrics, LLM-as-judge, benchmarking, and human labeling.
Solid security and data-privacy practices.
Experience with AI coding agents (Claude Code, Codex, Cursor) in daily workflow.
Genuine interest in the legal domain.
Bachelor's degree or equivalent in a relevant technical field (e.g., Computer Science, Software Engineering, Statistics, Data Science); advanced degree is a plus.
Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process.
Candidates must participate in interviews without AI assistance.

Skills

Systems thinking and anticipation of gaming.
Ability to critically analyze data and avoid being fooled by single numbers.
Reporting bounds and retiring unreliable results.
Extracting expertise from busy individuals and building user-adopted tools.
Macro-management approach: asking sharp questions, presenting options with trade-offs, and assisting in decision-making.
Entrepreneurial, accountable, and pragmatic mindset.
Excellent communication in English.

Location

Berlin
Remote (up to 3 days per week)
Work from anywhere within the EU (up to 20 days/year)

Work Type

Permanent employment
Remote
Hybrid
Full-time

Experience Level

Minimum 5 years in software engineering
At least 1 year building LLM-powered products in production

Education Level

Bachelor's degree or equivalent in a relevant technical field (e.g., Computer Science, Software Engineering, Statistics, Data Science)
Advanced degree is a plus

Benefits

Permanent employment from day one
Remote work & flexibility (up to 3 days per week)
Work abroad flexibly (up to 20 days within a twelve-month period)
30 vacation days (starting 2027, 26 days for 2026)
1 additional day off per year for volunteer work (Volunteer Day)
E-learning via LinkedIn
Online language training with goFluent
Other training and development opportunities
JobRad leasing for bicycle or e-bike
Company pension scheme
€300 budget for IT equipment for working from home
Modern on-site working in an attractive office in Berlin
Structured onboarding with a buddy program

About the Company

Libra is transforming how lawyers work using an AI platform that combines deep legal reasoning with generative technology.
Libra operates with the speed and ownership of a startup, backed by the scale and stability of an established leader.
Libra is an independent business unit within the Legal & Regulatory division of Wolters Kluwer.
Wolters Kluwer is a leading global provider of information, software, and services for professionals.
For 180 years, Wolters Kluwer has supported and simplified the work of experts and organizations through innovative solutions.
Wolters Kluwer relies on more than 20,000 colleagues worldwide.

Equal Opportunity

Our Interview Practices: To maintain a fair and genuine hiring process, we kindly ask that all candidates participate in interviews without the assistance of AI tools or external prompts.
Our interview process is designed to assess your individual skills, experiences, and communication style.
We value authenticity and want to ensure we’re getting to know you—not a digital assistant.
To help maintain this integrity, we ask to remove virtual backgrounds and include in-person interviews in our hiring process.
Please note that use of AI-generated responses or third-party support during interviews will be grounds for disqualification from the recruitment process.