Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

Tailor your resume to this Senior AI Evals Engineer (AI Quality & Safety) - Libra-Legal AI Assistant (m/w/d) role.

Rezi rewrites your resume against Wolters Kluwer's job description. Free.

Resume score gauge reading 58 out of 100

Don't guess if your resume is good enough.

See how it scores against the Senior AI Evals Engineer (AI Quality & Safety) - Libra-Legal AI Assistant (m/w/d) posting at Wolters Kluwer — free, in seconds.

About the Role

As an Evals Engineer (m/w/d) at Libra, you'll be the first dedicated hire on a new team that owns how we measure quality: the datasets, rubrics, judges and harnesses that decide whether an AI feature is good enough to ship, and the automated optimization that runs against them. You'll work in a lean, agile environment with AI Engineers, Legal Engineers and Product, based at the vibrant Merantix AI Campus in Berlin, surrounded by a community of AI innovators.

Responsibilities

Build and own the eval platform: datasets, judges, harnesses, regression gates, cost and quality in one view, on Python, FastAPI and Langfuse.
Make the eval platform self-service for AI Engineers to evaluate and tune their own features.
Map the quality landscape for research, drafting, summarisation and retrieval across different jurisdictions.
Design rubrics and set the standard for LLM-as-judge, turning expert judgment into version-controlled criteria.
Keep judges recalibrated as models and jurisdictions change.
Deep-dive results and traces to identify failure points and automate fixes.
Instrument the app to allow automated optimization of prompts, hyperparameters, and harness architecture.
Run automated optimization (GEPA, DSPy) against a fitness function.
Make cheaper models win by treating quality per euro as a first-class metric.
Own guardrails including ungrounded advice, invented citations, jurisdiction and language leakage, and prompt injection.
Design, red-team, and prove guardrails hold.

Requirements

Minimum 5 years in software engineering, with at least 1 year building LLM-powered products in production.
Strong Python skills, including FastAPI, modern tooling, and the data stack (pandas, numpy, notebooks).
Hands-on experience designing evaluations: datasets, rubrics, LLM-as-judge, benchmarking, human labelling.
Solid security and data-privacy practice.
Experience with AI coding agents (Claude Code, Codex, Cursor) in daily workflow.
Genuine interest in the legal domain.
Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process.
Candidates must participate in interviews without the assistance of AI tools or external prompts.
Use of AI-generated responses or third-party support during interviews will be grounds for disqualification.

Skills

Systems thinking and understanding of how systems can be gamed.
Ability to evaluate metrics critically and report bounds.
Skill in extracting expertise from busy individuals and building user-friendly tools.
Macro-management approach: asking sharp questions, presenting options with trade-offs, and assisting in decision-making.
Entrepreneurial, accountable, and pragmatic mindset.
Excellent communication in English.

Location

Berlin

Work Type

Hybrid

Experience Level

5+ years software engineering
1+ year building LLM-powered products

Education Level

Bachelor's degree or equivalent in a relevant technical field (e.g. Computer Science, Software Engineering, Statistics, Data Science)
Advanced degree is a plus

About the Company

Libra is transforming how lawyers work using an AI platform that combines deep legal reasoning with cutting-edge generative technology.
Libra operates with the speed and ownership of a startup, backed by the scale and stability of an established leader.
Libra is an independent business unit within the Legal & Regulatory division of Wolters Kluwer.
Wolters Kluwer is a leading global provider of information, software, and services for professionals.
Wolters Kluwer has supported experts and organizations through innovative solutions for 180 years, with over 20,000 colleagues worldwide.

Equal Opportunity

Our interview process is designed to assess individual skills, experiences, and communication style.
We value authenticity and want to ensure we’re getting to know you—not a digital assistant.
We ask to remove virtual backgrounds and include in-person interviews in our hiring process.