About the Role
This role owns quality for LOIS for Word, Filevine's AI-powered legal drafting product. You'll help build and own the QA function, including test strategy, process, and automation. You will work at the intersection of legal AI and product quality, evaluating AI output accuracy and UI behavior. A key aspect of this role is integrating quality into the feature planning and shipping process from the beginning.
Responsibilities
- Define and own the test strategy, process, and QA playbook for the product.
- Move QA upstream by partnering with product and engineering early, so test plans are ready before features ship.
- Evaluate AI output quality across dimensions like accuracy, citation reliability, and appropriately scoped reasoning.
- Design and maintain evaluation sets that account for non-deterministic output and catch regressions over time.
- Test how AI-generated edits apply to documents, including accuracy and formatting fidelity across document structures.
- Build automated guardrails integrated into CI/CD that cover critical flows.
- Write structured, reproducible bug reports.
- Partner with legal domain experts to understand how different customer types use the product, and translate that into test coverage.
- Approach QA with a product and design mindset.
- Compare behavior across product surfaces and document discrepancies.
Requirements
- 2+ years of QA experience, including ownership of test strategy for a product or major feature area.
- Track record of building QA process and automation from scratch in a fast-moving environment.
- Comfort operating in high ambiguity, where features ship quickly and you figure out how to test them fast.
- Strong analytical and problem-solving skills, with attention to both functional correctness and AI output quality.
- Ability to reason about non-deterministic systems and define what "correct" means when there's no single right answer.
- Excellent written communication, with the ability to write clear, concise, actionable bug reports backed by evidence.
- Experience with bug tracking tools and test management workflows.
Skills
- QA
- Test strategy
- Test process
- Automation
- AI quality
- Functional testing
- Analytical skills
- Problem-solving
- Written communication
- Bug tracking
- Test management
- CI/CD
- Programming
Experience Level
- 2+ years of QA experience
Salary/Compensations
- $105,000 - $150,000
Benefits
- Medical, Dental, & Vision Insurance (for full-time employees)
- Competitive & Fair Pay
- Maternity & paternity leave (for full-time employees)
- Short & long-term disability
- Opportunity to learn from a dedicated leadership team
- Top-of-the-line company swag
About the Company
- Filevine is a Legal AI company delivering Legal Operating Intelligence for the future of legal work.
- Grounded in a singular system of truth, Filevine brings together data, documents, workflows, and teams into one unified platform—where modern legal work happens with clarity and consistency.
- Powered by LOIS, the Legal Operating Intelligence System, Filevine connects context across every matter to transform legal operations from reactive to proactive.
- LOIS reads, understands, and reasons across your data to surface insight, automate complexity, and give professionals the clarity and confidence to see more, know more, and do more.
- Fueled by a team of exceptional collaborators and innovators, Filevine’s rapid growth has earned AI awards and recognition from Deloitte and Inc. as one of the most innovative and fastest-growing technology companies in the country.
