Impress employers and recruiters.
Choose from hundreds of resume examples.
Impress employers and recruiters.
Choose from hundreds of resume examples.
Target Resume Now
About the Role
We are seeking a Data Scientist to drive measurable improvements in our AI systems, specifically our multi-agent LLM pipeline. You will own the evaluation strategy, ground-truth corpus design, and statistical rigor required to ensure high-quality AI outputs.
Responsibilities
- Design, build, and maintain evaluation frameworks for multi-agent LLM pipelines.
- Curate and version synthetic and real-world test corpora across 20+ industry verticals.
- Quantify model performance using metrics like precision, recall, calibration, and drift detection.
- Partner with engineering to design prompt experiments and structured-output schema iterations.
- Improve vector-search retrieval, taxonomy refinement, and classification accuracy.
- Evaluate prompt strategies, tool-use patterns, and routing logic with cost/accuracy analysis.
- Profile production traces to identify failure modes and design experiments for remediation.
- Translate product goals into quantitative success metrics through cross-functional collaboration.
- Communicate findings via reports, dashboards, and decision memos for executive leadership.
Requirements
- Bachelor’s degree in Computer Science, Statistics, Data Science, Machine Learning, or related field (Master’s or PhD preferred).
- 3+ years of professional experience in data science, ML engineering, or AI evaluation.
- Strong Python proficiency (Pandas, NumPy, scikit-learn, PyTorch or TensorFlow).
- Experience evaluating LLM-based systems, including prompt experimentation and hallucination detection.
- Solid grasp of classical statistics, including hypothesis testing and power analysis.
- SQL proficiency for PostgreSQL and experience with embedded analytical databases.
- Experience with vector databases and embedding models.
- Ability to translate business goals into measurable evaluation criteria.
Skills
- Python
- Pandas
- NumPy
- scikit-learn
- PyTorch
- TensorFlow
- LLM evaluation
- Classical statistics
- SQL
- PostgreSQL
- Vector databases
- Embedding models
Location
- Atlanta, GA
Work Type
- Full-Time
- Hybrid
Experience Level
- 3+ years of professional experience
Education Level
- Bachelor’s degree in Computer Science, Statistics, Data Science, Machine Learning, or related quantitative discipline
- Master’s or PhD preferred
Salary/Compensations
- Competitive salary
Benefits
- Competitive benefits package
- Fast-paced, high-impact work environment
- Opportunity to work closely with executive leadership
- Access to cutting-edge technologies
- Culture of innovation, ownership, and growth
About the Company
- Datavault AI delivers high-performance computing software, Web 3.0 data-management solutions, and advanced audio technologies.
- Event Citadel is a provider of end-to-end event technology solutions including registration, ticketing, and attendee-engagement services.