About the Role
We're looking for an Applied AI Engineer to own the model layer that makes Soulside's documentation trustworthy. Your job is to build the post-training pipelines and evaluation systems that get our models clinically sound, defensible for medical necessity, and safe, and keep them there as we scale. This is a hands-on role for someone who lives at the intersection of applied ML and product.
Responsibilities
- Build post-training pipelines on open-source models—supervised fine-tuning, preference optimization (DPO/RLHF), LoRA/adapters, and distillation—for domain-specific clinical tasks.
- Fine-tune, deploy, and serve models across managed inference and fine-tuning platforms such as Fireworks AI, Baseten, and Together AI, and make pragmatic build-vs-buy calls on where each workload should run.
- Design and maintain rigorous evaluation sets for high-stakes tasks like clinical reasoning and AI note generation—defining metrics, curating gold-standard data, and building automated and human-in-the-loop eval harnesses.
- Turn eval results into a fast, trustworthy iteration loop: catch regressions before they ship, and quantify the impact of every model or prompt change.
- Optimize the full LLM pipeline—prompting, retrieval, structured output validation, latency, and cost.
- Partner with clinical experts to translate documentation and compliance requirements into model behavior and evaluation criteria.
- Monitor models in production for quality, drift, and failure modes, and close the loop back into training data and evals.
Requirements
- 3+ years in applied ML / AI engineering, or a Master's degree in a related field, with hands-on experience taking LLM-based systems into production.
- Practical experience with post-training / fine-tuning open-source models (e.g., Llama, Qwen, Mistral) using SFT, LoRA/PEFT, or preference-based methods.
- Experience serving or fine-tuning models on managed platforms such as Fireworks AI, Baseten, or Together AI (or comparable inference/training infra).
- Demonstrated ability to build evaluation frameworks for LLM tasks—you think in terms of measurable quality, not vibes.
- Strong Python and familiarity with the modern ML tooling ecosystem (PyTorch, Hugging Face, etc.).
- Solid grounding in prompt engineering and structured-output validation.
- Ability to thrive in a fast-paced, remote startup and communicate clearly with technical and clinical teammates.
- Experience with healthcare, clinical NLP, or other high-stakes / regulated domains.
- Familiarity with HIPAA and handling sensitive clinical data.
- RAG systems, retrieval quality tuning, or long-context document workflows.
- Experience with LLM observability, monitoring, and drift detection in production.
- Data pipeline and labeling workflow experience for curating high-quality training and eval sets.
- Open-source contributions in the ML/LLM ecosystem.
Skills
- Applied ML
- AI Engineering
- LLM
- Post-training pipelines
- Evaluation systems
- Supervised fine-tuning
- Preference optimization
- DPO/RLHF
- LoRA/adapters
- Distillation
- Model serving
- Fireworks AI
- Baseten
- Together AI
- Prompt engineering
- Structured output validation
- Python
- PyTorch
- Hugging Face
- Clinical NLP
- HIPAA
- RAG systems
- LLM observability
- Monitoring
- Drift detection
- Data pipeline
- Labeling workflows
Location
- US On-Site
Work Type
- On-Site
- Remote
Experience Level
- 3+ years
Education Level
- Master's degree
Benefits
- Competitive salary with a meaningful equity component
- Comprehensive health, dental, and vision insurance
- Flexible, remote-first culture
- Direct access to founders and influence on technical direction
- Professional development budget and conference attendance
About the Company
- Soulside AI is the specialist AI platform for behavioral health documentation and compliance.
- We generate audit-ready clinical documentation across individual and group sessions, virtual and in-person care, admissions, and treatment planning—and we embed real-time chart audits and payer-aligned compliance checks into everyday workflows.
- The result is immediate and measurable: higher-quality charts, stronger medical necessity, and hours given back to clinicians every week.
- We're backed by Counterpart Ventures, GreyMatter Capital, and One Mind, and we're a UCSF Rosenman Institute and One Mind Accelerator company.
- We've reached strong product-market fit and are scaling fast.
