Impress employers and recruiters.
Choose from hundreds of resume examples.
Impress employers and recruiters.
Choose from hundreds of resume examples.
Tailor your resume to this (Senior/Junior AI Inference Reliability Engineer, Compute Infra role.
Rezi rewrites your resume against Together AI's job description. Free.
Tailor your resume to this (Senior/Junior AI Inference Reliability Engineer, Compute Infra role.
Rezi rewrites your resume against Together AI's job description. Free.
Don't guess if your resume is good enough.
See how it scores against the (Senior/Junior AI Inference Reliability Engineer, Compute Infra posting at Together AI — free, in seconds.
Don't guess if your resume is good enough.
See how it scores against the (Senior/Junior AI Inference Reliability Engineer, Compute Infra posting at Together AI — free, in seconds.
About the Role
We are seeking a software engineer to manage the production environment of our Kubernetes-native control plane, which provisions and operates our GPU inference fleet. You will be responsible for maintaining the reliability of the platform, diagnosing and resolving complex issues, and enhancing the automated self-healing system. The core value is ensuring the decoupling remains effective even during failures and maintaining fleet efficiency without compromising stability. You will own SLOs, lead incident response, and implement systemic fixes based on postmortem analyses. A product mindset is essential, focusing on improving the on-call experience, alert clarity, and automated remediation.
Responsibilities
- Operate the provisioning state machine in production, owning the health of the host lifecycle pipeline, discovery, inference bring-up, GPU driver/CUDA validation, health checks, and decommission/RMA.
- Triage and extend automated self-healing by diagnosing live issues, driving fixes to production, and creating new detection/remediation plugins.
- Own end-to-end reliability of the pipeline, holding SLOs and error budgets for provisioning and fleet health.
- Lead incident response when idempotency, retry, rollback, or drift assumptions break down.
- Drive systemic fixes rather than just hotfixes.
- Partner with the inference/ML platform team from the production side, feeding back failure modes as requirements.
- Work alongside machine learning researchers, engineers, and product managers to bring new technologies into production.
- Introduce new techniques, tools, and architecture to improve performance, latency, throughput, and efficiency of the model inference stack.
- Build tools for visibility into bottlenecks and instability, then design and implement solutions.
- Lead reliability cross-functionally, building coalitions and managing stakeholders.
Requirements
- Experience operating a provisioning or host-lifecycle pipeline in production.
- Comfortable troubleshooting stuck or failed state transitions at fleet scale.
- Ability to translate real failure patterns back into improvements for the state machine’s design.
- Deep experience owning production reliability for large-scale distributed systems.
- Hands-on experience troubleshooting InfiniBand/RDMA networking issues.
- Familiarity with the operational realities of multi-tenant model serving.
- Demonstrated track record of eliminating operational toil through automation and tooling.
- Hands-on experience with Kubernetes controller/CRD patterns, reconciliation loops, and node-level health/remediation systems.
- Comfort with event-driven architectures, backpressure/circuit-breaking design.
- Treating observability (metrics, logs, traces) as a built-in system component.
- A software-engineering mindset, writing and shipping production code.
- Thinking about GPU infrastructure as software to be engineered.
Skills
- Kubernetes-native control plane
- GPU inference fleet management
- Manifest-driven API and controllers
- InfiniBand issues
- KV cache over RDMA
- Hero-model and LoRA-serving fault diagnosis
- Automated self-healing systems
- SLOs and error budgets
- Incident response
- Postmortem analysis
- Capacity efficiency
- Distributed systems
- GPU-serving systems
- Troubleshooting InfiniBand/RDMA
- Kubernetes controller/CRD-based platforms
- GPU infrastructure engineering
- Product mindset
- Host lifecycle pipeline
- Discovery
- Inference bring-up
- GPU driver/CUDA validation
- Health checks
- Decommission/RMA
- Node-scanning and remediation platform
- Model serving
- Multi-tenant model serving
- Operational toil elimination
- Automation
- Tooling
- Reconciliation loops
- Node-level health/remediation systems
- Event-driven architectures
- Backpressure/circuit-breaking design
- Observability (metrics, logs, traces)
- ML platform integration
- Performance optimization
- Latency reduction
- Throughput improvement
- Efficiency enhancement
- Bottleneck identification
- Instability source identification
- Cross-functional leadership
- Stakeholder management
- Roadmap planning
Work Type
- Full-time
Salary/Compensations
- $160,000 - $280,000
Benefits
- Startup equity
- Health insurance
- Other competitive benefits
About the Company
- Together AI, the AI Native Cloud, is purpose-built for AI engineers.
- AI application developers get high-performance inference that scales reliably, fine-tuning and reinforcement learning for creating frontier-level specialized models, and pre-training at massive scale for fully custom intelligence.
- Offers a marketplace of leading open models that teams can run, adapt, and own.
- Trusted by Cursor, Decagon, ElevenLabs, Salesforce, and Zoom.
- Serves 400+ trillion tokens a month.
Equal Opportunity
- Together AI is an Equal Opportunity Employer and is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and more.