Impress employers and recruiters.
Choose from hundreds of resume examples.
Impress employers and recruiters.
Choose from hundreds of resume examples.
Tailor your resume to this Software Engineer, Inference role.
Rezi rewrites your resume against Thinking Machines Lab's job description. Free.
Tailor your resume to this Software Engineer, Inference role.
Rezi rewrites your resume against Thinking Machines Lab's job description. Free.
Don't guess if your resume is good enough.
See how it scores against the Software Engineer, Inference posting at Thinking Machines Lab — free, in seconds.
Don't guess if your resume is good enough.
See how it scores against the Software Engineer, Inference posting at Thinking Machines Lab — free, in seconds.
About the Role
We're hiring a Software Engineer, Inference to own the reliability, scale, and efficiency of the systems that serve our models to real users. This role makes sure frontier model gains reach production safely and stay up, powering live serving and products built on our models. This is a production-facing systems role at the center of the company, bridging cutting-edge inference techniques with the reality of serving real traffic.
Responsibilities
- Operate and scale production inference systems serving live traffic, including Tinker's multi-tenant serving platform.
- Own the rollout process for new models, model versions, and inference optimizations, ensuring safe, incremental deployment.
- Build and improve observability, alerting, and capacity planning for quick detection, diagnosis, and resolution of production issues.
- Partner with inference and research teams to productionize new serving techniques without compromising reliability.
- Lead incident response for production inference issues, driving root cause analysis and durable fixes.
- Design for graceful degradation, failover, and redundancy to ensure resilient serving as usage grows.
- Manage capacity and cost tradeoffs for serving infrastructure as traffic and model sizes scale.
Requirements
- Experience operating large-scale, latency-sensitive production systems.
- Proficiency in Python and Go or another systems language.
- Experience with observability, monitoring, and incident response for production services.
- Strong understanding of distributed systems and their failure modes at scale.
- Experience running production inference for large language models or other large-scale ML systems.
- Experience with deployment and rollout systems, such as canarying, blue/green deploys, or feature flags.
- Experience with capacity planning and cost optimization for GPU or TPU infrastructure.
- Familiarity with inference-specific techniques, such as batching, caching, or quantization, and their operational implications.
- Comfortable being on-call and leading incident response for critical production systems.
- Comfortable working with high autonomy in a fast-changing, early-stage environment.
Skills
- Python
- Go
- Distributed systems
- Observability
- Monitoring
- Incident response
- Deployment systems
- Capacity planning
- Cost optimization
- Inference techniques (batching, caching, quantization)
Location
- San Francisco, CA
Work Type
- Full-time
Experience Level
- Mid-level
- Senior
Salary/Compensations
- $300,000 - $350,000 USD
Benefits
- Generous health, dental, and vision benefits
- Unlimited PTO
- Paid parental leave
- Relocation support
About the Company
- The mission of Thinking Machines is to build AI that extends human will and judgment.
- We are training frontier models with Inkling, developing Tinker to let people make models their own, and crafting interfaces that broaden human-AI communication.
- We believe the future worth building is human, and we're hiring people who want to build it.