Impress employers and recruiters.
Choose from hundreds of resume examples.

Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

About the Role

We're looking for an inference runtime engineer to push the boundaries of what's possible in LLM and diffusion model serving. You'll work at the core of vLLM, optimizing how models execute across diverse hardware and architectures. Your work will directly impact how the world runs AI inference.

Responsibilities

• Optimize how models execute across diverse hardware and architectures.
• Contribute performant and maintainable code.
• Debug complex ML codebases.
• Implement core features in vLLM or other inference engine projects.
• Contribute to vLLM integrations.
• Write widely-shared technical blogs or side projects on vLLM or LLM inference.

Requirements

• Bachelor's degree or equivalent experience in computer science, engineering, or similar.
• Deep understanding of transformer architectures and their variants.
• Strong programming skills in Python with experience in PyTorch internals.
• Experience with LLM inference systems (vLLM, TensorRT-LLM, SGLang, TGI).
• Ability to read and implement model architectures and inference techniques from research papers.
• Demonstrate the ability to contribute performant and maintainable code and debug in complex ML codebases.
• Deep understanding of KV-cache memory management, prefix caching, and hybrid model serving.
• Familiarity with RL frameworks and algorithms for LLMs.
• Experience with multimodal inference (audio/image/video/text).
• Contributions to open-source ML or system infrastructure projects.

Skills

• Python
• PyTorch internals
• LLM inference systems
• Transformer architectures
• KV-cache memory management
• Prefix caching
• Hybrid model serving
• RL frameworks
• Multimodal inference

Location

• San Francisco, California
• Remote in the US

Work Type

• Full-time

Experience Level

• Mid-level
• Senior

Education Level

• Bachelor's degree or equivalent experience

Salary/Compensations

• $200,000 - $400,000 USD + equity

Benefits

• Generous health, dental, and vision benefits
• 401(k) company match

About the Company

• Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster.
• Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware—a position that took years to build.