Impress employers and recruiters.
Choose from hundreds of resume examples.
Impress employers and recruiters.
Choose from hundreds of resume examples.
Tailor your resume to this Inference Engineering, Co-op role.
Rezi rewrites your resume against Inferact's job description. Free.
Tailor your resume to this Inference Engineering, Co-op role.
Rezi rewrites your resume against Inferact's job description. Free.
Don't guess if your resume is good enough.
See how it scores against the Inference Engineering, Co-op posting at Inferact — free, in seconds.
Don't guess if your resume is good enough.
See how it scores against the Inference Engineering, Co-op posting at Inferact — free, in seconds.
About the Role
We are seeking exceptional University of Waterloo co-op students to work on systems that optimize the speed, efficiency, and reliability of frontier AI models. You will tackle real engineering problems within the vLLM stack, from model execution and low-level accelerator code to distributed serving and cloud platforms. This role offers mentorship from senior team members and the opportunity to contribute to open-source projects and production systems, impacting developers and AI teams globally.
Responsibilities
- Bring new model architectures and inference techniques to life in vLLM.
- Implement ideas from research papers.
- Support mixture-of-experts, multimodal, diffusion, and agentic workloads.
- Improve scheduling, continuous batching, KV-cache memory management, prefix caching, and hybrid model serving.
- Build the distributed serving data plane for vLLM across multiple GPUs and nodes.
- Work on tensor, expert, or context parallelism.
- Implement prefill/decode separation and KV-cache transport.
- Develop fault tolerance and multi-tenancy features.
- Optimize high-performance communication using NCCL, DeepEP, NVSHMEM, RDMA, or InfiniBand.
- Raise hardware performance by writing and optimizing attention, GEMM, sampling, KV-cache, fused, and quantization kernels.
- Use CUDA, Triton, TileLang, CUTLASS/CUTE, or related tools.
- Reason about memory hierarchy, occupancy, and tensor cores.
- Prove speedups through profiling, correctness tests, and reproducible benchmarks.
- Make vLLM first-class on AMD accelerators.
- Work across ROCm, HIP, Triton, CK, AITER, kernels, runtime paths, quantization, compiler integration, and performance-regression infrastructure.
- Learn how AMD-specific execution, memory, and toolchain constraints shape inference.
- Make vLLM fast and correct on Google TPUs.
- Build backend, runtime, and compiler integrations with JAX, XLA, Pallas, MLIR, and related tooling.
- Inspect compiler artifacts.
- Work on lowering, fusion, and code generation.
- Benchmark production-relevant serving across correctness, latency, and throughput.
- Build the operational platform for large-scale, reliable inference.
- Work on Kubernetes and custom operators.
- Implement topology-aware GPU scheduling.
- Develop zero-downtime vLLM rollouts.
- Implement token-aware routing.
- Enhance observability and infrastructure-as-code.
- Develop automated recovery mechanisms.
- Manage bring-your-own-cloud or multi-cloud fleet management.
Requirements
- Currently pursuing a bachelor's, master's, or PhD degree in computer science, engineering, mathematics, or a related technical field.
- Eligible for a University of Waterloo co-op work term.
- Strong programming ability in Python, C++, Rust, Go, or another systems-oriented language.
- Strong computer science fundamentals.
- Ability to learn from research papers, technical documentation, and complex systems code.
- Evidence of building and debugging nontrivial software through coursework, research, prior internships, open-source work, or ambitious side projects.
- A builder mindset: define success, measure results, validate correctness, communicate clearly, and iterate until the system works.
Skills
- ML or inference systems
- Distributed systems
- GPU or accelerator programming
- Compilers
- High-performance computing
- Operating systems
- Networking
- Kubernetes
- Cloud infrastructure
- PyTorch
- vLLM
- SGLang
- TensorRT-LLM
- CUDA
- Triton
- TileLang
- ROCm/HIP
- JAX/XLA
- Pallas
- Kubernetes
- Helm
- Terraform
- Ray
- SLURM
- Profiling software
- Benchmarking software
- Validating numerical or systems correctness
- Building tests for performance regressions
- Systems beyond a single local process or device
- Multi-GPU systems
- Multi-node systems
- Research-cluster systems
- High-throughput systems
- Production-like workloads
- Open-source ML projects
- Open-source systems projects
- Open-source compiler projects
- Open-source infrastructure projects
- vLLM contributions
- Adjacent project contributions
- Research system building
- Benchmark suite creation
- Compiler project creation
- Kernel project creation
- Distributed service building
- Infrastructure tool creation
- Working across multiple layers of the stack
- Creating technical artifacts (papers, design documents, blog posts, demos, talks)
Location
- San Francisco, California
Work Type
- In-office
- Co-op
- Internship
Experience Level
- Co-op student
Education Level
- Bachelor's degree
- Master's degree
- PhD degree
Salary/Compensations
- Competitive compensation based on the applicable co-op market and candidate background
Benefits
- Housing stipend for the duration of the co-op term
About the Company
- Inferact's mission is to grow vLLM as the world's AI inference engine and accelerate AI progress by making inference cheaper and faster.
- Founded by the creators and core maintainers of vLLM, we sit at the intersection of models and hardware.