About the Role
AI hardware is critical for the future of AI economics. This role involves benchmarking GPUs, TPUs, and custom silicon, and building analysis to help the industry understand them. You will design and run performance benchmarks, extend our hardware benchmarking stack, build cost and throughput models, and work directly with chipmakers.
Responsibilities
- Benchmark AI Accelerators: Design and execute performance benchmarking across GPUs, TPUs and custom inference silicon, measuring throughput, latency and price-performance.
- Build Evaluation Methodology: Develop and maintain frameworks for measuring AI hardware performance and efficiency, extending products like AA-AgentPerf.
- Analyze Inference Economics: Understand the interplay of cost-per-token, utilization, and hardware choice to inform industry deployment and procurement decisions.
- Partner with Chipmakers: Work with leading hardware companies to benchmark their latest silicon, shape methodology, and set industry standards.
- Drive Strategic Analysis: Produce reports and data visualizations communicating hardware performance and economics to diverse audiences.
- Become AI-Native: Utilize AI tools to enhance workflows and maintain a competitive edge in AI benchmarking.
Requirements
- 3+ years of professional experience, including at least 2 years at an AI chip company.
- Strong analytical and critical thinking skills.
- Proficiency in Python and data analysis.
- Deep familiarity with AI accelerators and the inference software stack (e.g. CUDA, TensorRT, vLLM).
- Understanding of inference economics: cost-per-token, utilization, and trade-offs in deployment decisions.
- Genuine, demonstrable interest and knowledge of Frontier AI.
Skills
- Python
- Data analysis
- AI accelerators
- Inference software stack
- CUDA
- TensorRT
- vLLM
- Inference economics
Location
- San Francisco
Work Type
- On-site
- Full-time
Experience Level
- 3+ years of professional experience
- 2+ years at an AI chip company
Benefits
- Competitive compensation including equity
About the Company
- Artificial Analysis is the leading independent AI benchmarking company.
- We support labs, engineers and enterprises to understand AI capabilities and make critical decisions about their AI strategies.
- Our benchmarks are trusted by hundreds of thousands of users and are the go-to reference for leading AI labs including OpenAI, Google, Meta, NVIDIA and Anthropic, and major publications.
- We are a team of 40+, on track to double by end of year, backed by prominent industry leaders.
