Software Engineer, Data Infrastructure at Thinking Machines Lab | CA, US | Rezi

Software Engineer, Data Infrastructure at Thinking Machines Lab

Software Engineer, Data Infrastructure

Thinking Machines Lab · CA, US

3 weeks ago

Software Engineer, Data Infrastructure

Thinking Machines Lab · CA, US

24 days ago
Resume preview

Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

About the Role

We are seeking an engineer to join our team and contribute to data infrastructure. You will architect and scale the core infrastructure for distributed training pipelines, multimodal data catalogs, and intelligent processing systems handling petabytes of data. This role is crucial for enabling research breakthroughs by working directly with researchers to accelerate experiments, develop new datasets, and improve infrastructure efficiency.

Responsibilities

  • Design, build, and operate scalable, fault-tolerant infrastructure for LLM Research: distributed compute, data orchestration, and storage across modalities.
  • Develop high-throughput systems for data ingestion, processing, and transformation — including training data catalogs, deduplication, quality checks, and search.
  • Build systems for traceability, reproducibility, and robust quality control at every stage of the data lifecycle.
  • Implement and maintain monitoring and alerting to support platform reliability and performance.
  • Collaborate with research teams to unlock new features, improve data quality, and accelerate training cycles.

Requirements

  • Bachelor’s degree or equivalent experience in computer science, engineering, or similar.
  • Proficiency in at least one backend language (Python or Rust).
  • Fluent in distributed compute frameworks such as Apache Spark or Ray.
  • Deeply familiar with cloud infrastructure, data lake architectures, and batch and streaming pipelines.
  • Comfort operating across the stack and owning projects end-to-end.
  • Thrive in a highly collaborative environment involving many, different cross-functional partners and subject matter experts.
  • A bias for action with a mindset to take initiative to work across different stacks and different teams where you spot the opportunity to make sure something ships.

Skills

  • Python
  • Rust
  • Apache Spark
  • Ray
  • Cloud infrastructure
  • Data lake architectures
  • Batch and streaming pipelines
  • Kafka
  • dbt
  • Terraform
  • Airflow
  • Web crawler development
  • Deduplication
  • Data mining
  • Search scaling
  • File formats
  • Storage systems (Parquet, Delta Lake)
  • Documentation
  • Testing
  • Tooling

Location

  • San Francisco, California

Work Type

  • Onsite

Experience Level

  • Mid-level
  • Senior

Education Level

  • Bachelor's degree or equivalent experience

Salary/Compensations

  • $350,000 - $475,000 USD

Benefits

  • Generous health, dental, and vision benefits
  • Unlimited PTO
  • Paid parental leave
  • Relocation support

About the Company

  • The mission of Thinking Machines is to build AI that extends human will and judgment.

Equal Opportunity

  • We sponsor visas. While we can't guarantee success for every candidate or role, if you're the right fit, we're committed to working through the visa process together.