Senior Software Engineer, Spark Platform at DoorDash USA | CA, US | Rezi

Senior Software Engineer, Spark Platform at DoorDash USA

Senior Software Engineer, Spark Platform

DoorDash USA · CA, US

3 weeks ago

Senior Software Engineer, Spark Platform

DoorDash USA · CA, US

25 days ago
Resume preview

Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now
Resume preview

Tailor your resume to this Senior Software Engineer, Spark Platform role.

Rezi rewrites your resume against DoorDash USA's job description. Free.

Resume score gauge reading 58 out of 100

Don't guess if your resume is good enough.

See how it scores against the Senior Software Engineer, Spark Platform posting at DoorDash USA — free, in seconds.

About the Role

As a Senior Software Engineer on Spark Platform, you will set the technical direction for our in-house Spark deployment and shape the architecture that will run DoorDash's data, analytics, and ML compute for the next five years and beyond. You will own the deep, cross-cutting problems that span the runtime, the shuffle service, the scheduler, and the overall service reliability — making the architectural calls that compound across the platform's lifetime. You will partner with the Engineering Manager on technical roadmap, hiring, and team shape, and act as the senior technical voice in cross-team partnerships with Data Engineering, ML Platform, and product engineering teams that depend on the platform.

Responsibilities

  • Set the multi-year technical direction for an in-house Spark-on-Kubernetes platform — runtime, shuffle, scheduler, reliability — and make the architectural calls that compound for years.
  • Own the deepest distributed-systems problems on the team: shuffle architecture, multi-tenant scheduling, runtime performance, and the failure modes that only show up at scale.
  • Partner with the Engineering Manager on technical roadmap, hiring, interview design, and team shape as the team continues to grow.
  • Uplevel the rest of the team through design reviews, mentorship, and raising the bar on what we ship.
  • Represent Spark Platform in cross-team architecture forums and shape how data, analytics, and ML workloads land on the platform.

Requirements

  • B.S., M.S., or PhD in Computer Science or equivalent.
  • 6+ years of industry experience designing and operating distributed systems at scale.
  • Deep, hands-on experience with Apache Spark — internals, query execution, shuffle, the executor/driver model — at platform scale on Amazon EMR, Databricks, or an in-house deployment, with a focus on platform operations (runtime upgrades, cluster lifecycle, shuffle, observability, multi-tenant scheduling) rather than authoring individual Spark jobs.
  • Production experience with one or more of: remote/external shuffle systems (Celeborn, Magnet, Cosco, or similar), batch/big-data schedulers (YuniKorn, Volcano, Kueue, or the Spark-on-Kubernetes operator), or the observability and SRE patterns that make distributed compute platforms operable.
  • Strong fluency operating workloads on Kubernetes in production — operator patterns, executor pod lifecycle, network topology, and the multi-tenant failure modes that show up at scale.
  • Familiarity with data lake table formats such as Apache Iceberg or Delta Lake, and with the query and SQL engines that read them.
  • Track record of acting as a technical leader on a platform team — setting direction, mentoring, and partnering with management on roadmap and hiring.
  • Professional experience with Scala, Java, Python, or Go; strong SQL.
  • You are located or willing to relocate to the Bay Area, Seattle, or NYC.

Skills

  • Apache Spark
  • Kubernetes
  • Scala
  • Java
  • Python
  • Go
  • SQL

Location

  • San Francisco
  • Sunnyvale
  • Seattle
  • New York City
  • Bay Area

Work Type

  • Hybrid

Experience Level

  • Senior
  • 6+ years of industry experience

Education Level

  • B.S., M.S., or PhD in Computer Science or equivalent

Salary/Compensations

  • $159,800—$235,000 USD

Benefits

  • 401(k) plan with employer matching
  • 16 weeks of paid parental leave
  • Wellness benefits
  • Commuter benefits match
  • Paid time off
  • Paid sick leave
  • Medical benefits
  • Dental benefits
  • Vision benefits
  • 11 paid holidays
  • Disability and basic life insurance
  • Family-forming assistance
  • Mental health program
  • Flexible paid time off/vacation (for salaried roles)
  • 80 hours of paid sick time per year (for salaried roles)

About the Company

  • At DoorDash, our mission to empower local economies shapes how our team members move quickly, learn, and reiterate in order to make impactful decisions that display empathy for our range of users—from Dashers to merchant partners to consumers.
  • We are a technology and logistics company that started by enabling door-to-door delivery, and we are looking for team members who can help us go from a company that is known as the place you order food to a company that people turn to for any and all goods.
  • DoorDash is growing rapidly and changing constantly, which gives our team members the opportunity to share their unique perspectives, solve new challenges, and own their careers.
  • We're committed to supporting employees’ happiness, healthiness, and overall well-being by providing comprehensive benefits and perks including premium healthcare, wellness expense reimbursement, paid parental leave and more.

Equal Opportunity

  • We’re committed to growing and empowering a more inclusive community within our company, industry, and cities. That’s why we hire and cultivate diverse teams of people from all backgrounds, experiences, and perspectives. We believe that true innovation happens when everyone has room at the table and the tools, resources, and opportunity to excel.
  • In keeping with our beliefs and goals, no employee or applicant will face discrimination or harassment based on: race, color, ancestry, national origin, religion, age, gender, marital/domestic partner status, sexual orientation, gender identity or expression, disability status, or veteran status.
  • Above and beyond discrimination and harassment based on “protected categories,” we also strive to prevent other subtler forms of inappropriate behavior (i.e., stereotyping) from ever gaining a foothold in our office.
  • Whether blatant or hidden, barriers to success have no place at DoorDash.
  • We value a diverse workforce – people who identify as women, non-binary or gender non-conforming, LGBTQIA+, American Indian or Native Alaskan, Black or African American, Hispanic or Latinx, Native Hawaiian or Other Pacific Islander, differently-abled, caretakers and parents, and veterans are strongly encouraged to apply.
  • Pursuant to the San Francisco Fair Chance Ordinance, Los Angeles Fair Chance Initiative for Hiring Ordinance, and any other state or local hiring regulations, we will consider for employment any qualified applicant, including those with arrest and conviction records, in a manner consistent with the applicable regulation.
  • If you need any accommodations, please inform your recruiting contact upon initial connection.