About the Role
Architect, design, implement, enhance, and maintain highly scalable, available, secure, and elastic cloud-ready data solutions using cutting-edge technologies to support predictive and prescriptive analytics needs. Act as a trusted partner and advisor to solutions architects and data scientists, playing a crucial part in the analytics solution lifecycle from prototype to production and operations.
Responsibilities
- Work with data management, data science, decision science, and technology teams to address supply chain data needs in demand and supply planning, replenishment, pricing, and optimization.
- Develop/refine data requirements, design/develop data deliverables, and optimize data pipelines in non-production and production environments.
- Design, build, and manage/monitor data pipelines for data structures encompassing data transformation, data models, schemas, metadata, and workload management.
- Integrate analytics and data science output into business processes and workflows.
- Build and optimize data pipelines, pipeline architectures, and integrated datasets, including ETL/ELT, data replication/CI-CD, API design, and access.
- Work with and optimize existing ETL processes and data integration and preparation flows, assisting in their move to production.
- Work with popular data discovery, analytics, and BI and AI tools in a semantic-layer data discovery context.
- Apply agile methodologies, DevOps, and DataOps principles to data pipelines to improve communication, integration, reuse, and automation of data flows.
- Implement Agentic AI capability to drive efficiency and opportunity.
Requirements
- Bachelor’s degree in computer science, data management, information systems, information science or a related field; advanced degree preferred.
- 3+ years in data engineering building production data pipelines (batch and/or streaming) with Spark on cloud.
- 2+ years hands-on Azure Databricks (PySpark/Scala, Spark SQL, Delta Lake).
- Experience with Delta Lake operations (MERGE/CDC, OPTIMIZE/Z-ORDER, VACUUM, partitioning, schema evolution).
- Experience with Unity Catalog (RBAC, permissions, lineage, data masking/row-level access).
- Experience with Databricks Jobs/Workflows or Delta Live Tables.
- Experience with Azure Data Factory for orchestration (pipelines, triggers, parameterization, IRs) and integration with ADLS Gen2, Key Vault.
- Strong SQL skills across large datasets, including performance tuning (joins, partitions, file sizing).
- Experience with data quality at scale (e.g., Great Expectations/Deequ), monitoring and alerting; debug/backfill playbooks.
- Experience with DevOps for data: Git branching, code reviews, unit/integration testing (pytest/dbx), CI/CD (Azure DevOps/GitHub Actions).
- Experience with Infrastructure as Code (Terraform or Bicep) for Databricks workspaces, cluster policies, ADF, storage.
- Experience with Observability & cost control: Azure Monitor/Log Analytics; cluster sizing, autoscaling, Photon; cost/perf trade-offs.
- Proven experience collaborating with cross-functional stakeholders (analytics, data governance, product, security) to ship and support data products.
Skills
- Spark
- Azure Databricks
- PySpark
- Scala
- Spark SQL
- Delta Lake
- Unity Catalog
- Azure Data Factory
- ADLS Gen2
- Key Vault
- SQL
- Great Expectations
- Deequ
- Git
- pytest
- dbx
- Azure DevOps
- GitHub Actions
- Terraform
- Bicep
- Azure Monitor
- Log Analytics
- Photon
Location
- Chicago, IL
Work Type
- Hybrid
Experience Level
- 3+ years in data engineering
- 2+ years hands-on Azure Databricks
Education Level
- Bachelor’s degree in computer science, data management, information systems, information science or a related field
- Advanced degree preferred
Salary/Compensations
- $100,000-$115,000
Benefits
- Generous medical, dental, vision and other great benefits
- Paid parental and medical leave programs
- 401(k) with a company match component and profit sharing
- 15 days of paid time off plus company holidays
- Tuition reimbursement and student loan repayment assistance
About the Company
- HAVI is a global, privately owned company focused on innovating, optimizing and managing the supply chains of leading brands.
- Offering services in marketing analytics, packaging, supply chain management and logistics, HAVI partners with companies to address challenges big and small across the supply chain, from commodity to customer.
- Founded in 1974, HAVI employs more than 10,000 people and serves customers in more than 100 countries.
Equal Opportunity
- We are an equal opportunity employer and we value diversity at our company.
- We do not discriminate on the basis of race, religion, color, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status.
- We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment.
- Please contact us to request accommodation.
