About the Role
The Lead Data Engineer will shape, build, and evolve the data capabilities for clients' accounts platforms, providing technical leadership in designing and implementing scalable, reliable, and well-governed data solutions including pipelines, reference data services, and operational data stores.
Responsibilities
- Lead the design, development, and ongoing evolution of robust, scalable data ingestion and processing pipelines.
- Provide technical leadership across ETL/ELT processes, data transformation, and data integration services.
- Design and implement operational data stores and reference data services supporting the Accounts platform and its consumers.
- Establish and maintain effective data quality, validation, monitoring, reconciliation, and observability controls.
- Contribute to the evolution of data architecture, ensuring solutions are scalable, resilient, secure, and aligned with wider technology strategy.
- Define and promote data engineering standards, patterns, and best practices across the team.
- Support the development and implementation of data governance, lineage, and data management practices.
- Work closely with software engineers, architects, product teams, and other stakeholders to understand data requirements and translate them into effective technical solutions.
- Provide technical guidance and mentoring to other data engineers, helping to raise engineering capability and consistency across the team.
- Champion automation, Infrastructure as Code, and modern engineering practices across the data landscape.
- Identify opportunities to improve the reliability, performance, maintainability, and cost-effectiveness of data services and pipelines.
Requirements
- Strong SQL skills and extensive experience working with relational and/or modern database technologies.
- Proven commercial experience developing data solutions using Python.
- Strong understanding of modern data engineering principles, practices, and design patterns.
- Experience designing and implementing ETL/ELT pipelines and data processing frameworks.
- Experience working with data quality, reconciliation, monitoring, and operational data management.
- Experience contributing to data architecture and governance standards.
- Experience with Infrastructure as Code, ideally using Terraform.
- Experience working with cloud-based data platforms and modern engineering tooling.
- Experience leading technical delivery and making pragmatic architectural decisions in a complex technology environment.
Skills
- SQL
- Python
- Data Engineering
- ETL/ELT
- Data Processing
- Data Quality
- Data Governance
- Infrastructure as Code
- Terraform
- Cloud Data Platforms
- Modern Engineering Tooling
Location
- UK
- Europe
Work Type
- Blended Working
- Full-time
Experience Level
- Lead
Benefits
- Opportunities to work on projects with big clients and the chance to produce meaningful work that makes a difference to people’s lives.
- A “Blended Working” model allowing work from home, clubhouse, or client locations.
- A dedicated career scrum team for career goal development.
- An annual budget for training and upskilling with allocated days off.
- Monthly and quarterly team socials.
- A safe environment to be yourself and challenge yourself.
About the Company
- AND Digital is a tech company focused on accelerating digital delivery and closing the digital skills gap.
- Founded in 2014, they help organizations build better digital products and stronger digital teams.
- Their unique model prioritizes clients through regional 'Clubs', fostering close relationships.
- They have experienced organic growth, now numbering over 1,300 people with Clubs across the UK and Europe, with plans for global expansion.
- Their mission is to close the world's digital skills gap.
- Company values are wonder, share, and delight, influencing their work culture.
- They believe collaboration, ambition, curiosity, and fun drive innovation.
Equal Opportunity
- AND Digital embraces diversity and is committed to equal opportunities.
- They are actively recruiting for a diverse and inclusive workforce.
- They ensure they do everything they can to support applications and offer adjustments to the application or interview process upon request.
