Impress employers and recruiters.
Choose from hundreds of resume examples.

Impress employers and recruiters.
Choose from hundreds of resume examples.

Target Resume Now

About the Role

Pantheon is building a cloud-native, AI-ready Data Platform using the Databricks Lakehouse architecture. This role is crucial for designing and implementing new data pipelines to adapt to business and technology changes, establishing Pantheon as a data-centric organization. You will work with a modern Azure tech stack and contribute to a small, skilled team focused on delivering best-in-class solutions.

Responsibilities

• Design, build, and maintain scalable, secure, and high-performance data pipelines on Azure using Azure Databricks, Azure Data Factory, and Azure Functions.
• Develop and optimize batch and streaming data processing solutions using PySpark and SQL.
• Implement robust data transformation layers using dbt.
• Collaborate with business analysts, QA teams, and stakeholders to translate data requirements into technical solutions.
• Ensure data quality, reliability, and observability through automated testing, monitoring, logging, and alerting.
• Lead performance tuning, cost optimization, and capacity planning for Databricks and associated Azure services.
• Implement and maintain CI/CD pipelines using Azure DevOps.
• Enforce data governance, security, and compliance standards.
• Contribute to architectural decisions and provide technical leadership, mentoring junior engineers.
• Produce clear technical documentation and contribute to knowledge sharing.
• Enable ML/AI workloads on the Databricks data platform.
• Support Databricks AI capabilities.
• Collaborate with the AI Product Team to deliver use cases.
• Enable RAG pipelines / vector storage patterns to support AI products.

Requirements

• Significant commercial experience (typically 5+ years) in data engineering roles.
• Demonstrable experience designing and operating production-grade data platforms.
• Strong hands-on experience with Azure Databricks, including cluster configuration, job orchestration, and performance optimization.
• Proven experience building data pipelines with Databricks and Azure Data Factory, integrating with Azure-native services.
• Advanced experience with Python for data engineering, including PySpark for distributed data processing.
• Strong SQL expertise, with experience designing and optimizing complex analytical queries and data models.
• Practical experience using dbt in a production environment.
• Experience implementing CI/CD pipelines using Azure DevOps or equivalent tooling.
• Data as a Product mindset.
• Experience with AI / ML / GenAI Enablement.
• Experience in Agile SW development environment.
• Strong problem-solving ability and a pragmatic, engineering-led mindset.
• Excellent communication skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders.
• Leadership and mentoring capability.

Skills

• Azure Databricks (Spark, Delta Lake, performance tuning)
• Python and PySpark
• SQL (advanced querying, optimisation, and data modelling)
• Azure Data Factory
• dbt
• Azure DevOps (Git, CI/CD pipelines, release management)
• Lakehouse architecture (Databricks Unity Catalog, Delta Lake optimization techniques)
• Data modelling (star schemas, data vault, or lakehouse-aligned approaches)
• Data quality, testing frameworks, and monitoring/observability
• Development with coding agents (e.g., Anthropic Claude Code, OpenAI Codex etc)

Location

• Remote

Work Type

• Full-time

Experience Level

• Senior

About the Company

• Pantheon has been at the forefront of private markets investing for more than 40 years, earning a reputation for an innovative approach to investing in secondaries, co-investments, and primary fund investments, as well as capital formation across commingled funds, evergreen vehicles and customized solutions.
• Our specialist investment capabilities span multiple strategies across private equity, infrastructure and real assets, and private credit.
• Through our collaborative and committed culture, we find new ways to solve complex problems together and deliver innovative investment opportunities across private markets.
• Pantheon currently manages approximately $82.3 billion in AUM across all its strategies, serving more than 750 institutional and 638 private wealth clients worldwide.

Equal Opportunity

• Pantheon is an Equal Opportunities employer, we are committed to building a diverse and inclusive workforce so if you're excited about this role but your past experience doesn't perfectly align we'd still encourage you to apply.