About the Role
We are seeking a skilled mid-level+ Data Engineer to focus on quality assurance, quality checking, and ETL processes. The role involves ensuring data integrity from file transfer services to S3 and Snowflake, supporting critical downstream applications and reporting systems.
Responsibilities
- Implement and maintain data quality for accuracy and reliability throughout the ETL process.
- Design, develop, and optimize ETL workflows for data transfer from file services to S3 and Snowflake.
- Ensure seamless data integration into the data platform for consumption by downstream applications and reporting tools.
- Address data quality challenges, including inconsistencies in source data that impact ingestion and may cause load failures or backouts.
- Collaborate with business owners, data analysts, BI teams, and stakeholders to understand data requirements and deliver data solutions.
- Monitor pipelines, identify issues, and implement solutions to maintain data flow and integrity.
Requirements
- Demonstrated mid-level+ experience in data engineering, with an emphasis on data quality assurance and ETL processes.
- Expertise in Python, PyPI, and SQL.
- Strong understanding of cybersecurity principles related to code development, DevOps, data access, and fundamental cybersecurity.
- Understanding of fundamental public-cloud capabilities.
- Proven capacity to comprehend business needs and convert them into technical requirements.
- Demonstrated excellence in communication and collaboration abilities.
- Proven capacity to define success, deliver, and operate in an agile setting.
Skills
- Python
- PyPI
- SQL
- Data Quality Assurance
- ETL Processes
- Data Integration
- Data Quality Management
- Cybersecurity Principles
- Public Cloud Capabilities
- Agile Methodologies
Experience Level
- Mid-level+
