About the Role
Design, develop, and optimize scalable cloud-based data solutions on Microsoft Azure. The role involves building robust data pipelines, transforming raw data into analytics-ready datasets, implementing best practices for data engineering, and collaborating with cross-functional teams to deliver high-quality, scalable data solutions.
Responsibilities
- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF V2).
- Build automated data ingestion frameworks to process bulk data from multiple source systems into target platforms.
- Implement Watermark and Change Data Capture (CDC) strategies for incremental data loading.
- Design and develop modern Azure cloud data architectures for enterprise-scale applications.
- Create and manage secure, scalable, and cost-effective hosting solutions on Microsoft Azure.
- Develop, optimize, and maintain dbt models, macros, seeds, snapshots, and tests to transform raw data into analytics-ready datasets.
- Work extensively with Snowflake to develop and optimize data models, transformations, and performance.
- Implement version control, documentation, and deployment best practices for dbt projects.
- Handle and process multiple file formats including JSON, Parquet, and other structured or semi-structured data.
- Collaborate with Data Analysts, Data Scientists, Architects, and Business teams to understand data requirements.
- Optimize data pipelines for performance, scalability, reliability, and cost efficiency.
- Ensure data quality by implementing testing, validation, and monitoring mechanisms.
- Troubleshoot and resolve data pipeline and platform-related issues.
- Follow industry best practices for security, governance, and data management.
Requirements
- Bachelor’s degree in computer science, Information Technology, Engineering, or a related field.
- 5–10 years of experience in Data Engineering or Cloud Data Engineering.
- Strong expertise in Microsoft Azure Data Services.
- Hands-on experience with Azure Data Factory (ADF V2).
- Experience designing and implementing Bulk Data Ingestion Frameworks.
- Experience designing and implementing Watermark-based Incremental Loads.
- Experience designing and implementing Change Data Capture (CDC).
- Strong understanding of Azure cloud architecture and data platform design.
- Experience working with Snowflake as a cloud data warehouse.
- Hands-on experience with dbt (Data Build Tool), including dbt Models, Macros, Tests, Documentation, and Version Control.
- Experience processing data in formats such as JSON, Parquet, and CSV.
- Strong SQL programming and query optimization skills.
- Experience with Git-based version control.
Skills
- Azure Data Factory (ADF V2)
- Data Ingestion Frameworks
- Watermark and Change Data Capture (CDC)
- Azure Cloud Data Architecture
- dbt (Data Build Tool)
- Snowflake
- JSON
- Parquet
- SQL
- Git
Location
- Remote
Work Type
- Remote
- Full-time
Experience Level
- 6+ years
- 5-10 years
Education Level
- Bachelor’s degree in computer science, Information Technology, Engineering, or a related field.
Benefits
- Dental insurance
- Health insurance
- Paid time off
- Training & development
- Vision insurance
About the Company
- Rysun Labs (formerly KCS – Krish Compusoft Services) is an AI, Data & Digital innovation partner of choice for enterprises.
- Rysun guides and accelerates the AI & Data strategy and Digital Transformation programs for Fortune 2000 enterprises and product startups to shape remarkable customer experiences and intelligent operations.
- The team delivers innovative, specialized solutions that help High-tech, Retail & Ecommerce, and Energy companies to outperform competition and lead the change in their industry.
- Rysun partners with Microsoft, Google and AWS to bring the best of enterprise technology to its customers.
- Rysun believes in quality-first and is CMMI Level 5, ISO 9001 & 27001 certified.
- The team has a growth mindset fueled by a strong culture of collaboration that unifies its global teams across US, UK, India & South Africa (a proud Level 2 B-BBEE Contributor).
