About the Role
Design, build, and implement generalized large-scale, sophisticated data pipelines using Nifi for downstream analytics and data science for our Sport Performance products. Design and develop scalable Nifi ingestion pipelines within AWS cloud services to consume real-time and batch data from external sources. Ensure the seamless integration of AWS-based tools for data storage, processing, and analytics.
Responsibilities
- Design, build, and implement generalized large-scale, sophisticated data pipelines using Nifi for downstream analytics and data science for our Sport Performance products.
- Design and develop scalable Nifi ingestion pipelines within AWS cloud services to consume real-time and batch data from external sources.
- Ensure the seamless integration of AWS-based tools for data storage, processing, and analytics.
- Responsible for ETL development and warehousing using Python and Java.
- Create data pipeline triggers and filters within ETL (extract, transform, and load) process to ensure appropriate optimization of data flowing through system and resource usage.
- Implement monitoring and error handling for all new parts of the data pipeline to ensure observability and alerting is available.
- Establish rigorous unit testing across the data pipeline to ensure robustness of the system.
- Design and create data models for use throughout the ETL system.
- Utilize Kafka to efficiently and to effectively store data to move throughout the data pipeline and for downstream data science and analytics usage.
- Design data architecture and data models for both internal and external representations of data.
- Build the data transforms within the data pipeline to convert data from external to internal representations.
- Conduct data analytics and debugging of bad data by writing SQL queries.
- Build automated cleaning of data to remove bad or unusable data from downstream consumers with logging to understand the frequency and depth of the underlying issues.
- Collaborate with other engineering teams to adopt standard methodologies, drive scalability, and increase consistency across systems.
- Maintain awareness of company standards and technology guidance; use JIRA, an Agile project mgmt. tool, to ensure efficient data development; collaborate with peers to align projects with overall direction.
- Follow best practices across Data Engineering to ensure scalable, consistent data architecture and system.
- Utilize Java language to build data processor in Nifi framework.
- Utilize Docker to ensure consistent, repeatable, and isolated environment for software development and testing.
- Work in a self-driven, independent fashion to meet Sport driven deadlines.
Requirements
- Master’s degree in Computer Science, Computer Engineering, or closely related field and 1 year experience as a data engineer or related occupation.
- 1 year of experience with Python, Java, Kafka, AWS, and Docker
- 1 year of experience with ETL Development and Warehousing
- 1 year of experience with Analytic and debugging using SQL
- 1 year of experience in Agile development environment
- 1 year of experience Designing data architecture
Skills
- Nifi
- AWS cloud services
- Python
- Java
- ETL
- Warehousing
- Kafka
- SQL
- JIRA
- Docker
- Data Engineering
- Data Architecture
- Data Models
Location
- U.S.
Work Type
- Telecommuting
- Remote
Experience Level
- 1 year experience as a data engineer or related occupation
Education Level
- Master’s degree in Computer Science, Computer Engineering, or closely related field
Salary/Compensations
- Base pay within range, dependent on job-related, non-discriminatory factors including but not limited to experience, qualifications, and location.
Benefits
- Comprehensive benefits package
- Performance bonus program
- Equity stock purchase
- 401k contribution
- Collaborative environment
- Offices in Europe, Asia and US
- Social events and teambuilding
- Flexibility to manage your workday and tasks with autonomy
- Balance of structure and autonomy
- Vibrant and inclusive community
- Women in Tech and Pride groups
- Global Employee Assistance Programme
- Calm and Reulay app
- Online training videos
- Flexible working hours
About the Company
- We’re the world’s leading sports technology company, at the intersection between sports, media, and betting.
- More than 1,700 sports federations, media outlets, betting operators, and consumer platforms across 120 countries rely on our know-how and technology to boost their business.
Equal Opportunity
- All qualified applicants will receive consideration for employment without regard to age, race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or status as a protected veteran.
- We encourage you to apply even if you only meet most of the requirements (but not 100% of the listed criteria) – we believe skills evolve over time.
- If you’re willing to learn and grow with us, we invite you to join our team!
