Impress employers and recruiters.
Choose from hundreds of resume examples.
Impress employers and recruiters.
Choose from hundreds of resume examples.
Target Resume Now
About the Role
Join a specialized technical team focused on building the infrastructure that delivers massive amounts of web data for the training of advanced AI models. This organization operates a massive distributed crawler and manages complex pipelines for ingesting, segmenting, and annotating billions of data points. The successful candidate will lead efforts to gather and analyze data, optimize scraping processes, and support the scaling of high-quality public web data accessibility. This role is ideal for a lean, technical builder who thrives in a fast-paced environment without bureaucratic red tape.
Responsibilities
- Write, test, and refine high-performance code to extract data from various online sources, ensuring maximum reliability and efficiency.
- Manage complex data retrieval tasks, including handling pagination and dynamic content loaded via AJAX.
- Clean and format extracted data to ensure it meets rigorous quality standards for downstream analysis and processing.
- Store and manage scraped data in appropriate databases, optimizing for both access speed and long-term data integrity.
- Regularly monitor scraping processes and infrastructure to identify and resolve issues, ensuring a continuous and stable data flow.
Requirements
- Demonstrated ability to extract data from complex websites with minimal supervision, supported by a portfolio of past projects.
- Advanced skills in Python or JavaScript, specifically with libraries and frameworks such as BeautifulSoup, Scrapy, or Selenium.
- Strong knowledge of asynchronous programming, multithreading, and distributed scraping architectures.
- In-depth knowledge of HTML, CSS, JavaScript, and the Document Object Model (DOM).
- Experience with NoSQL databases (e.g., MongoDB, Cassandra), including the ability to design efficient storage solutions.
- Experience deploying and managing large-scale scraping jobs using cloud services such as AWS, Google Cloud, or Azure.
Skills
- Python
- JavaScript
- BeautifulSoup
- Scrapy
- Selenium
- Asynchronous programming
- Multithreading
- Distributed scraping architectures
- HTML
- CSS
- DOM
- NoSQL databases
- MongoDB
- Cassandra
- AWS
- Google Cloud
- Azure
- Machine learning algorithms
- Data cleaning
- Data categorization
- Predictive analysis
- Open-source projects
Location
- Remote
Work Type
- Remote
- Full-time
Experience Level
- Advanced
Salary/Compensations
- $75,000 - $100,000
Benefits
- Comprehensive benefits and equity package
About the Company
- MLabs is committed to offering equal opportunities to all candidates.
- We ensure no discrimination, accessible job adverts, and providing information in accessible formats.
- Our goal is to foster a diverse, inclusive workplace with equal opportunities for all.
- MLabs Ltd collects and processes personal information for recruitment purposes only.
- Information is managed securely in accordance with MLabs Ltd’s Privacy Policy and Information Security Policy, and in compliance with applicable data protection laws.
- Your data may be shared only with clients and trusted partners where necessary for recruitment purposes.
Equal Opportunity
- MLabs is committed to offering equal opportunities to all candidates.
- We ensure no discrimination, accessible job adverts, and providing information in accessible formats.
- Our goal is to foster a diverse, inclusive workplace with equal opportunities for all.
- If you need any reasonable adjustments during any part of the hiring process or you would like to see the job-advert in an accessible format please let us know at the earliest opportunity by emailing human-resources@mlabs.city.