ETL Data Engineer

As an ETL Data Engineer, you will be responsible for extracting and ingesting data from websites using web crawling and scraping tools. You will design, develop, and maintain tools, services, and workflows to support large-scale data collection, analysis, reporting, and data management.

You will also ensure the quality and accuracy of extracted data by performing validation, troubleshooting scraping issues, and optimizing workflows. The role requires identifying and resolving failures in data pipelines while ensuring scalability and efficiency in data processing.

• Perform data extraction, transformation, and loading (ETL) processes while resolving errors and validating filter criteria.
• Understand and manage data flow processes, including inputs, outputs, and database connectivity.
• Work with ETL jobs and transformations, including inspecting and validating data outputs.
• Understand Salesforce basics, including objects, fields, and lookup relationships.
• Work with database schemas, tables, and views to manage structured data.
• Provide technical expertise in data warehouse development, including ETL/ELT technologies and Change Data Capture (CDC) implementations.
• Analyze fallout reports, identify bugs and errors, and generate output files for troubleshooting.
• Research, design, develop, and enhance ETL pipelines and database functions.
• Design and implement source-to-target data transformations and loading processes.
• Monitor and maintain web scraping workflows to ensure reliability and scalability.

• Bachelor's Degree in Computer Science, Information Technology, or related field, or equivalent practical experience.
• 2 – 5 years of experience in ETL development or data engineering.
• Experience running large-scale web scraping or crawling processes.
• Strong knowledge of Linux/UNIX environments.
• Understanding of HTTP, HTML, JavaScript, and networking concepts.
• Experience with tools and techniques for data crawling, extraction, and processing.
• Strong communication skills in English (written and spoken).

• Familiarity with system monitoring and administration tools.
• Experience with version control systems (e.g., Git) and code review practices.
• Knowledge of Python and SQL for data processing and automation.