Senior Data Engineer

Senior Data Engineer

Role Overview

• Design, develop, and maintain data pipelines and ETL workflows using AWS services.
• Implement orchestration and automation for data workflows.
• Work with large datasets to ensure data integrity, scalability, and performance.
• Collaborate with stakeholders to understand data requirements and deliver solutions.
• Deploy changes directly to production environments with confidence and accountability.
• Support migration efforts to Databricks and optimize workflows for performance

Qualifications

• 7 to 10 years of experience and above
• Experience with data lake architectures, big data technologies, and data pipeline orchestration.
• Familiarity with CI/CD practices for data engineering.
• AWS Certification (e.g., AWS Certified Data Analytics – Specialty or Solutions Architect) is a plus.
• Strong problem-solving skills and attention to detail.

Key Skills and Requirements

• Expert-level fluency in AWS services relevant to data engineering, including Glue, S3, Lambda, Step Functions, and other related services.
• Strong proficiency in PySpark for distributed data processing.
• Advanced SQL skills for querying and optimizing data operations.
• Comfortable with pushing changes live without formal review, ensuring quality through self-validation.
• Databricks experience is a plus.
• Ability to work independently and manage tasks in a siloed environment.
• Currently based in Kuala Lumpur, Malaysia