Data Engineer

Data Engineer

Kuala Lumpur

12 Months Contract

Rm 7500 -Rm 8000

Immediate Joiner – 1 Month Notice Only – NO VISA PROVIDED

Required Qualifications

Mandatory - Minimum 3+ years of experience in Data Engineering, Data Warehousing, Data Platform Engineering, or related roles.

·      Strong hands-on experience in:

o  SQL

o  Python

o  PySpark

o  Terraform

• Bachelor's Degree in Computer Science, Information Technology, Data Engineering, Engineering, Information Systems, or a related discipline.
• Minimum 3+ years of experience in Data Engineering, Data Warehousing, Data Platform Engineering, - Insurance Domain
• Strong hands-on experience in:
• SQL
• Python
• PySpark
• Terraform
• Strong experience with AWS services including:
• AWS Glue
• Amazon Redshift
• Amazon S3
• Amazon RDS
• Amazon SNS
• AWS Step Functions
• Amazon CloudWatch
• AWS IAM
• Amazon EventBridge
• AWS DataZone
• Experience designing and supporting enterprise Data Lake and Data Warehouse solutions.
• Experience working with Apache Iceberg tables and large-scale data processing platforms.
• Strong understanding of ETL/ELT methodologies, data modeling, and data integration techniques.
• Experience implementing Infrastructure as Code (IaC) using Terraform.
• Strong understanding of cloud security, IAM permissions, and access control principles.
• Experience planning, executing, and supporting Disaster Recovery (DR) activities, including failover testing, recovery validation, and business continuity processes.
• Knowledge of Agentic AI, Generative AI, and AI-assisted software development practices.
• Experience utilizing AI-powered coding assistants and modern AI development workflows to improve engineering productivity, accelerate delivery, and enhance code quality.
• Strong analytical, problem-solving, troubleshooting, and communication skills.
• Ability to work independently and collaboratively in a fast-paced environment.

Preferred Qualifications

• AWS Certifications, such as:
• AWS Certified Data Engineer
• AWS Certified Solutions Architect
• AWS Certified Developer
• AWS Certified Cloud Practitioner
• Experience with Power BI.
• Knowledge of SAP BusinessObjects (SAP BO).
• Experience implementing AWS DataZone and enterprise data governance frameworks.
• Experience with Data Lakehouse architecture.
• Familiarity with DevOps, CI/CD, and cloud automation practices.
• Experience within the Insurance or Financial Services industry.

Key Competencies

• Data Engineering Excellence
• AWS Cloud Technologies
• Data Lake & Data Warehouse Architecture
• ETL/ELT Development
• Apache Iceberg
• Infrastructure as Code (Terraform)
• AWS DataZone
• Data Governance & Metadata Management
• Disaster Recovery & Business Continuity
• AI-Enabled Data Engineering
• Agentic AI & Generative AI
• Cloud Security & IAM
• Data Quality Management
• Technical Documentation
• Knowledge Transfer & Mentoring
• Stakeholder Management
• Problem Solving & Critical Thinking
• Continuous Improvement & Innovation

Technical Environment

Cloud Platform

• AWS

AWS Services

• AWS Glue
• Amazon Redshift
• Amazon S3
• Amazon RDS
• Amazon SNS
• AWS Step Functions
• Amazon CloudWatch
• AWS IAM
• Amazon EventBridge
• AWS DataZone

Programming & Development

• SQL
• Python
• PySpark
• Terraform
• Git

Data Technologies

• Apache Iceberg
• Data Lake
• Data Warehouse
• ETL/ELT Frameworks
• Data Governance

Analytics & Reporting

• Power BI
• SAP BusinessObjects (SAP BO)

Emerging Technologies

• Agentic AI
• Generative AI
• AI-Assisted Software Development

Nice-to-Have Technologies

• Data Lakehouse Architecture
• Cloud Automation
• CI/CD Pipelines

Key Responsibilities

Data Engineering & Data Platform Development

• Design, develop, and maintain scalable, reliable, and secure data pipelines to support enterprise analytics and reporting initiatives.
• Build and optimize ETL/ELT processes to ingest, transform, and deliver data from various source systems into Data Lake and Data Warehouse environments.
• Design, implement, and maintain cloud-native data solutions on AWS.
• Develop scalable batch and near real-time data processing solutions.
• Develop and maintain Apache Iceberg datasets to support enterprise analytical workloads.
• Ensure data from source systems is successfully loaded into the enterprise Data Lake daily with accuracy, completeness, and timeliness.
• Monitor and troubleshoot data ingestion processes and resolve data-related issues proactively.
• Conduct root cause analysis and implement sustainable solutions for recurring incidents.
• Optimize data processing performance, scalability, and operational efficiency.
• Support enterprise reporting, analytics, and regulatory data requirements.

AWS Cloud Platform & Infrastructure

• Design, build, and support AWS-based data platforms using:
• AWS Glue
• Amazon Redshift
• Amazon S3
• Amazon RDS
• Amazon SNS
• AWS Step Functions
• Amazon CloudWatch
• AWS IAM
• Amazon EventBridge
• AWS DataZone
• Develop and maintain Infrastructure as Code (IaC) solutions using Terraform.
• Automate environment provisioning, deployment, and operational processes.
• Implement monitoring, alerting, and operational support frameworks to ensure platform stability and high availability.
• Optimize cloud infrastructure for performance, security, reliability, and cost efficiency.
• Support and enhance CI/CD deployment pipelines and DevOps practices.

Disaster Recovery & Business Continuity

• Participate in Disaster Recovery (DR) planning, validation, and execution activities for critical data platforms and services.
• Perform Disaster Recovery (DR) drills, failover, failback, and recovery procedures within established Recovery Time Objective (RTO) and Recovery Point Objective (RPO) requirements.
• Ensure data pipelines, Data Lake, and Data Warehouse platforms can be recovered and restored during disaster scenarios.
• Maintain and regularly update Disaster Recovery documentation, runbooks, and recovery procedures.
• Collaborate with infrastructure, security, and application teams to ensure business continuity readiness.

Data Governance & Quality Management

• Establish and maintain data quality validation, reconciliation, and monitoring controls.
• Ensure data accuracy, consistency, completeness, and reliability across the data platform.
• Support enterprise data governance initiatives through AWS DataZone, including metadata management, data ownership, and data discoverability.
• Collaborate with stakeholders to define and implement data standards, governance policies, and best practices.
• Ensure compliance with enterprise security, privacy, regulatory, and audit requirements.

Stakeholder Collaboration

• Collaborate closely with Business Analysts, Data Analysts, BI Developers, Architects, Product Owners, and business stakeholders to understand and deliver data requirements.
• Translate business requirements into scalable technical designs and data models.
• Provide technical leadership and guidance on data engineering best practices.
• Partner with cross-functional teams to continuously improve data platform capabilities.

Documentation & Knowledge Transfer

• Create and maintain comprehensive technical documentation, including:
• Solution Design Documents
• Technical Specifications
• Data Flow Diagrams
• Runbooks
• Support Guides
• Operational Procedures
• Disaster Recovery Procedures
• Conduct Knowledge Transfer (KT) sessions to ensure team members understand implemented solutions, processes, and support procedures.
• Promote knowledge sharing, engineering standards, and best practices across the team.
• Mentor junior engineers and support team capability development.