Data Engineer

We are hiring for an IT client Data and Sr. Data Engineer in Malaysia. No visa sponsorship will be given.

This is for locals Malaysians only.

Role Summary

We are seeking a highly motivated and skilled Big Data Platform Engineer to design, develop, and maintain scalable data processing platforms and pipelines. The ideal candidate should have strong expertise in Python, Apache Spark, SQL, and experience working in Linux-based environments with exposure to DevOps practices. The role involves building reliable, high-performance data solutions that support enterprise-scale analytics and business-critical applications.

Key Responsibilities

Data Engineering & Development

• Design, develop, and maintain scalable data pipelines using Python and Apache Spark.
• Implement efficient ETL/ELT processes for large-scale structured and unstructured datasets.
• Develop and optimize complex SQL queries, data models, and transformations.
• Ensure data quality, integrity, and reliability across the platform.

Mandatory Skills

Core Big Data Platform Skills

• Strong programming experience in Python
• Hands-on expertise with Apache Spark (PySpark preferred)
• Strong SQL development and query optimization skills
• Apache Airflow for workflow orchestration and scheduling
• Kubernetes (AKS/SKE) for container orchestration and deployment. AKS/EKS/GKE also fine.
• MinIO / S3 Compatible Object Storage like AWS S3 or other S3-compatible object storage experience
• Apache Iceberg/Delta Lake / Apache Hudi

Technical Competencies

• Data Pipeline Development
• ETL/ELT Processing
• Data Modelling
• Performance Tuning
• Version Control (Git)
• Agile/Scrum Delivery Model

Desired Experience

• 6 to 8 years of experience or 8 to 12 years or 12+ years of experience in Data Engineering or Big Data Platform Development.
• Experience working with enterprise-scale data platforms.
• Experience in distributed computing environments.
• Strong analytical and problem-solving skills.