Senior Data Engineer

We are hiring for a tech company with end client in banking Data and Senior Data Engineers based in Malaysia.

Role Summary

 We are seeking a highly motivated and skilled Big Data Platform Engineer to design, develop, and maintain scalable data processing platforms and pipelines. The ideal candidate should have strong expertise in Python, Apache Spark, SQL, and experience working in Linux-based environments with exposure to DevOps practices. The role involves building reliable, high-performance data solutions that support enterprise-scale analytics and business-critical applications.

 Key Responsibilities

Data Engineering & Development

• Design, develop, and maintain scalable data pipelines using Python and Apache Spark.
• Implement efficient ETL/ELT processes for large-scale structured and unstructured datasets.
• Develop and optimize complex SQL queries, data models, and transformations.
• Ensure data quality, integrity, and reliability across the platform.

Platform Operations

• Work with Linux-based environments for deployment, troubleshooting, and performance tuning.
• Develop and maintain shell scripts for automation and operational tasks.
• Monitor and optimize Spark jobs for performance, scalability, and resource utilization.

DevOps & Automation

• Implement CI/CD pipelines and deployment automation.
• Participate in infrastructure provisioning, monitoring, and release management activities.
• Collaborate with DevOps teams to improve platform reliability and operational efficiency.

Mandatory Experience

6-8 years of experience or 8- 12 years of experience or 12 + years of experience in data engineering or Big data platform development.

Core Big Data Platform

MinIO - S3 or AWS S3 or other S3-compatible object storage

Apache delta lake/ apache Hudi

Kuberntes SKE/AKS /EKS/GKE

SQL

Devops

Linux and shell scripting

Python and spark

If this sounds like you then please apply!