Data Engineer/Senior Data Engineer
Infinite Computer Solutions Pte Ltd- Posted a month ago
- Be among the first 50 applicants
Job Description
Role: Data Engineer/Senior Data Engineer
Location: Kuala Lumpur(Onsite)
Duration: 12 Months (Renewable) Contract
Databricks.
Implement scalable data processing and transformation logic in PySpark and SQL on Databricks (Delta
Lake / lakehouse patterns). Ingest and integrate data from diverse batch and streaming sources into
ADLS Gen2 and Delta Lake.
Build curated, documented datasets for analytics and ML (support dimensional models where
appropriate).
Implement automated data quality checks, unit/integration tests, and validation gates. Monitor pipeline
health, implement logging, alerting and dashboards; perform RCA and production incident remediation.
Optimize pipeline performance and cost: cluster sizing, job scheduling, partitioning, caching and query
tuning.
Implement and maintain CI/CD for data jobs and infrastructure (Terraform/ARM/Bicep, Azure DevOps or
GitHub Actions). Candidate knows Cantonese .
Interested candidates can connect on +6586533349 (WhatsApp chat only)
More Info
Key Skills
About Company
