The Opportunity
Own the data engineering quality bar across our group data platforms. You'll architect scalable data solutions, drive innovation, and ensure robust data governance, working closely with the UK-based Lead Data Engineer and lead a team of Data Engineers in Malaysia.
The successful candidate will be expected to partner with key stakeholders to deliver high-quality data solutions aligned with organisational goals.
Core Responsibilities
- Collaboration: Work closely with UK teams and the UK Lead Data Engineer to ensure alignment on data strategy, architecture, and processes.
- Big Data Architecture: Design and implement scalable, secure big data solutions using Azure Fabric.
- Data Modelling & Governance: Develop and maintain robust data models and enforce data governance standards across all environments.
- Pipeline Management: Build, optimise, and manage ELT/ETL pipelines, leveraging DBT and modern orchestration tools.
- System Integration: Deliver hands-on system-to-system integrations (HR, Finance, CRM, ERP, Operational Systems), ensuring reliability and scalability.
- API & Integration Patterns: Architect and maintain APIs, monitor integrations, and implement effective error handling strategies.
- Code Quality & Agile Delivery: Champion CI/CD, conduct code reviews, and drive Agile methodologies within the data engineering team.
- SQL & Python Excellence: Apply strong SQL and Python skills to solve complex data challenges and automate processes.
- AI & Machine Learning: Collaborate with data scientists to enable and support AI/ML initiatives, ensuring data readiness and scalability.
- Medallion & Data Architecture: Implement medallion architecture and best practices for data storage, processing, and consumption.
- Innovation: Evaluate and recommend new tools and technologies to enhance data capabilities and drive continuous improvement.
- Mentoring & Management: Coach and develop other data engineers, fostering capability, confidence, and technical excellence.
Essential Skills & Tech Stack
- 5+ years commercial experience in data engineering, with hands-on expertise in Azure Fabric, Databricks, and Snowflake in production environments.
- Proven experience leading a multi-disciplined data engineering team.
- Deep understanding of big data architecture, data modelling, and data governance.
- Strong experience with CI/CD, code reviews, and Agile delivery (Scrum/Kanban).
- Advanced skills in SQL and Python.
- Hands-on experience with DBT, ELT/ETL pipeline development, and system integrations.
- Familiarity with APIs, integration patterns, monitoring, and error handling.
- Experience supporting AI/Machine Learning initiatives.
- Track record of recommending and implementing innovative data solutions.
- Demonstrated ability to mentor and manage other data engineers.
