Own end-to-end engineering delivery for data pipelines, curated datasets, and ML engineering outcomes (feature/serving pipelines and model integration), ensuring strong engineering practices (version control, testing, CI/CD), data quality, observability, and operational readiness
Client Details
Our client haven't just been in the business of supply chain; they have been in the business of re-imagining connections between people and the products they love to create a better future
Description
- Lead and grow the Data Engineering & ML Engineering team (hiring, onboarding, performance management, coaching, career development).
- Act as a hands-on technical lead (player/coach): drive technical direction, review designs and code, unblock complex engineering problems, and set reusable patterns and standards
- Own end-to-end engineering delivery for data pipelines and curated datasets (data products) on Azure Databricks, ensuring quality, reliability, security and operational readiness.
- Lead ML engineering delivery in partnership with Data Science and MLOps (feature engineering pipelines, training/inference integration, model packaging and deployment readiness, monitoring inputs/outputs).
- Enforce engineering standards: CI/CD, automated testing, code reviews, documentation, data contracts, and data quality validation/reconciliation
Profile
A successful Data Engineering Team Lead should have:
- Core tech stack (hands-on): Azure (ADLS, ADF), Azure Databricks (Spark/Delta/Lakehouse patterns), Python, SQL, Git, CI/CD (Azure DevOps or GitHub), orchestration (ADF and/or Databricks Workflows/Jobs), and observability/monitoring for data pipelines. Preferred: dbt and modern data quality tooling (validation, reconciliation, data contracts
- 8+ years in data and/or software engineering roles, including hands-on delivery on Azure Databricks, with 3+ years of people management
- Solid Python, SQL, and Spark engineering skills (performance tuning, partitioning strategies, cost/performance trade-offs in Databricks jobs and clusters).
Job Offer
- The People! The Work! The Environment
- Competitive Salary: we offer a salary that reflects your experience, and expertise
- Work-Life Balance: Hybrid Working - 3 days in office, 2 days from home
- Professional Growth: access to ongoing training
- Inclusive Environment: we prioritize diversity and inclusion, ensuring everyone feels welcome and valued.
If you are passionate about data engineering and leadership and are looking for a permanent role, we encourage you to apply now!
To apply online please click the Apply button below. For a confidential discussion about this role please contact Arshanaa Lechumi on +60323024087.
