Principal Data Engineer

Role Mission:

To provide technical leadership within StarHub's Digital Experience Platform (DXP) Data organization by designing, delivering, and operationalizing complex data pipelines, curated datasets, and reusable engineering patterns on the cloud-native data platform. This role drives technical excellence across data ingestion, transformation, modeling, DataOps, and production reliability to enable trusted, scalable, and self-service analytics across business domains.

Accountabilities:

1. Own technical delivery of complex, high-impact data engineering initiatives across ingestion, transformation, modeling, and operational stabilization.
2. Serve as the senior technical leader within the Data Engineering function, setting implementation direction, reviewing design quality, and uplifting engineering standards across the team.
3. Drive production reliability, observability, and root-cause elimination for critical pipelines and datasets.
4. Develop reusable engineering patterns, frameworks, and automation to improve delivery speed, quality, and maintainability.
5. Partner with Data Architecture, Platform Engineering, Data Quality Stewards, BI, and business stakeholders to translate requirements into trusted and scalable data products.
6. Coach and mentor engineers through design reviews, code reviews, troubleshooting, and day-to-day technical guidance without direct people management responsibility.

Responsibilities:

1. Technical Delivery & Solution Design: Lead design and implementation of complex ingestion, transformation, and curated data model solutions across Datapipe, Snowflake, and AWS, ensuring scalable, reusable, and cost-efficient patterns.
2. Engineering Standards & Quality: Establish and enforce practical engineering standards across SQL, Python, DAG design, CI/CD, testing, observability, RBAC-aware implementation, and cost-aware design.
3. Operational Excellence: Own production stability for critical pipelines and datasets, including incident triage, recovery leadership, RCA, and preventative improvement actions.
4. Reusable Enablement: Build reusable components, templates, runbooks, and agentic delivery patterns to reduce duplicated effort, improve maintainability, and raise engineering velocity.
5. Data Quality & Trusted Data: Embed automated data quality controls into pipelines and curated layers, including validation, anomaly detection, reconciliation, and schema drift checks.
6. Collaboration & Enablement: Work with architects, stewards, platform engineers, BI teams, and business stakeholders to shape requirements into implementable data contracts and trusted datasets for self-service analytics.
7. Technical Leadership by Influence: Act as the senior technical escalation point for difficult engineering and production issues, while coaching Senior Data Engineers and Data Engineers through design and implementation guidance.

Team Scope/ Stakeholders:

1. Scope: Complex pipelines, curated datasets, reusable engineering patterns, and production reliability across the DXP Data Platform (C360, Datapipe ingestion solution based on Apache Airbyte & Airflow, Snowflake, SageMaker, Cloud native skills).
2. Decision Rights: Technical design decisions within assigned initiatives, implementation patterns, code quality expectations, incident recovery actions, and recommendations on engineering prioritization and standards uplift.
3. Stakeholders: Data Engineering, Platform Engineering, Architecture & Governance, BI, Data Science, Data Quality Stewards, Business Data Owners, Infrastructure, Cybersecurity/ISO, and Application domain teams.
4. Resources: Individual contributor role operating as the senior-most hands-on engineer within the Data Engineering team, with responsibility to guide and uplift engineers across Singapore, Malaysia and India through technical leadership.

Minimum Profile/ Track Record:

1. 7–10+ years of experience in cloud-native data engineering, with strong hands-on architecture, delivery, and production support experience on AWS & Snowflake.
2. Strong track record delivering complex data engineering initiatives independently, with the ability to operate across both build and run responsibilities.
3. Experience partnering with BI and business teams to design modelled datasets and enable self-service analytics.
4. Demonstrated technical leadership through design reviews, code reviews, mentoring, and troubleshooting guidance without formal team management responsibility.
5. Deep hands-on technical expertise, including:
6. Snowflake: schema design, Streams/Tasks, Stored Procedures, UDFs, RBAC-aware development, performance tuning, cost monitoring, Cortex AI, and Streamlit.
7. Airflow or similar data orchestration tools: DAG design, orchestration, scheduling, dependency management, retry patterns, and observability.
8. Python and SQL: pipeline scripting, transformation logic, data validation, and operational tooling.
9. ELT/ETL frameworks: Airbyte, Fivetran, and custom connector understanding or development.
10. AWS services: S3 (data lake structures and archival), Lambda, KMS, Transfer Family, CloudWatch, and SageMaker.
11. Demonstrated success delivering medallion architecture (Bronze/Silver/Gold) and enabling self-service data use cases.
12. Experience implementing automated data quality controls, remediation workflows, and data lineage-aware engineering practices across enterprise datasets.
13. Familiarity with machine learning or AI integration using platforms like AWS SageMaker.
14. Proven ability to troubleshoot complex data issues, lead root-cause analysis, and improve production stability through mechanisms rather than repeated manual intervention.
15. Track record of raising team engineering quality through reusable patterns, operational discipline, and technical coaching.